Welcome!

@DevOpsSummit Authors: Elizabeth White, Yeshim Deniz, Pat Romanski, Aruna Ravichandran, Liz McMillan

Related Topics: @CloudExpo, Microservices Expo, @BigDataExpo, @ThingsExpo, @DevOpsSummit

@CloudExpo: Article

Data Lake and APIs | @CloudExpo #API #AI #ML #DX #CloudNative #DataLake

In this article we take you through the connected world of APIs and how you can reap maximum value

The Omnipresent World of APIs

These days, APIs have become an integral part of the digital transformation journey for all enterprises. Every digital innovation story is connected to APIs . But have you ever pondered over to know what are the source of these APIs?

Let me explain - APIs sources can be varied, internal or external, solving different purposes, but mostly categorized into the following two categories:

  • Data lakes is a term used to represent disconnected but relevant data that are used by various business units within an enterprise.  APIs are created as the easy access points for these siloed data lakes.
  • These days, organizations put in  conscious efforts in designing meaningful and pragmatic APIs for all the apps that get developed internally.  API-First or API Driven Development, where APIs are the first app artifacts to be generated, are adapted by enterprises as a part of their app development efforts.  In modern app development, as much as the apps share APIs generated internally, they also have to deal with a plethora of external APIs.  These external API are fed from SaaS services (eg. Google Calendar), API marketplaces (mashape) and integration clouds(e.g. Zapier).

With new technologies like IoT gaining traction,  there is going to be a proliferation of  APIs very soon.

API Management tools
How do enterprises manage these flurry of APIs that get generated?  How do they leverage these APIs to make them as easily accessible touch points for both internal and external stake holders, complete with security, subscription, plans, API rate limiting etc?  The answer is enterprises use API Management platforms/tools for the said purpose.

API management platforms help enterprises manage all aspects of APIs as defined above, ease the process of API access to all stakeholders and in general address all the management aspects of the entire API lifecycle.

Typical API sharing and consumption process
API Management tools typically take as input, all the APIs at its disposable and attaches multiple attributes like security, rate limiting, subscription plans etc. to each of those APIs.  These tools also provide something called as an API or a Developer portal.  This portal is a one point portal for searching and subscribing to APIs.

The typical process looks like as is shown in the following set of images for a fictitious company called Movi.  We have also used screens from WaveMaker low code platform for some illustrations.

This is a templatized approach followed by organizations who try to share their APIs with external (as well as internal) stakeholders.  But is this  process as shown in step 4, effective? Do organizations get the RoI from their investment on APIs quickly?

Loopholes with the current API management setup - Low RoI
The fact is that APIs, at the end of the day, are entities consumed inside an app.  With that in mind, consider some of the glaring issues with the current API consumption model, as listed below:

  • There exists competitive barriers for rapid proliferation of these APIs by enabling a much larger audience to build apps using those APIs.  Competitive barriers are because of  of a big super set of users with great ideas but no app coding skills than there are users with app coding skills.
  • Turnaround time from when APIs are published to when the API is really consumed in an app is longer due to the complexities surrounding the API documentation and consumption related technical intricacies.

What we need is to :

  • Remove the technical friction for anyone wanting to start building apps
  • Make it easier and faster for anyone to build apps to your APIs to increase the RoI faster

Let's fix it.  Remove the friction between app building and API access

The issues seem to have a very easy solution but probably needs the enterprise move up in the value chain of becoming a "Digital Enterprise".  The answer lies in adopting a low code development platform that can work out wonders in solving these issues.  Let me explain these in simple visual steps below:

1) Let's start with a simple change to the existing process as shown in figure-1, by adding an app creation step.  This will help in users to not just subscribe to an API plan but also start working on app instantly.  More details in the next steps will make this simple addition work wonders.

2) Start the developer portal registration exercise as always...

3) However during the registration process, we make sure to add an instant app creation step to facilitate users.  The button click will lead the users to a low code platform for visual app creation.  It's all in the mind!  This simple change in the registration workflow allows the user to instantly start creating an app and the exercise is not just limited to API access.

4) Once the users clicks the button, they get logged into a low code platform.  Note we have used screen mocks from WaveMaker low code platform.  But the user is not left in lurch.  He is not left to the mercy of documentation and forums to start integrating the APIs into the app.  A templatized app pre exists inside the platform with all the relevant artifacts for consuming the APIs and this  can help users to significantly speed up app creation.

5)  Inside the Movi app, pre-integrate with the most commonly used APIs in the application.  This will take away all the complexities associated with the API, like documentation, parameters, headers etc. and lets the user focus on just app business logic and end user experience

6) A well defined templated app inside a low code platform would also provide pre-existing drag-n-drop components like search widgets that have pre-built integrations to the search APIs.  This is value addition beyond just simple API integration.  Now we are talking about UI components that will let you build the customer user experience in front of your eyes using in-built WYSIWYG screens.  This is powerful, since you are not focussing on technology anymore, but business imperatives.

7) Low code platforms doesn't stop there.  It will not just let you build your app easily, but also test them take them to delivery fast.  It covers the entire app delivery life cycle.  Check out the 1-click testing and deploy features highlighted in the below figure.

If you noticed in the above article, it was the low code platform which is the unsung hero solving the bigger problem of API RoI.  Low code platforms in one swoop through its easy drag and drop based visual coding and app template based approach put a clean facade over technical complexities associated with APIs and coding and app.  Low code platforms lets you make an app from an idea in a matter of days instead of weeks, months and in some cases more than a year to create an app.

Benefits of the new API consumption model

The below picture captures all the benefits of the new model succinctly.

Conclusion
In the modern digital world, it's imperative for organizations to move up in the digital value chain.  Investing in a low code platform is one good way to do that.  What we discussed today is one such value proposition for companies with existing investment in API infrastructure.  But low code platform based solution can be successfully  applied for various other scenarios like faster response to the needs of the customer, enabling 3rd party digital channel stakeholders, and even empowering employees within a company.

More Stories By Karthick Viswanathan

Karthick heads the Products, Marketing and Evangelism efforts for WaveMaker. As one of the original product managers at WaveMaker, he led the technology and design for WaveMakerOnline, WaveMaker's PaaS offering. He also established the Customer Success practice for WaveMakerOnline. Prior to WaveMaker, Karthick occupied various roles in engineering and product management at PegaSystems, IBM and Oracle. He can discuss in-depth, code-level details and just as easily shift focus to business and go-to-market strategies. He was one of the seeders for the product management teams for PegaSystems in India and also a key technology contributor behind Oracle Warehouse Builder features. He writes and blogs about modern app development methodologies, including Rapid Application Development, digital transformation, and the API economy.

@DevOpsSummit Stories
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their associated CPUs, memory storage and network) into one or more large servers capable of handling the biggest Big Data problems and most unpredictable workloads.
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant that knows everything and can respond to your emotions and verbal commands!
Infoblox delivers Actionable Network Intelligence to enterprise, government, and service provider customers around the world. They are the industry leader in DNS, DHCP, and IP address management, the category known as DDI. We empower thousands of organizations to control and secure their networks from the core-enabling them to increase efficiency and visibility, improve customer service, and meet compliance requirements.
In his session at 21st Cloud Expo, Michael Burley, a Senior Business Development Executive in IT Services at NetApp, will describe how NetApp designed a three-year program of work to migrate 25PB of a major telco's enterprise data to a new STaaS platform, and then secured a long-term contract to manage and operate the platform. This significant program blended the best of NetApp’s solutions and services capabilities to enable this telco’s successful adoption of private cloud storage and launching of virtual storage services to its enterprise market.
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lavi, a Nutanix DevOps Solution Architect, explored the ways that Nutanix technologies empower teams to react faster than ever before and connect teams in ways that were either too complex or simply impossible with traditional infrastructures.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Cloud Expo, Inc. has announced today that Andi Mann and Aruna Ravichandran have been named Co-Chairs of @DevOpsSummit at Cloud Expo Silicon Valley which will take place Oct. 31-Nov. 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. "DevOps is at the intersection of technology and business-optimizing tools, organizations and processes to bring measurable improvements in productivity and profitability," said Aruna Ravichandran, vice president, DevOps product and solutions marketing, CA Technologies. "It's this results-driven combination of technology and business that makes me so passionate about DevOps and its future in the industry. I am truly honored to take on this co-chair role, and look forward to working with the DevOps Summit team at Cloud Expo and attendees to advance DevOps."
Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, will discuss how given the magnitude of today's application ecosystem, tweaking existing software to stitch various components together leads to sub-optimal solutions. This definitely deserves a re-think, and paves the way for a new breed of lightweight application servers that are micro-services and DevOps ready!
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
The dynamic nature of the cloud means that change is a constant when it comes to modern cloud-based infrastructure. Delivering modern applications to end users, therefore, is a constantly shifting challenge. Delivery automation helps IT Ops teams ensure that apps are providing an optimal end user experience over hybrid-cloud and multi-cloud environments, no matter what the current state of the infrastructure is. To employ a delivery automation strategy that reflects your business rules, making real-time decisions based on a combination of real user monitoring, synthetic testing, APM, NGINX / local load balancers, and other data sources, is critical.
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp empowers global organizations to unleash the full potential of their data to expand customer touchpoints, foster greater innovation and optimize their operations.
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. That means serverless is also changing the way we leverage public clouds. Truth-be-told, many enterprise IT shops were so happy to get out of the management of physical servers within a data center that many limitations of the existing public IaaS clouds were forgiven. However, now that we’ve lived a few years with public IaaS clouds, developers and CloudOps pros are giving a huge thumbs down to the ...
Enterprises are adopting Kubernetes to accelerate the development and the delivery of cloud-native applications. However, sharing a Kubernetes cluster between members of the same team can be challenging. And, sharing clusters across multiple teams is even harder. Kubernetes offers several constructs to help implement segmentation and isolation. However, these primitives can be complex to understand and apply. As a result, it’s becoming common for enterprises to end up with several clusters. This leads to a waste of cloud resources and increased operational overhead.
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere delivers a more modern architectural approach to storage that doesn't require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbuilding of data centers to house increasing amounts of storage infrastructure.
Containers are rapidly finding their way into enterprise data centers, but change is difficult. How do enterprises transform their architecture with technologies like containers without losing the reliable components of their current solutions? In his session at @DevOpsSummit at 21st Cloud Expo, Tony Campbell, Director, Educational Services at CoreOS, will explore the challenges organizations are facing today as they move to containers and go over how Kubernetes applications can deploy with legacy components, and also go over automated capabilities provided by operators to auto-update Kubernetes with zero downtime for current and secure deployments.
SYS-CON Events announced today that Avere Systems, a leading provider of hybrid cloud enablement solutions, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere Systems was created by file systems experts determined to reinvent storage by changing the way enterprises thought about and bought storage resources. With decades of experience behind the company’s founders, Avere got its start in 2008 with a mission to use fast, flash-based storage in the most efficient, effective manner possible. What the team had discovered was a technology that optimized storage resources and reduced dependencies on sprawling storage installations. Launched as the Avere OS, this advanced file system not only boosted performance within standard, on-premises, network-attached storage systems but ...
Today most companies are adopting or evaluating container technology - Docker in particular - to speed up application deployment, drive down cost, ease management and make application delivery more flexible overall. As with most new architectures, this dream takes significant work to become a reality. Even when you do get your application componentized enough and packaged properly, there are still challenges for DevOps teams to making the shift to continuous delivery and achieving that reduction in cost and increase in speed. Sometimes in order to reduce complexity teams compromise features or change requirements
The next XaaS is CICDaaS. Why? Because CICD saves developers a huge amount of time. CD is an especially great option for projects that require multiple and frequent contributions to be integrated. But… securing CICD best practices is an emerging, essential, yet little understood practice for DevOps teams and their Cloud Service Providers. The only way to get CICD to work in a highly secure environment takes collaboration, patience and persistence. Building CICD in the cloud requires rigorous architectural and coordination work to minimize the volatility of the cloud environment and leverage the security features of the cloud to the benefit of the CICD pipeline.
SYS-CON Events announced today that SkyScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SkyScale is a world-class provider of cloud-based, ultra-fast multi-GPU hardware platforms for lease to customers desiring the fastest performance available as a service anywhere in the world. SkyScale builds, configures, and manages dedicated systems strategically located in maximum-security facilities, allowing customers to focus on results while minimizing capital equipment investment.
As you move to the cloud, your network should be efficient, secure, and easy to manage. An enterprise adopting a hybrid or public cloud needs systems and tools that provide: Agility: ability to deliver applications and services faster, even in complex hybrid environments Easier manageability: enable reliable connectivity with complete oversight as the data center network evolves Greater efficiency: eliminate wasted effort while reducing errors and optimize asset utilization Security: implement always-vigilant DNS security
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, will discuss how by using new techniques such as feature flagging, rollouts, and traffic splitting, experimentation is no longer just the future for marketing teams, it’s quickly becoming an essential practice for high-performing development teams as well.
DevOps at Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.