Welcome!

@DevOpsSummit Authors: Yeshim Deniz, Pat Romanski, Elizabeth White, Liz McMillan, Zakia Bouachraoui

Related Topics: @DevOpsSummit, Microservices Expo, Containers Expo Blog, @CloudExpo, @DXWorldExpo

@DevOpsSummit: Blog Post

Data Demands of #DevOps | @DevOpsSummit #Docker #Microservices

The data as a service solution: new technologies deliver data at the speed of DevOps

Data Demands of DevOps | Part 2

In Part 1 of this article, we explored how data all too often becomes the critical bottleneck in software development, delaying projects and undermining the benefits of DevOps tools and processes. In Part 2, we'll look at the emerging category of Data as a Service solutions, which turn data from a drag to a driver.

Data as a Service
In order to deliver on the promise of DevOps and hit continuous release targets for even the largest, most complex and integrated applications, companies need solutions that provide the same flexibility for data as for code bases, the same automation and repeatability for data as for configurations. They need Data as a Service. DaaS solutions offer a single, integrated platform that serves up faithful copies of source data as easily as codes or configurations, and have sophisticated features to enable collaboration, project agility, and strong governance.

Provision and Branch
The most fundamental capability of any DaaS solution is the ability to deliver multiple copies of data promptly and with sophisticated automation. In order to deliver true DevOps capabilities, data standup should take no more time and effort than containerized code delivery or automated configuration: a few keystrokes and a few minutes.

DaaS solutions often deliver this capability through sophisticated block sharing and virtual files. Instead of moving data from system to system, such solutions keep a single repository of record, and then create virtual data instances by pointing to the correct set of blocks within the repository. That allows data provisioning to occur rapidly and automatically, and decouples time and effort from the size of the data set.

Of course, the ability to provision a full initial copy of the data is not enough. In order for developers and testers to achieve the flexibility they need, the DaaS solution must be able to branch the data as easily as code. A DevOps-ready DaaS solution will enable end users to spin off additional copies of the data they are working on, with whatever adjustments they have made, or from production as of many points in time. With this capability, developers and testers can keep code and data in sync, even as they pursue parallel projects, working on different versions or tests.

Bookmark and Share
DevOps isn't just about self-sufficiency. It's also about sophisticated collaboration. Without a DaaS solution, often data can be the bottleneck to efficient collaboration.

For example, suppose a QA staffer is reviewing a new piece of code for bugs. We're in a DevOps workplace, so this isn't dull, automated testing-perhaps it's an advanced scenario test, or a complicated A-B test setup. Let's say the tester finds a bug. He sends a note to the developer, with whom they've been working closely, outlining the bug. The developer uses the note and automated configuration tools, etc., to get her code into the same state, but she's unable to reproduce the bug. She lets the tester know she can't find it. The tester verifies the bug, and together, dev and test confirm that their code is in the same state. So the difference must be in the data.

With a legacy solution, there are two options. Either the developer would have to file an ops ticket to get her data into the right state-a process that could take days or weeks, and might fail repeatedly, depending on how the tester got his data-or she can take over the tester's data set. That will let her run down the bug quickly, in exchange for preventing the tester from doing any work at all. Either way, the process is broken. And if we imagine that the code being tested is part of a major push, or a daily feature cadence, or even a crucial patch to a bug running rampant in production, it becomes clear how disruptive this data management task can be.

With a DaaS solution, users can save data at any state, and share a copy of that data to any other user, with the same few clicks they would use to share code. Developers and testers don't contend for the same data. They can even skip the process of checking to see if the problem is data mismatch. Instead, they share data readily for every task, as easily and naturally as they share code or underlying hardware resources.

Refresh and Reset
Along with initial environment setup and collaborative debugging, test cycles are some of the most voracious consumers of data in the software development lifecycle. With legacy data delivery methods, testers often have to wait many hours for data to be provisioned to their test environment, in order to run a fifteen-minute test. This creates a very low ceiling on the number of test cycles available in a day, and can prevent the early detection and collaborative resolution of issues that are the keys to DevOps quality.

A DaaS solution can refresh an environment in minutes, accelerating the test cycle by a factor of ten. However, top-line solutions can do even more. A refresh would repopulate the test environment with data from production. But a strong DaaS solution can simply rewind the data state to that immediately before the test. This means that any changes to the data will need to be made only once. A test cycle characterized by long wait times for data and repeated set-up activities can be replaced by one where each test is followed by a rapid, effortless reset, and any data set-up is performed just once.

Governance
The DevOps movement drives cross-functional collaboration to meet the needs of both developers and operations staff. A good DaaS solution will serve both groups' stakeholders. The above capabilities have outlined some of the benefits that a DaaS solution can provide to Dev and Test teams, but the solution should meet Ops needs as well.

To do that, it needs a distinct set of permissions and management interfaces, so that Ops can carefully manage existing infrastructure and resources, even as Dev and Test staff spin up their own environments as-needed. A well-designed DaaS tool will not only save Ops time and effort by automating some of the dullest and most repetitive data-delivery tasks, it will also provide a full view of the team's resources for optimal management.

Conclusion
The growing acceptance of the DevOps philosophy, and the maturing ecosystem of associated tools, promises to revolutionize software development across industries, replacing outdated processes and models with collaborative teams that can truly deliver business value at digital speeds. Data as a Service solutions will be a key component of this revolution, enabling the full stack of environment creation, sharing, and management, leading to an overall doubling of project delivery.

More Stories By Louis Evans

Louis Evans is a Product Marketing Manager at Delphix. He is a subject-matter expert developing content, surveys and best practices pertinent to the DevOps community. Evans is also a speaker at DevOps focused industry events. He is a graduate of Harvard College, with a degree in Social Studies and Mathematics.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@DevOpsSummit Stories
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a member of the Society of Information Management (SIM) Atlanta Chapter. She received a Business and Economics degree with a minor in Computer Science from St. Andrews Presbyterian University (Laurinburg, North Carolina). She resides in metro-Atlanta (Georgia).
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. His expertise is in automating deployment, management, and problem resolution in these environments, allowing his teams to run large transactional applications with high availability and the speed the consumer demands.
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
Most DevOps journeys involve several phases of maturity. Research shows that the inflection point where organizations begin to see maximum value is when they implement tight integration deploying their code to their infrastructure. Success at this level is the last barrier to at-will deployment. Storage, for instance, is more capable than where we read and write data. In his session at @DevOpsSummit at 20th Cloud Expo, Josh Atwell, a Developer Advocate for NetApp, will discuss the role and value extensible storage infrastructure has in accelerating software development activities, improve code quality, reveal multiple deployment options through automated testing, and support continuous integration efforts. All this will be described using tools common in DevOps organizations.
Andi Mann, Chief Technology Advocate at Splunk, is an accomplished digital business executive with extensive global expertise as a strategist, technologist, innovator, marketer, and communicator. For over 30 years across five continents, he has built success with Fortune 500 corporations, vendors, governments, and as a leading research analyst and consultant.