Welcome!

DevOps Journal Authors: Carmen Gonzalez, Liz McMillan, Aater Suleman, Mike Kavis, Andrew Phillips

Related Topics: DevOps Journal, Java, Virtualization, Cloud Expo, Big Data Journal, SDN Journal

DevOps Journal: Article

The DevOps Database | Part 2

Applying Systems Thinking to Database Change Management

In my first post in this series, I discussed the underpinning principles of all DevOps patterns as eloquently stated by Gene Kim, author of "The Phoenix Project."  In this post I'd like to dig a little deeper into The First Way.  As a refresher:

The First Way: Systems Thinking - This Way stresses the performance of the entire system of value delivery.  Instead of becoming laser focused on the part of the process for which an individual or team is responsible, the individual or team works to understand the entire process from requirements generation to customer delivery.  The goal is to eliminate the delivery impediments that arise when a project transitions from one isolated silo to another.  Understanding the entire system allows business, development, and operations to work towards a common goal in a consistent manner.

When we started Datical, our first step was to perform extensive market validation.  We quickly learned that database schema management was going to be tough nut to crack. In the scores of conversations we had with people throughout the ALM spectrum, we learned that the process for managing and updating the database schema that supports an application was at best murky and at worst a black box.  So how do we elevate a process owned and understood by a few people in an organization to the level of visibility required to understand that process as part of a larger system?

What follows is a list of concepts pertaining to System Thinking that we've rallied around in providing our database change management solution Datical DB. Instead of the black box, database change management can become a transparent and flexible part of your value delivery system.

Start With Reality
When beginning a new project based on previous development, don't rely on a stack of SQL scripts on a file server or even in a source code control system. As you know, databases evolve over time and sometimes out of process changes happen.  When a database schema is modified to resolve an error condition or performance degradation, these alterations are usually handled in a support ticketing system.  You can never be certain that they were added to the stack of scripts used to build out a fresh environment.  In light of this, any database change management solution should start by generating a baseline from the working system: your production schema.  This ensures that design and development activity are taking into account not only those schema objects generated in Dev, but also those which originated in other stops in the system.

Don't Script! Model!
To keep this blog post short(er) I'll point you now to a blog post I wrote a few months ago on modelling vs. scripting.

Version Twice, Deploy Once
The first level of versioning any database change management tool should provide is versioning of the gold standard. Versioning the scripts or model that you use to build a fresh database instance for your application provides a ton of benefits.  You can track how your schema has evolved over time and across releases, you can tightly couple a schema definition to the version of the application it supports using the same branch/tag/merge workflow that you use for your application code, and you can quickly stand up a new database instance that you know is correct for any released version or experimental branch you are working in.  The second level of versioning takes place in each database instance.  Tracking the version of the schema deployed on a specific instance makes deployment and troubleshooting much easier.  Will this application build work with the schema on this instance?  What changes need to be applied to this database to catch it up to the latest version?  Were the right changes applied to this test database to validate a closed defect?  If you are tracking the individual changes applied in a database and the impetus for those changes, these questions can be answered quickly and easily.

Unify Your Modes of Delivery
We've found that a lot of uncertainty was generated when database updates were handled by several individuals using their own tools and methods to affect the required changes.  To remove this uncertainty, all database change must be executed using the same tools and processes across departments and individuals.   The tricky part: a continuous integration system has different requirements than a developer working iteratively or a DBA processing a batch of changes in a headless environment during a maintenance window.  In order to unify your database change process, the solution you use should be accessible to all of these individuals while maintaining the consistency of deployment activities.  That's why Datical DB provides a rich GUI experience for developers that helps them craft & deploy changes in dev environments; tight integrations with popular build and release automation frameworks that preserve the frameworks' workflows while providing Datical DB functionality; and a command line interface that allows users in headless environments to deploy changes in the exact same manner that the GUI or a 3rd-party integration would.  By unifying the modes of delivery you are constantly testing your release practices. By the time the production push rolls around, your system works.

More Stories By Pete Pickerill

Pete Pickerill is Vice President of Products and Co-founder of Datical. Pete is a software industry veteran who has built his career in Austin’s technology sector. Prior to co-founding Datical, he was employee number one at Phurnace Software and helped lead the company to a high profile acquisition by BMC Software, Inc. Pete has spent the majority of his career in successful startups and the companies that acquired them including Loop One (acquired by NeoPost Solutions), WholeSecurity (acquired by Symantec, Inc.) and Phurnace Software.

@DevOpsSummit Stories
We live in a time when seconds – even milliseconds – can have a dramatic economic impact on your company’s future. With technology being the primary conduit for consumer interaction, the user experience is at center stage. User experience will be a deciding factor in separating the future winners from the losers. By building more speed and agility into the application delivery process, DevOps promises great rewards. However, with this promise, also come significant ramifications for failure. In his session at DevOps Summit, Tom Lounibos, SOASTA CEO, will explore both the benefits and the som...
The 4th International DevOps Summit, co-located with16th International Cloud Expo – being held June 9-11, 2015, at the Javits Center in New York City, NY – announces that its Call for Papers is now open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Among the proven benefits, DevOps is correlated with 2...
Our expectations for buying all sorts of consumer goods has gone through a radical transformation we now take for granted. Why should we not expect this same level of service from IT businesses? We accept the status quo for how software delivery exists today but would reject it without hesitation if it were applied to pretty much any other online consumer experience. Take pizza delivery as an example. Fifteen years ago ordering a pizza meant trying to choose an item from a grease-stained menu somebody shoved under your door. You'd make a phone call and end up speaking to somebody who sounded ...
When an enterprise builds a hybrid IaaS cloud connecting its data center to one or more public clouds, security is often a major topic along with the other challenges involved. Security is closely intertwined with the networking choices made for the hybrid cloud. Traditional networking approaches for building a hybrid cloud try to kludge together the enterprise infrastructure with the public cloud. Consequently this approach requires risky, deep "surgery" including changes to firewalls, subnets and other modifications to the corporate security infrastructure. Connecting a public cloud to the ...
SYS-CON Events announced today that Grid Dynamics, the leading provider of scalable eCommerce technology solutions, will exhibit at DevOps Summit Silicon Valley, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Grid Dynamics is a leading provider of open, scalable, next-generation commerce technology solutions for Tier 1 retail. Grid Dynamics has in-depth expertise in commerce technologies, wide involvement in the open source community and a modern, global workforce. Great companies, partnered with Grid Dynamics, gain a sustainable business...
Docker offers a new, lightweight approach to application portability. Applications are shipped using a common container format and managed with a high-level API. Their processes run within isolated namespaces that abstract the operating environment independently of the distribution, versions, network setup, and other details of this environment. This "containerization" has often been nicknamed "the new virtualization." But containers are more than lightweight virtual machines. Beyond their smaller footprint, shorter boot times, and higher consolidation factors, they also bring a lot of new fea...
For better or worse, DevOps has gone mainstream. All doubt was removed when IBM and HP threw up their respective DevOps microsites. Where are we on the hype cycle? It's hard to say for sure but there's a feeling we're heading for the "Peak of Inflated Expectations". What does this mean for the Enterprise? Should they avoid DevOps? Definitely not. Should they be cautious though? Absolutely. The truth is that DevOps and the Enterprise are at best strange bedfellows. The movement has its roots in the tech community's elite. Open source projects and methodologies driven by the alumni of companies ...
SYS-CON Events announced today that Serena Software will exhibit at DevOps Summit Silicon Valley, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Serena Software supports DevOps and Continuous Delivery by providing application deployment automation and software release management solutions to replace slow and error-prone manual processes. 2,500 enterprises around the world trust Serena to help them develop and deploy better software.
We are all here because we are sold on the transformative promise of The Cloud. But what good is all of this ephemeral, on-demand infrastructure if your usage doesn't actually improve the agility and speed of your business? How must Operations adapt in order to avoid stifling your Cloud initiative? In his session at DevOps Summit, Damon Edwards, co-founder and managing partner of the DTO Solutions, will highlight the successful organizational, process, and tooling patterns of high-performing companies that have reshaped their Operations to enable the business to get full value from their Clo...
SYS-CON Events announced today that O'Reilly Media has been named “Media Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. O'Reilly Media spreads the knowledge of innovators through its books, online services, magazines, and conferences. Since 1978, O'Reilly Media has been a chronicler and catalyst of cutting-edge development, homing in on the technology trends that really matter and spurring their adoption by amplifying "faint signals" from the alpha geeks who are creating the future. An...
SYS-CON Events announced today that Gigaom Research has been named "Media Sponsor" of SYS-CON's 15th International Cloud Expo®, which will take place on November 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Ashar Baig, Research Director, Cloud, at Gigaom Research, will also lead a Power Panel on the topic "Choosing the Right Cloud Option." Gigaom Research provides timely, in-depth analysis of emerging technologies for individual and corporate subscribers. Gigaom Research's network of 200+ independent analysts provides new content daily that bridges the gap between break...
Sanjeev Sharma is the latest author to join DevOps Journal. Sanjeev is a solution architect and DevOps Worldwide lead with Rational Software, an IBM brand and the author of 'DevOps for Dummies.' DevOps Journal is focused on this critical enterprise IT topic in the world of cloud computing. SYS-CON Media CEO Carmen Gonzalez is founder and publisher of DevOps Journal, and Roger Strukhoff, long-time SYS-CON editor and the conference chair of Cloud Expo is the editor of the world's leading DevOps resource.
SYS-CON Events announced today that IBM is holding a Bluemix Developer Playground on November 5, 10:30 am to 5:30 pm at 15th Cloud Expo. 15th Cloud Expo, co-located with @ThingsExpo, Big Data Expo, and DevOps Summit is taking place Nov. 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. The labs, for developers of all levels, will highlight the ease of use of Bluemix, its services and functionality and provide short-term introductory projects that developers can complete between sessions. Developers will be able spend as much time as they want working on specific DevOps pro...
When you set off to build an app that will change the world, designing your system architecture to be reliable and scalable is important but the stark reality is that, for your MVP, you probably had a “need for speed” (of development). You didn’t know what all the axes were to scale your application, where your stress points would be, and what weird and wonderful ways your customers would use it down the road. In a world of zero-downtime services, landing the plane to figure it out is not an option. In his session at DevOps Summit, Andrew Miklas, CTO of PagerDuty, will share lessons learned ...
SYS-CON Events announced today that SOASTA, the leader in cloud and mobile testing, will exhibit at DevOps Summit Silicon Valley, which will take place November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. SOASTA is the leader in cloud testing. Its web and mobile test automation and monitoring solutions, CloudTest, TouchTest and mPulse, enable developers, QA professionals and IT operations teams to test and monitor users with unprecedented speed, scale, precision and visibility. The innovative product set streamlines test creation, automates provisioning and execution, ...
Founded in 1997, ActiveState is a global leader providing software application development and management solutions. The Company's products include: Stackato, a commercially supported Platform-as-a-Service (PaaS) that harnesses open source technologies such as Cloud Foundry and Docker; dynamic language distributions ActivePerl, ActivePython and ActiveTcl; and developer tools such as the popular Komodo Edit and Komodo IDE. Headquartered in Vancouver, Canada, ActiveState is trusted by customers and partners worldwide, across many industries including telecommunications, aerospace, software, fina...
SYS-CON Events announced today that ElasticBox is holding a Hackathon at DevOps Summit, November 6 from 12 pm -4 pm at the Santa Clara Convention Center in Santa Clara, CA. You can enter as an individual or team of up to 10 developers. A New Star Is Born Every Month! All completed ElasticBoxes will then be sent to a judging panel - 12 winners will be featured on the ElasticBox website in 2015. All entrants will receive five full enterprise licenses for one year + ElasticBox headphones + ElasticBox T-shirt. Winners can also choose to interview with ElasticBox to join one of the fastest growi...
SYS-CON Events announced today that Calm.io has been named “Bronze Sponsor” of DevOps Summit Silicon Valley, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Calm.io is a cloud orchestration platform for AWS, vCenter, OpenStack, or bare metal, that runs your CL tools puppet, Chef, shell, git, Jenkins, nagios, and will soon support New Relic and Docker. It can run hosted, or on premise and provides VM automation / expiry, self-service portals, audit, approvals, and budgeting.
Blue Box has closed a $10 million Series B financing. The round was led by a strategic investor and included participation from prior investors including Voyager Capital and Founders Collective, as well as the Blue Box executive team. This round follows a $4.3 million Series A closed in December of 2012 and led by Voyager Capital. In May of this year, the company announced general availability of its private cloud as a service offering, Blue Box Cloud. Since that release, the company has demonstrated market validation through customer adoption, positive reviews from industry analysts and k...
The speed of product development has increased massively in the past 10 years. At the same time our formal secure development and SDL methodologies have fallen behind. This forces product developers to choose between rapid release times and security. In his session at DevOps Summit, Michael Murray, Director of Cyber Security Consulting and Assessment at GE Healthcare, will examine the problems and present some solutions for moving security in to the DevOps lifecycle to ensure that we get fast AND secure.