Welcome!

@DevOpsSummit Authors: Zakia Bouachraoui, Elizabeth White, Liz McMillan, Pat Romanski, Yeshim Deniz

Related Topics: @DevOpsSummit

@DevOpsSummit: Blog Post

Keys to Production Readiness By @Papa_Fire | @DevOpsSummit [#DevOps]

With a bit of wizardry from Chef, anyone can create a pretty reliable replica of the production environment on demand

Data Quantity, Quality & Frequency: Keys to Production Readiness

Despite the fact that majority of developers firmly believe that "it worked on my laptop" is a poor excuse for production failures, most don't truly understand why it is virtually impossible to make your development environment representative of production.

When asked, the primary reason for the production/development difference everyone mentions is technology stack spec/configuration differences. While it's true, thanks to the black magic of Cloud (capitalization intended) with a bit of wizardry from Chef, anyone can create a pretty reliable replica of the production environment on demand. The actual main issue with reliable production mirroring is complex, but can be described in one word - data.

Quantity of Data
Most of the time developers don't have the full dataset to work with in their development environment. For example, testing an application against a 10 row table vs a 10,000,000 one will likely produce significantly different results. As a most basic example, N+1 problem will not be noticeable on the former, but will bring your production to its knees with the latter. Even if a developer decides to be diligent and attempt to re-create the full production data store into a personal development environment, the data will be out of sync as soon as the import is finished. With developer's luck, in accordance with Murphy's Law, it's the 10,000,001st record that will be the straw that breaks the back of your application in production.

Quality of Data
Every user is ... special. Production data helps to find edge cases that would not have been thought about even in the wildest developer [dream/nightmare]. If you're testing a form with a name field, chances are you'll test with "test test" or your own name. Same chances would suggest that your name is not "Geschwindigkeitsüberschreitung Füße." Make your peace with this one. You can never reproduce every production data scenario in your development environment. Ever. No matter how many data validators you write or how many test suites you create, you will not think about accounting for Wolfe+585, Sr. in the said name field.

Frequency of Data
This part actually reinforces the two points above - scale and predictability. Test results by a single developer, in an isolated development environment, will not provide an adequate representation of the same functionality with 10,000 concurrent connections. Similarly, even if you devise a load testing suite to test for predictable traffic patterns, you cannot account for the unknown. As a very real example, a hacker's brute force attack on your password protected admin section can, in best case scenario, lock up authentication for all your users. Worst case - your production is back on its knees.

And because (for all the reasons) development will never truly represent production, identifying and troubleshooting issues in production becomes critical for companies. But in order to do that, developers need to have access to said production (if you care about time-to-solution). Give it to them. The practice of instilling the culture that shares the responsibility for production readiness between operations and development teams has gone a long way over the past few years, whether you call it DevOps or not. Use that knowledge and experience to your advantage.

More Stories By Leon Fayer

Leon Fayer is Vice President at OmniTI, a provider of web infrastructures and applications for companies that require scalable, high performance, mission critical solutions. He possesses a proven background of both web application development and production deployment for complex systems and in his current role advises clients about critical aspects of project strategies and plans to help ensure project success. Leon can be contacted at [email protected]

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@DevOpsSummit Stories
@CloudEXPO and @ExpoDX, two of the most influential technology events in the world, have hosted hundreds of sponsors and exhibitors since our launch 10 years ago. @CloudEXPO and @ExpoDX New York and Silicon Valley provide a full year of face-to-face marketing opportunities for your company. Each sponsorship and exhibit package comes with pre and post-show marketing programs. By sponsoring and exhibiting in New York and Silicon Valley, you reach a full complement of decision makers and buyers in multiple vertical markets. Our delegate profiles can be located in our show prospectus.
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Data Theorem is a leading provider of modern application security. Its core mission is to analyze and secure any modern application anytime, anywhere. The Data Theorem Analyzer Engine continuously scans APIs and mobile applications in search of security flaws and data privacy gaps. Data Theorem products help organizations build safer applications that maximize data security and brand protection. The company has detected more than 300 million application eavesdropping incidents and currently secures more than 4,000 modern applications for its Enterprise customers around the world.
Rafay enables developers to automate the distribution, operations, cross-region scaling and lifecycle management of containerized microservices across public and private clouds, and service provider networks. Rafay's platform is built around foundational elements that together deliver an optimal abstraction layer across disparate infrastructure, making it easy for developers to scale and operate applications across any number of locations or regions. Consumed as a service, Rafay's platform eliminates the need to build an in-house platform or developing any specialized compute distribution capabilities. The platform significantly simplifies the deployment of containerized apps anywhere. Organizations can now achieve their desired levels of reliability, availability and performance with any combination of public cloud environments through a developer-friendly SaaS offering. From deploying ...
Kubernetes is a new and revolutionary open-sourced system for managing containers across multiple hosts in a cluster. Ansible is a simple IT automation tool for just about any requirement for reproducible environments. In his session at @DevOpsSummit at 18th Cloud Expo, Patrick Galbraith, a principal engineer at HPE, discussed how to build a fully functional Kubernetes cluster on a number of virtual machines or bare-metal hosts. Also included will be a brief demonstration of running a Galera MySQL cluster as a Kubernetes application.