Welcome!

@DevOpsSummit Authors: Yeshim Deniz, Flint Brenton, Liz McMillan, Elizabeth White, Pat Romanski

Related Topics: @DevOpsSummit

@DevOpsSummit: Blog Post

Keys to Production Readiness By @Papa_Fire | @DevOpsSummit [#DevOps]

With a bit of wizardry from Chef, anyone can create a pretty reliable replica of the production environment on demand

Data Quantity, Quality & Frequency: Keys to Production Readiness

Despite the fact that majority of developers firmly believe that "it worked on my laptop" is a poor excuse for production failures, most don't truly understand why it is virtually impossible to make your development environment representative of production.

When asked, the primary reason for the production/development difference everyone mentions is technology stack spec/configuration differences. While it's true, thanks to the black magic of Cloud (capitalization intended) with a bit of wizardry from Chef, anyone can create a pretty reliable replica of the production environment on demand. The actual main issue with reliable production mirroring is complex, but can be described in one word - data.

Quantity of Data
Most of the time developers don't have the full dataset to work with in their development environment. For example, testing an application against a 10 row table vs a 10,000,000 one will likely produce significantly different results. As a most basic example, N+1 problem will not be noticeable on the former, but will bring your production to its knees with the latter. Even if a developer decides to be diligent and attempt to re-create the full production data store into a personal development environment, the data will be out of sync as soon as the import is finished. With developer's luck, in accordance with Murphy's Law, it's the 10,000,001st record that will be the straw that breaks the back of your application in production.

Quality of Data
Every user is ... special. Production data helps to find edge cases that would not have been thought about even in the wildest developer [dream/nightmare]. If you're testing a form with a name field, chances are you'll test with "test test" or your own name. Same chances would suggest that your name is not "Geschwindigkeitsüberschreitung Füße." Make your peace with this one. You can never reproduce every production data scenario in your development environment. Ever. No matter how many data validators you write or how many test suites you create, you will not think about accounting for Wolfe+585, Sr. in the said name field.

Frequency of Data
This part actually reinforces the two points above - scale and predictability. Test results by a single developer, in an isolated development environment, will not provide an adequate representation of the same functionality with 10,000 concurrent connections. Similarly, even if you devise a load testing suite to test for predictable traffic patterns, you cannot account for the unknown. As a very real example, a hacker's brute force attack on your password protected admin section can, in best case scenario, lock up authentication for all your users. Worst case - your production is back on its knees.

And because (for all the reasons) development will never truly represent production, identifying and troubleshooting issues in production becomes critical for companies. But in order to do that, developers need to have access to said production (if you care about time-to-solution). Give it to them. The practice of instilling the culture that shares the responsibility for production readiness between operations and development teams has gone a long way over the past few years, whether you call it DevOps or not. Use that knowledge and experience to your advantage.

More Stories By Leon Fayer

Leon Fayer is Vice President at OmniTI, a provider of web infrastructures and applications for companies that require scalable, high performance, mission critical solutions. He possesses a proven background of both web application development and production deployment for complex systems and in his current role advises clients about critical aspects of project strategies and plans to help ensure project success. Leon can be contacted at [email protected]

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@DevOpsSummit Stories
JETRO showcased Japan Digital Transformation Pavilion at SYS-CON's 21st International Cloud Expo® at the Santa Clara Convention Center in Santa Clara, CA. The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get tailored market studies; and more.
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a member of the Society of Information Management (SIM) Atlanta Chapter. She received a Business and Economics degree with a minor in Computer Science from St. Andrews Presbyterian University (Laurinburg, North Carolina). She resides in metro-Atlanta (Georgia).
In this presentation, you will learn first hand what works and what doesn't while architecting and deploying OpenStack. Some of the topics will include:- best practices for creating repeatable deployments of OpenStack- multi-site considerations- how to customize OpenStack to integrate with your existing systems and security best practices.
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
It is ironic, but perhaps not unexpected, that many organizations who want the benefits of using an Agile approach to deliver software use a waterfall approach to adopting Agile practices: they form plans, they set milestones, and they measure progress by how many teams they have engaged. Old habits die hard, but like most waterfall software projects, most waterfall-style Agile adoption efforts fail to produce the results desired. The problem is that to get the results they want, they have to change their culture and cultures are very hard to change. To paraphrase Peter Drucker, "culture eats Agile for breakfast." Successful approaches are opportunistic and leverage the power of self-organization to achieve lasting change.