Welcome!

DevOps Journal Authors: Roger Strukhoff, Trevor Parsons, Dmitriy Stepanov, Rex Morrow, Datical, Elizabeth White

Related Topics: DevOps Journal, Java, Linux, Cloud Expo, Big Data Journal, SDN Journal

DevOps Journal: Blog Feed Post

Chicken or Egg: Abstraction and Automation

But will abstraction necessarily lead to automation?

For many SDN and DevOps enthusiasts, the natural outcome of this wave of technological change is a highly-automated network that is well-orchestrated with surrounding systems and applications. One of the prevailing thoughts is that this level of automation is a well-formed abstraction layer. With the abstractions in place, the army of network engineers will be unencumbered by device configuration, and automation will ensure.

Or will it?

First off, let me say that abstraction is absolutely necessary. There is no doubt that networking will only advance if we can both remove unnecessary elements and simplify those that remain. We have to accomplish this in a way that is vendor (and ideally technology) agnostic. Abstraction is clearly the path forward.

But will abstraction necessarily lead to automation?
For the vast majority of network engineers who are designing and actively managing networks today, automation means writing shell or Perl scripts. The scripts themselves qualify as automation insofar as they remove keystrokes, but they basically execute the same serial logic that has dominated networking devices for decades.

When you want to make a switch or a router do something, you specify some configuration. Then you specify some other configuration. And again and again until you get through the litany of parameters that collectively make the device work. This workflow is so ingrained in our collective psyche that we inherently serialize the tasks required to make the network work.

There have actually been companies that have done a decent job of breaking the habit of serialized configuration. Juniper’s flagship operating system Junos moved to a more code-like representation of configuration, making no assumptions about the ordering of specific tasks. But our training runs deep, and even the Juniper guys will tell you that the biggest barrier to entry is familiarity with the UI.

We are addicted to our serialized behavior.
One of the side effects of highly-serialized configuration is that we tend to think extremely linearly and transactionally. There are a lot of network engineers for whom any kind of object-oriented approach is almost too foreign to really embrace. So when they try to automate tasks, they fall back into a sequencing of steps, repeated as many times as necessary. Automation without reuse is painfully difficult to propagate beyond only the most repetitive tasks.

And so we end up in a scenario where automation is basically synonymous with scripting, and where the value is largely applied only to the most frequently-executed tasks.

Where could automation take us?
If we think through where automation could take us, we ideally aim a little higher. Automation could mean the automated exchange of data between collaborative systems in support of some task. For instance, you might want your servers to communicate to your network so that when a new application is spun up (or a VM moves), you get corresponding policy changes, firewall or load balancer changes, and potentially network capacity allocation.

For most network engineers, the idea that infrastructure communicates and dynamically provides a service is science fiction. Our serialized mode of operation simply doesn’t support this kind of multilateral communication. Even if the abstractions remove some of the configuration complexity, the mental block is around sequencing.

If the current networking model has taught us anything, it should be that our network engineers are quite capable of managing tons of inputs and outputs. Now, whether that ought to be a requirement for the job is another question entirely. But as a group, network engineers are certainly capable of handling a lot of variables. That abstraction reduces these variables to the most meaningful is very interesting, but it wouldn’t seem that input management is the biggest bottleneck to automation.

The barrier to automation

Rather, the biggest barrier to automation is that workflow is so structured. First, it was the devices themselves that forced the structure. Then it was the processes (ITIL anyone?) that forced it. The end result is that we have built a discipline so dependent on structure that it actually impedes our own progress.

If we want to get to automation, we need to find a way to work around—or perhaps work within—this structure.

What we are really talking about is changing how we think about provisioning and managing a network. Why do you think there is so much angst when people talk about network engineers needing to learn to code? It’s because moving from a serialized set of steps to an object-oriented way of thinking about the problem is extremely difficult.

People aren’t pushing back because learning a new language is hard. Or at least they shouldn’t be. Look at any networking device configuration and tell me that you aren’t already a master coder. The biggest difference is that you you are using an interpreted language called Cisco CLI (or Junos CLI or whatever CLI).

What next?
What we need to do is bite off small (dare I say tiny) workflow elements, automate those, and then string them together into larger workflows. This implies a couple of things. First, we need to think less about discrete capabilities and more about how they exist within some broader workflow context. Second, we need to understand how these building blocks fit together. It’s the connecting of individual workflows that forms the basis for automation, and those connections highlight the pieces of information that flow across workflow boundaries.

More bluntly, the data that stitches together workflows ends up being the stuff that needs to be in an abstraction. It very well could be that getting the automation parts right will help us get to better abstractions.

Obviously we have to work the process from both ends – abstraction down, and workflow up. I don’t think it’s as simple as one or the other, which is why abstraction and automation might be a networking incarnation of the age-old chicken-and-egg question.

[Today’s fun fact: A parrot’s vocabulary is generally no more than twenty words. Who knew parrot’s and politicians had so much in common?]

The post Chicken or egg: Abstraction and automation appeared first on Plexxi.

More Stories By Michael Bushong

The best marketing efforts leverage deep technology understanding with a highly-approachable means of communicating. Plexxi's Vice President of Marketing Michael Bushong has acquired these skills having spent 12 years at Juniper Networks where he led product management, product strategy and product marketing organizations for Juniper's flagship operating system, Junos. Michael spent the last several years at Juniper leading their SDN efforts across both service provider and enterprise markets. Prior to Juniper, Michael spent time at database supplier Sybase, and ASIC design tool companies Synopsis and Magma Design Automation. Michael's undergraduate work at the University of California Berkeley in advanced fluid mechanics and heat transfer lend new meaning to the marketing phrase "This isn't rocket science."

@DevOpsSummit Stories
We had three quick questions for Mike Kail, and he had three quick answers. Mike is SVP of Infrastructure at Yahoo!, and formerly VP of IT Operations at Netflix. He'll be speaking at @DevOpsSummit about his experiences in integrating DevOps on a big scale in big-scale projects. Here's what we asked and what he said: DevOps Journal: You mention “eventual consistency” as a goal. Is there a timeframe? Mike Kail: It is really a strategy for successful transformation instead of a strict timeline DevOps Journal: Who has the steepest learning curve, "dev" "ops"? Mike: It is not about sides of the house, it is about one's ability to shift and evolve. Hire smart people who like getting uncomfortable and embrace rapid change DevOps Journal: How important is buy-in from top management? Mike: Very important. Thankfully I have, and provide, that.
Having just joined a large technology company with 20 years of history, it would be suicidal to believe that I can immediately move the entire organization to the DevOps mindset and model. For those not familiar with the term, “Eventual Consistency” is a model used in distributed computing to ensure high availability. In this context, it’s a model for replicating best practices and automation across IT teams and business units. The logical place to start with automation is the on-boarding of a new employee. That process should be as seamless and streamlined as possible, with a pristine source of truth. The goal is to populate a list of attributes and replicate them out to the various systems, and that’s applicable to either a new employee or an existing one who changes roles. Core infrastructure deployment is also at the base of the DevOps stack. Automate the provisioning of compute, network, and storage, and provide continuous insight into the utilization.
The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential. The DevOps Summit at Cloud Expo--to be held November 4-6 at the Santa Clara Convention Center in the heart of Silicon Valley--will expand the DevOps community, enable a wide sharing of knowledge, and educate delegates and technology providers alike. Recent research has shown that DevOps dramatically reduces development time, the amount of enterprise IT professionals put out fires, and support time generally. Time spent on infrastructure development is significantly increased, and DevOps practitioners report more software releases and higher quality.
DevOps Summit at Cloud Expo Silicon Valley announced today a limited time free "Expo Plus" registration option. On site registration price of $1,95 will be set to 'free' for delegates who register during this offer perios. To take advantage of this opportunity, attendees can use the coupon code, and secure their registration to attend all keynotes, DevOps Summit sessions at Cloud Expo, expo floor, and SYS-CON.tv power panels. Registration page is located at the DevOps Summit site.
Software is eating the world. Companies that were not previously in the technology space now find themselves competing with Google and Amazon on speed of innovation. As the innovation cycle accelerates, companies must embrace rapid and constant change to both applications and their infrastructure, and find a way to deliver speed and agility of development without sacrificing reliability or efficiency of operations. In her keynote DevOps Summit, Victoria Livschitz, CEO of Qubell, will discuss how IT organizations can automate just-in-time assembly of application environments – each built for a specific purpose with the right infrastructure, components, service, data and tools – and deliver this automation to developers as a self-service. Victoria’s keynote will include remarks by Kira Makagon, EVP of Innovation at RingCentral, and Ratnakar Lavu, EVP of Digital Technology at Kohl’s.
DevOps Summit at Cloud Expo Silicon Valley announced today a limited time free "Expo Plus" registration option through September. On site registration price of $1,95 will be set to 'free' for delegates who register during special offer. To take advantage of this opportunity, attendees can use the coupon code, and secure their registration to attend all keynotes, DevOps Summit sessions at Cloud Expo, expo floor, and SYS-CON.tv power panels. Registration page is located at the DevOps Summit site. Your DevOps Summit registration will also allow access to @ThingsExpo sessions and exhibits. Register For DevOps Summit "FREE" (limited time) ▸ Here
Despite the fact that majority of developers firmly believe that “it worked on my laptop” is a poor excuse for production failures, most don’t truly understand why it is virtually impossible to make your development environment representative of production. When asked, the primary reason for the production/development difference everyone mentions is technology stack spec/configuration differences. While it’s true, thanks to the black magic of Cloud (capitalization intended) with a bit of wizardry from Chef, anyone can create a pretty reliable replica of the production environment on demand. The actual main issue with reliable production mirroring is complex, but can be described in one word – data.
SYS-CON Events announced today that AppDynamics will exhibit at DevOps Summit Silicon Valley, which will take place November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Digital businesses like yours need a way to turn data into actual results. AppDynamics is ushering in the next digital age – the age of the software-defined business. AppDynamics’ mission is to deliver true application intelligence that helps your software-defined business run faster, leaner, and more efficient. You get the visibility and control you need to identify problems, find the root cause, and instantly connect the dots to get applications – and user transactions – on track and running flawlessly. AppDynamics provides the certainty that your most complex, business-critical applications are performing at the highest level and that the data and information generated by these applications can be harnessed for ongoing business advantage and impact.

BOULDER, Colo., Sept. 24, 2014 /PRNewswire/ -- VictorOps, the leading collaboration and incident management platform for DevOps teams, is hosting a webinar that will discuss how to take full advantage of your project post-mortems with or without a template.

DevOps Journal: Cloud, Big Data, and the IoT all carry disruption within enterprise IT. The same goes with DevOps. Which of these is the major disruptor, in your opinion? Andi Mann: It may well be cloud, because it fundamentally enables all the rest. Cloud scale is why we are now considering Big Data; cloud connectivity is a key enabler of IoT; cloud agility has enabled DevOps to take hold. But in the end, the cloud is “just” a platform, while the results of DevOps speak for themselves--like an average 20+% improvement in areas like revenue, new products, time to market, quality improvement, and more . DevOps Journal: Cloud has been gaining traction for the longest time in this group, too. Andi: Yes, it is still early days for DevOps, with a minority yet to adopt DevOps, so its biggest impact is yet to be seen. This is an incredible opportunity to radically accelerate the pace of change, which in turn will upend all the other disruptions. This is why I think DevOps...
These days, implementing automatic deployment for .NET web projects is easier than ever. Drastic improvements started in Visual Studio 2010 when basic deployment strategies and tools were incorporated into VS itself. Yet, documentation was quite poor at that time, so you had to scour the Internet to find good tutorials in blogs or conference videos. Things have been constantly improving since then; now, we have even more functionality available out-of-the-box and documentation provided in a way that allows you to get started from zero understanding of the process to working deployment in less than two days. Some good places to start are this tutorial for an end-to-end guide on configuration and MSDN’s overview.
Azul Systems Inc. (Azul), the award-winning leader in Java runtime solutions, today announced that its OpenJDK-based Zulu 8 offering is now freely available on Docker. Zulu 8 is a 100% open source, fully tested, compatibility verified, and trusted binary distribution of the OpenJDK 8 platform. Azul has also made Zulu versions compliant with earlier Java SE 7 and Java SE 6 standards available on Docker in the same format.
Founded in 1997, ActiveState is a global leader providing software application development and management solutions. The Company's products include: Stackato, a commercially supported Platform-as-a-Service (PaaS) that harnesses open source technologies such as Cloud Foundry and Docker; dynamic language distributions ActivePerl, ActivePython and ActiveTcl; and developer tools such as the popular Komodo Edit and Komodo IDE. Headquartered in Vancouver, Canada, ActiveState is trusted by customers and partners worldwide, across many industries including telecommunications, aerospace, software, financial services and CPG. ActiveState is proven for the enterprise: More than two million developers and 97% of Fortune 1000 companies use ActiveState's solutions to develop, distribute, and manage their software applications. Global customers like Bank of America, CA, Cisco, HP, Lockheed Martin and Siemens rely on ActiveState for faster development, ensuring IT governance and compliance, and accele...
DevOps Summit at Cloud Expo Silicon Valley announced today a limited time free "Expo Plus" registration option. On site registration price of $1,95 will be set to 'free' for delegates who register during this offer perios. To take advantage of this opportunity, attendees can use the coupon code, and secure their registration to attend all keynotes, DevOps Summit sessions at Cloud Expo, expo floor, and SYS-CON.tv power panels. Registration page is located at the DevOps Summit site.
Leading provider of Continuous Delivery and DevOps software XebiaLabs today announced enhanced integration between Puppet and XebiaLabs' XL Deploy, the deployment automation solution that supports DevOps and Continuous Delivery teams. XL Deploy in combination with Puppet means one seamless automation process to deploy your apps.
PagerDuty, the leader in operations performance management, announced the public release of its Advanced Analytics tools, which provide insights IT teams can use to improve team and system performance. Leveraging PagerDuty’s robust data on key operational metrics like incident frequency and time to respond and resolve, companies can now drive even faster incident resolution. The new capabilities further expand PagerDuty’s operations performance platform by giving managers the ability to analyze and improve key drivers of uptime.
In today's application economy, enterprise organizations realize that it's their applications that are the heart and soul of their business. If their application users have a bad experience, their revenue and reputation are at stake. In his session at 15th Cloud Expo, Anand Akela, Senior Director of Product Marketing for Application Performance Management at CA Technologies, will discuss how a user-centric Application Performance Management solution can help inspire your users with every application transaction.
SYS-CON Events announced today that Serena Software will exhibit at DevOps Summit Silicon Valley, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Serena Software supports DevOps and Continuous Delivery by providing application deployment automation and software release management solutions to replace slow and error-prone manual processes. 2,500 enterprises around the world trust Serena to help them develop and deploy better software.
Qubell, an innovator in application deployment and configuration management, empowers online companies to do what they have never been able to do before: put into consumers' hands innovative new features and services, almost as fast as they can conceive them, without sacrificing control, reliability or uptime. Qubell emerged from stealth in the summer of 2013 (see related press release) and announced that Kohl's completed its initial implementation (see press release). Founded by pioneers in enterprise cloud applications and services, Qubell has its headquarters in Menlo Park, Calif. For more information, visit qubell.com.
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With “smart” appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user’s habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can’t be addressed without the kinds of agile software development and infrastructure approaches pioneered by the DevOps movement.