Welcome!

@DevOpsSummit Authors: Zakia Bouachraoui, Roger Strukhoff, Liz McMillan, Stackify Blog, Pat Romanski

Related Topics: @DevOpsSummit, Microservices Expo, Containers Expo Blog

@DevOpsSummit: Article

Sharding for Scale | @DevOpsSummit #BigData #DevOps #Microservices

Where you decide to shard for scalability impacts the complexity of the entire architecture

Sharding for Scale: In the App or in the Network?

Sharding has become a popular means of achieving scalability in application architectures in which read/write data separation is not only possible, but desirable to achieve new heights of concurrency. The premise is that by splitting up read and write duties, it is possible to get better overall performance at the cost of a slight delay in consistency. That is, it takes a bit of time to replicate changes initiated by a "write" to the read-only master database. It's eventually consistent, and it's generally considered an acceptable trade off when searching for higher and higher scalability.

While the most well-known cases of read/write separation and sharding are based on geography - east coast versus west coast, for example - there are other cases where localized sharding has also been put into play with great success. Generally these types of architectures base their sharding decisions on user names, splitting them up between databases based on statistical analysis of occurrences. This achieves greater scalability at the data layer by better distributing the rate at which writes (which are generally speaking a blocking action) occur, and thus achieving greater scale and concurrency for only a slight period of inconsistency.

The mechanism is, in theory, quite simple and is loosely based on an algorithmic principle taught in most computer science algorithm classes: hashing. Basically when a request comes in, the application looks at some piece of data - like the user name - and based on that data decides which of X databases to send it to. How that division is determined is not as relevant (to this discussion, anyway) as the action itself. It's like registration at an event or in college where you're split up based on the first letter of your last name. You remember, every one whose name starts with A-G go in this line, you others go over there, in those lines.

That's sharding. And it's most commonly implemented in the application, where the connection to a database is created and used to manage the data that is the lifeblood of every application and business today.

Now, I told you all that to share another approach to sharding; one that takes advantage of programmability in the network (data path programmability, to be precise).

Shard in the Network or the App?
In the first scenario, in which sharding occurs in the application, there's almost certainly (I'd be willing to bet real live money) a load balancing service in front. It's distributing requests to a pool (cluster) of application instances, each of which individually decides which database to talk to given the data available. If we insert some intelligence - some programmability - into that load balancing service, we move the sharding decision in front of the application.

Now when a request comes in, the load balancing service examines the data available and decides to which application instance the request should be sent. The data is likely the same - a user identity - but may be something more applicable to the service, say a product name or number extricated from the URL of a RESTful API fronting a microservice.

sharding - network versus application

Basically what you're doing is taking the block of code responsible for sharding from inside the application and moving it into the network.

In the diagram to your left (or at least it was on the left when I wrote this) illustrates. The reason the "apps" in the "network" illustration are colored is to highlight that each of them is dedicated to a specific database. The code - the app itself - is all the same. There's no difference except for the configuration that tells it "you are dedicated to database A-G".

This is starkly different from the "in the application" sharding example in which all instances are exactly the same, including the configuration, as each one may be at any time talking to any one of the databases, depending on the data received.

Now, I believe it's obvious (because I colored all the database connections) that when you shard in the application, the complexity of the network is pretty high, as well as the load on each database as it has to maintain connections with each and every application instance. Operational Axiom #2 tells us "as load increases, performance decreases" so it's likely we're seeing an impact on performance (in a negative way) from a sharding in the application architectural approach.

Conversely, the network complexity in the sharding in the network approach is fairly low (and straight forward) and actually simplifies the entire architecture. The load on the databases themselves remains lower because there is only one instance (or pool of instances) with which it needs to manage connections.

The negative of the "in the network" approach is that you have another component (service) that must be managed - that means application lifecycle management applies -  and there are likely separate configurations necessary for each of the pools (clusters) responsible for scaling out each application instance (because each pool only talks to one of the databases). But this negative also means that the code responsible for sharding is localized, it is itself a "network microservice" that is small and isolated, meaning it can be tweaked independently of the application code. That's a positive, especially if there might be a need to increase the level of sharding or change its core mechanism required to scale the application. That's one of the reasons microservices are growing more popular; the localization and isolation ensure the ability to change without disruptive impact on other services or applications.

Taking advantage of programmability in the network to achieve new levels of scalability while simplifying your architecture is another reason programmability in the network is an invaluable tool in your architectural toolbox.

More Stories By Lori MacVittie

Lori MacVittie is responsible for education and evangelism of application services available across F5’s entire product suite. Her role includes authorship of technical materials and participation in a number of community-based forums and industry standards organizations, among other efforts. MacVittie has extensive programming experience as an application architect, as well as network and systems development and administration expertise. Prior to joining F5, MacVittie was an award-winning Senior Technology Editor at Network Computing Magazine, where she conducted product research and evaluation focused on integration with application and network architectures, and authored articles on a variety of topics aimed at IT professionals. Her most recent area of focus included SOA-related products and architectures. She holds a B.S. in Information and Computing Science from the University of Wisconsin at Green Bay, and an M.S. in Computer Science from Nova Southeastern University.

@DevOpsSummit Stories
Docker and Kubernetes are key elements of modern cloud native deployment automations. After building your microservices, common practice is to create docker images and create YAML files to automate the deployment with Docker and Kubernetes. Writing these YAMLs, Dockerfile descriptors are really painful and error prone.Ballerina is a new cloud-native programing language which understands the architecture around it - the compiler is environment aware of microservices directly deployable into infrastructures like Docker and Kubernetes.
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throughout enterprises of all sizes.
As you know, enterprise IT conversation over the past year have often centered upon the open-source Kubernetes container orchestration system. In fact, Kubernetes has emerged as the key technology -- and even primary platform -- of cloud migrations for a wide variety of organizations. Kubernetes is critical to forward-looking enterprises that continue to push their IT infrastructures toward maximum functionality, scalability, and flexibility. As they do so, IT professionals are also embracing the reality of Serverless architectures, which are critical to developing and operating real-time applications and services. Serverless is particularly important as enterprises of all sizes develop and deploy Internet of Things (IoT) initiatives.
DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and controlling infrastructure. The rise of Site Reliability Engineering (SRE) is part of that redefinition of operations vs development roles in organizations.
When a company wants to develop an application, it must worry about many aspects: selecting the infrastructure, building the technical stack, defining the storage strategy, configuring networks, setting up monitoring and logging, and on top of that, the company needs to worry about high availability, flexibility, scalability, data processing, machine learning, etc. Going to the cloud infrastructure can help you solving these problems to a level, but what if we have a better way to do things. As a pioneer in serverless notion, Google Cloud offers a complete platform for each of those necessities letting users to just write code, send messages, assign jobs, build models, and gain insights without deploying a single machine. So cloud compute on its own is not enough, we need to think about all of the pieces we need to move architecture from the bottom, up towards the top of the stack. Wi...