Welcome!

@DevOpsSummit Authors: Zakia Bouachraoui, Stackify Blog, Jnan Dash, Liz McMillan, Janakiram MSV

Related Topics: @DevOpsSummit, Linux Containers, @CloudExpo, @DXWorldExpo

@DevOpsSummit: Blog Post

Keepalived and HAProxy in AWS | @DevOpsSummit [#DevOps]

We're going to explore high availability and load balancing using Keepalived and HAProxy

Keepalived and HAProxy in AWS: An Exploratory Guide

This article originally published on the Logentries Blog.

We're going to explore high availability and load balancing using Keepalived and HAProxy.

Keepalived is a routing software designed to provide simple and robust facilities for load balancing and high-availability to Linux systems and Linux-based infrastructures.

HAProxy is an open source load balancer/reverse proxy generally used for load balancing web services, but also has the functionality to load balance TCP traffic.

Together, Keepalived and HAProxy provide some unique benefits

Specifically, they provide a low cost solution for high availability compared to proprietary hardware-based load balancers.

Keepalived and Haproxy in AWS

Keepalived uses the VRRP protocol to detect when HAProxy is down and fails with minimal downtime.

The following documentation is focused on setting up Keepalived in AWS (or potentially any cloud) where multicast is not supported, and you're using a debian-based OS.

Keepalived and HAProxy Setup

First, download the latest version of Keepalived.

(http://www.keepalived.org/software/keepalived-1.2.13.tar.gz)

You will need this version since the previous version in your repository is probably too old to support VRRP Unicast.

wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz
tar xf keepalived-1.2.13.tar.gz
cd keepalived-1.2.13/
./configure
make && make install

cp keepalived/etc/init.d/keepalived.init /etc/init.d/keepalived

mkdir /etc/keepalived/

Keepalived and HAProxy Architecture

We're going to assume two load balancer servers are running Keepalived and HAProxy, where one is the master, the other is a failover, and we have multiple web servers.

Keepalived and HAProxy AWS Architecture

In this scenario, we have an elastic IP on ServerA while ServerB is set as a passive server not being hit by the public internet. We also have five web servers running our app.

By default ServerA is set as MASTER state and ServerB is set as BACKUP, which means on startup the servers know their state, and also means on a failover/fail back scenario both servers know their role.

In a failure scenario where ServerA becomes unresponsive (i.e HAProxy process hangs, server dies etc.) ServerB will be unable to contact serverA. Therefore, ServerA enters a FAULT mode and ServerB becomes master. When ServerB becomes master it runs a script which will basically disassociate the Elastic IP from ServerA and assign it to ServerB. ServerB is now the only active HAProxy server and handles requests to the app servers.

Configs

A few online references seem to have incorrect configs for setting up Keepalived in a Unicast environment, however the following instructions seem to work correctly.

We need to set up our master and slave configs for Keepalived slightly different as they both have different roles to play in failover. The vrrp_script is used to verify the other party is functioning correctly. We also have in our notify_master section, /etc/keepalived/master.sh, a script which assigns the Elastic IP to the master in a case of failure or recovery.

Server A (master) (/etc/keepalived/keepalived.conf)

ServerB - BackupStarting Keepalived

Keepalived startup messages are in /var/log/syslog, to see if your process started up correctly you should see an output similar to below:

ServerA - Master

There are many ways to test connectivity, tshark, tcpdump etc, here are some sample commands:Verify Connectivity

tcpdump "ip proto 112" 
tshark -f "vrrp"

The reason we need to verify connectivity with either of the commands above is because Keepalived uses multicast, which is not supported in AWS. The commands verify our config is correctly sending via Unicast.


Verify Connectivity Keepalived Haproxy AWS tshark

Testing Failover

Assuming you have your failover.sh script setup correctly, on stopping HAProxy on ServerA, you should see it entering a FAULT state and see ServerB entering a MASTER state. When ServerB enters a master state it will run a script and reassign the Elastic IP to ServerB.

More Stories By Trevor Parsons

Trevor Parsons is Chief Scientist and Co-founder of Logentries. Trevor has over 10 years experience in enterprise software and, in particular, has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland.

@DevOpsSummit Stories
Cloud-Native thinking and Serverless Computing are now the norm in financial services, manufacturing, telco, healthcare, transportation, energy, media, entertainment, retail and other consumer industries, as well as the public sector. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential. DevOpsSUMMIT at CloudEXPO expands the DevOps community, enable a wide sharing of knowledge, and educate delegates and technology providers alike.
Docker is sweeping across startups and enterprises alike, changing the way we build and ship applications. It's the most prominent and widely known software container platform, and it's particularly useful for eliminating common challenges when collaborating on code (like the "it works on my machine" phenomenon that most devs know all too well). With Docker, you can run and manage apps side-by-side - in isolated containers - resulting in better compute density. It's something that many developers don't think about, but you can even use Docker with ASP.NET.
If you are part of the cloud development community, you certainly know about “serverless computing,” almost a misnomer. Because it implies there are no servers which is untrue. However the servers are hidden from the developers. This model eliminates operational complexity and increases developer productivity. We came from monolithic computing to client-server to services to microservices to the serverless model. In other words, our systems have slowly “dissolved” from monolithic to function-by-function. Software is developed and deployed as individual functions – a first-class object and cloud runs it for you. These functions are triggered by events that follow certain rules. Functions are written in a fixed set of languages, with a fixed set of programming models and cloud-specific syntax and semantics. Cloud-specific services can be invoked to perform complex tasks. So for cloud-na...
Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex to learn. This is because Kubernetes is more of a toolset than a ready solution. Hence it’s essential to know when and how to apply the appropriate Kubernetes constructs.
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It's clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. That means serverless is also changing the way we leverage public clouds. Truth-be-told, many enterprise IT shops were so happy to get out of the management of physical servers within a data center that many limitations of the existing public IaaS clouds were forgiven. However, now that we've lived a few years with public IaaS clouds, developers and CloudOps pros are giving a huge thumbs down to the...