Load Balancing

July 10th, 2010

How and when to implement load balancing are some of the early questions a start-up CTO must answer.

In essence, load balancers are designed to achieve three primary objectives:

  • High-availability (HA) – Does the site have the capability to remain accessible if there is a system failure? In particular, the failure of an individual server will not render some part or all of a site inaccessbile.

  • Scalability – Is there the ability to easily increase the load capacity of the site and adapt to increased traffic without impacting performance?

  • Predicatability – Do you have confidence that the site will perform as expected with regards to performance and availability?

To better understand today’s Application Delivery Controllers (ADC) I think it is important to first understand load balancing from a historical perspective.

In the early days of the Internet boom purpose built load balancers were not available and reliable mainframes that had the required processing capability didn’t have web server software and were far too expensive for a start-up budget. Therefore, attempts were made to use existing technology, the most prevalent (and still used) being round-robin DNS where the DNS server would return a different order of resolved IP addresses with each request. Using this technique didn’t require any additional hardware (cheap) and provided nice scalability as you only needed a new server and a new DNS entry to scale. However, this method doesn’t address High Availability or Predictability. The DNS server doesn’t have the intelligence to determine if a server is actually working and could (and would) route traffic to servers that weren’t currently available. It could also route traffic to servers that were already extremely loaded, loading them even more, resulting in poor performance when there existed the possibility that other servers were under-loaded and could provide excellent response.

To resolve these issues, attempts were made to implement load balancing directly within the application or operating system. In the early 2000’s we were an early adopter of both Window Server and Coldfusion’s clustering. The basic concept here was that each server listed to both a Cluster IP in addition to their own physical IP address. Whichever server in the cluster responded to the service request first would then perform a redirect to the server’s physical IP address and the session would start. There were varying degrees of intelligence built into these techniques and, on the surface, scalability was good. However, as the cluster grew (say above 10 servers) internal network traffic could begin to impact end-user traffic along with processor utilization. Obviously, this solution also improved HA and Predictability, although I always felt that the solution was fragile and there always seemed to be scenarios where something happened to the system that the load balancing didn’t catch. You were also dependent upon the application vendor to support functionality which was not their primary line of business and didn’t always receive the attention it needed. On several occasions we spoke directly to the primary clustering developer regarding issues at Allaire and I was always extremely nervous that this individual would move on leaving us and the product dead in the water.

Fortunately, it was not long before purpose built load balancing devices came along. These boxes were application neutral and resided outside the application servers themselves. These devices could control exactly which server received the connection and more importantly employed increasingly complex monitors to make that determination. Using a pair of load balancers (for HA) they significantly improved the Scalability, HA and Predictability of a given site along with providing the capability to provide load distribution in addition to load balancing. A final added benefit was providing the ability to easily take servers off-line for required maintenance and upgrades with no effect on end users. I always felt much more comfortable when we were finally able to purchase a pair of gently used load balancers off eBay during the dot com bust and use them instead of ColdFusion’s load balancing.