Server Load Balancing
September 09, 2010 Thursday

Why is load balancing of servers needed?

If there is only one web server responding to all the incoming HTTP requests for your website, the capacity of the web server may not be able to handle high volumes of incoming traffic once the website becomes popular. The website's pages will load slowly as some of the users will have to wait until the web server is free to process their requests. The increase in traffic and connections to your website can lead to a point where upgrading the server hardware will no longer be cost effective. In order to achieve web server scalability, more servers need to be added to distribute the load among the group of servers, which is also known as a server cluster. The load distribution among these servers is known as load balancing. Load balancing applies to all types of servers (application server, database server), however, we will be devoting this section for load balancing of web servers (HTTP server) only.

About load balancing mechanism - IP Spraying

When multiple web servers are present in a server group, the HTTP traffic needs to be evenly distributed among the servers. In the process, these servers must appear as one web server to the web client, for example an internet browser. The load balancing mechanism used for spreading HTTP requests is known as IP Spraying. The equipment used for IP spraying is also called the 'load dispatcher' or 'network dispatcher' or simply, the 'load balancer'. In this case, the IP sprayer intercepts each HTTP request, and redirects them to a server in the server cluster. Depending on the type of sprayer involved, the architecture can provide scalability, load balancing and failover requirements.

Server Load Balancing Algorithms

Types of load balancing

Load balancing of servers by an IP sprayer can be implemented in different ways. These methods of load balancing can be set up in the load balancer based on available load balancing types. There are various algorithms used to distribute the load among the available servers.

  • Random Allocation

    In a random allocation, the HTTP requests are assigned to any server picked randomly among the group of servers. In such a case, one of the servers may be assigned many more requests to process, while the other servers are sitting idle. However, on average, each server gets its share of the load due to the random selection.

    Pros: Simple to implement.

    Cons: Can lead to overloading of one server while under-utilization of others.

  • Round-Robin Allocation

    In a round-robin algorithm, the IP sprayer assigns the requests to a list of the servers on a rotating basis. The first request is allocated to a server picked randomly from the group, so that if more than one IP sprayer is involved, not all the first requests go to the same server. For the subsequent requests, the IP sprayer follows the circular order to redirect the request. Once a server is assigned a request, the server is moved to the end of the list. This keeps the servers equally assigned.

    Pros: Better than random allocation because the requests are equally divided among the available servers in an orderly fashion.

    Cons: Round robin algorithm is not enough for load balancing based on processing overhead required and if the server specifications are not identical to each other in the server group.

  • Weighted Round-Robin Allocation

    Weighted Round-Robin is an advanced version of the round-robin that eliminates the deficiencies of the plain round robin algorithm. In case of a weighted round-robin, one can assign a weight to each server in the group so that if one server is capable of handling twice as much load as the other, the powerful server gets a weight of 2. In such cases, the IP sprayer will assign two requests to the powerful server for each request assigned to the weaker one.

    Pros: Takes care of the capacity of the servers in the group.

    Cons: Does not consider the advanced load balancing requirements such as processsing times for each individual request.

The configuration of a load balancing software or hardware should be decided on the particular requirement. For example, if the website wants to load balance servers for static HTML pages or light database driven dynamic webpages, round robin will be sufficient. However, if some of the requests take longer than the others to process, then advanced load balancing algorithms are used. The load balancer should be able to provide intelligent monitoring to distribute the load, directing them to the servers that are capable of handling them better than the others in the cluster of server.

Server Load Balancing Methods

Methods of load balancing

There are various ways in which load balancing can be achieved. The deciding factors for choosing one over the other depends on the requirement, available features, complexity of implementation, and cost. For example, using a hardware load balancing equipment is very costly compared to the software version.

  • Round Robin DNS Load Balancing

    The in-built round-robin feature of BIND of a DNS server can be used to load balance multiple web servers. It is one of the early adopted load balancing techniques to cycle through the IP addresses corresponding to a group of servers in a cluser. The details on the implementation is discussed here.

    Pros: Very simple, inexpensive and easy to implement.

    Cons: The DNS server does not have any knowledge of the server availability and will continue to point to an unavailable server. It can only differentiate by IP address, but not by server port. The IP address can also be cached by other nameservers and requests may not be sent to the load balancing DNS server.
     

  • Hardware Load Balancing

    Hardware load balancers can route TCP/IP packets to various servers in a cluster. These types of load balancers are often found to provide a robust topology with high availability, but comes for a much higher cost.

    Pros: Uses circuit level network gateway to route traffic.

    Cons: Higher costs compared to software versions.

  • Software Load Balancing

    Most commonly used load balancers are software based, and often comes as an integrated component of expensive web server and application server software packages.

    Pros: Cheaper than hardware load balancers. More configurable based on requirements. Can incorporate intelligent routing based on multiple input parameters.

    Cons: Need to provide additional hardware to isolate the load balancer.

Round Robin DNS Load Balancing

How DNS load balancing works

When the request comes to the DNS server to resolve the domain name, it gives out one of the several canonical names in a rotated order. This redirects the request to one of the several servers in a server group. Once the BIND feature of DNS resolves the domain to one of the servers, subsequent requests from the same client are sent to the same server.

DNS load balancing implementation (Multiple CNAMES)

This approach works for BIND 4 name servers, where multiple CNAMES are not considered as a configuration error. Assuming there are 4 web servers in the cluster configured with IP addresses 123.45.67.[1-4], add all of them to the DNS with Address records (A Names) as below. The srv[1-4] can be set to any name you want, such as foo[1-4], but should match the next step. 

srv1   IN  A   123.45.67.1
srv2   IN  A   123.45.67.2
srv3   IN  A   123.45.67.3
srv4   IN  A   123.45.67.4

Add the following canonical names to resolve www.domain.com to one of these servers. 

www    IN  CNAME   srv1.domain.tld.
       IN  CNAME   srv2.domain.tld.
       IN  CNAME   srv3.domain.tld.
       IN  CNAME   srv4.domain.tld.

The DNS server will resolve the www.domain.com to one of the listed servers in a rotated manner. That will spread the requests over the group of servers.

Note: The requests sent to http://domain.com (without 'www') should be forwarded to http://www.domain.com in this case to work. For BIND 8 name servers, the above approach will throw an error for multiple CNAMES. This can be avoided by an explicit multiple CNAME configuration option as shown below. 

options {
   multiple-cnames yes;
};

DNS load balancing implementation (Multiple A Records)

This above approach with multiple CNAMES for one domain name is not a valid DNS server configuration for BIND 9 and above. In this case, multiple A records are used. 

www.domain.tld.  60  IN  A  123.45.67.1
www.domain.tld.  60  IN  A  123.45.67.2
www.domain.tld.  60  IN  A  123.45.67.3
www.domain.tld.  60  IN  A  123.45.67.4

The TTL value should be kept to a low value, so that the DNS cache is refreshed faster.

Other considerations

The DNS based load balancing method shown above does not take care of various potential issues such as unavailable servers (if one server goes down), or DNS caching by other nameservers. The DNS server does not have any knowledge of the server availability and will continue to point to an unavailable server. It can only differentiate by IP address, but not by server port. The IP address can also be cached by other nameservers, hence requests may not be sent to the load balancing DNS server.