Purpose

This article presents some ideas for structuring high traffic applications and web sites in such a way as to minimize potential bottlenecks and enable application availability in the event of hardware and software failures.

Applications and web sites that are required to handle large volumes of traffic require sophisticated load balancing techniques in order to utilize clusters of server machines. The clusters of servers coupled with multiple networks typically provide scalability as well as failover in the event of hardware or network failure. Nevertheless, each server in the cluster often requires access to a central data store for some portion of its work. For example, a web server may generate a product list but needs to view product availability available only on the master database server.

Realizing that not every application take advantage of the techniques described here, we present ideas that can dramatically improve application performance and availability in many situations.

Principles

We try to address areas of application architectures that are problematic and present some alternative techniques to achieve similar goals.

Databases are the bottlenecks in many high traffic applications

• Traditionally each “front-end” machine has to interact with the master database for each transaction
• Caching of static data such as product catalogs partially alleviates the issue
• When central/master database servers have problems, the web site may not be accessible at all

Cluster of real-time independent servers

• Scalability is achieved by adding additional web and application servers to the cluster
• Architectures scales linearly with additional “front-end” hardware while keeping master database loads very light

Local transaction accumulation

• Transactions such as advertisement views and purchases are logged to local databases
• Periodic processes send accumulated local transactions to master databases
• Primary key segmentation schemes are required to avoid conflicts in the master databases
• Sticky load balancing algorithms finalize transactions on correct servers

Asynchronous local and cache data replication from master databases

• Information such as product catalogs and other “static” data is replicated to be accessed locally by “front-end” servers
• Can be done using database replication techniques and selecting only specific tables for replication

Advantages

If you are able to achieve the goal of linear scalability, your growth will be smooth and as simple as adding new hardware. Your system will have redundancy and failover built in and allow you to perform maintenance without disrupting service. Here are some key factors that your system will take advantage of:

• Load balancers can detect unresponsiveness from a specific web server and redirect traffic to others in real-time
• Master database unavailability does not impede site performance. This is due to the fact that transactions are retained locally on “front-end” servers until they are sent to the master database
• 24/7/365 high-availability is simple and maintenance can be performed at any time on any server without impacting application availability. This can typically be done on “front-end” web servers but not on the master databases without impeding production performance. The architecture we propose will enable you to maintain any component of your system without service disruption.