Best practices for adding scalability

My thesis is that a you can’t have a good SaaS application that doesn’t scale.  By definition the need for scalability is driven by customer demand but there is demand and there is DEMAND. A handful of lucky organizations (Google, Twitter, Facebook) are faced with industrial strength volume every minute of every day. Organizations with this type of DEMAND can afford to have entire divisions dedicated to managing scalability. Most people are dealing with optimizing their resources for linear growth or the happy situation where their application (Instragram) catches fire (in some cases overnight). A scalable architecture makes it possible to expand to cloud services such as EC2 and Azure or even locally hosted capacity. Absent a scalable architecture an organization is faced with curating a collection of tightly coupled servers and overseeing a maintenance nightmare.

Scalability is the ability to handle additional load by adding more computational resources.  Performance is not scalability, however, improving system performance mitigates to some degree the need for scalability.  Performance is the number of operations per unit of time that a system can handle (e.g., words / second, pages served / day, etc.).  There are two types of scalability – vertical and horizontal.

Vertical scalability is achieved by by adding more power (more RAM, faster CPU) to a single machine.  Vertical scalability typically results in incremental improvements.  Horizontal scalability is accommodating more load by distributing processing to multiple computers.  Where vertical scalability is relatively trivial to implement, horizontal scalability is much more complex.  Conversely, horizontal scalability offers theoretically unlimited capacity.  Google is the classic example of infinite horizontal scalability using thousands of low-cost commodity servers.

If you have the luxury of working off of a blank sheet of paper or have the flexibility to implement a major new technology stack some of the better solutions for implementing scalability include ActiveMQ, and Hadoop. Microsoft’s AppFabric Service Bus promises capability in this area for Azure hosted applications. Many times scalability was considered when an application was first created but has proven to be inadequate for current demand.  The following are suggestions for improving an existing application’s scalability.

Microsoft’s Five Commandments of Designing for Scalability

  • Do Not Wait– A process should never wait longer than necessary.
  • Do Not Fight for Resources – Acquire resources as late as possible and then release them as soon as possible.
  • Design for Commutability– Two or more operations are said to be commutative if they can be applied in any order and still obtain the same result.
  • Design for Interchangeability – Manage resources such that they can be interchangeable (i.e., database connection).  Keep server side components as stateless as possible.
  • Partition Resources and Activities – Minimizing relationships between resources and between activities

Microsoft’s Best Practices for Scalability

  • Use Clustering Technologiessuch as load balancers, message brokers, and other solutions that implement a decoupled architecture.
  • Consider logical vs. physical tierssuch as the model view controller (MVC) architecture.
  • Isolate transactional methodssuch that components that implement methods that implement transactions are distinct from those that do not.
  • Eliminate Business Layer State such that wherever possible server-side objects are stateless.

Shahzad Bhatti’s Ten Commandments for Scalable Architecture

  1. Divide and conquer – Design a loosely coupled and shared nothing architecture.
  2. Use messaging oriented middleware (ESB) to communicate with the services.
  3. Resource management – Manage http sessions and remove them for static contents, close all resources after usage such as database connections.
  4. Replicate data – For write intensive systems use master-master scheme to replicate database and for read intensive systems use master-slave configuration.
  5. Partition data (Sharding) – Use multiple databases to partition the data.
  6. Avoid single point of failure – Identify any kind of single point of failures in hardware, software, network, power supply.
  7. Bring processing closer to the data – Instead of transmitting large amount of data over the network, bring the processing closer to the data.
  8. Design for service failures and crashes – Write your services as idempotent so that retries can be done safely.
  9. Dynamic Resources – Design service frameworks so that resources can be removed or added automatically and clients can automatically discover them.
  10. Smart Caching – Cache expensive operations and contents as much as possible.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: