Load Testing - What is Design LoadIn short, the Design Load for a website, or environment, is the total amount of traffic (or load) that the system is designed to support.  This is the metric that describes the scalability of a system.  Design load goes hand-and-hand with performance criteria as a key part of a solution’s technical requirements.

The term ‘Design Load’ is borrowed from more traditional engineering practices, in which it measures the amount of physical force that a component must be able to withstand.  For example, when designing a new home or office building, the Design Load for floors and roofs will specify the minimum weight that the floors and roofs must be able to support. 

Design Load Aggregates Multiple Considerations

The Design Load for a floor will incorporate a number of different elements, including the weight of furniture, equipment, people, and the weight of the floor itself.  Each of these elements will vary depending of the planned usage, planned occupancy, and building materials.  Similarly, a website’s Design Load includes a number of different elements:

  • The site’s current traffic – usually pulled analytics data
  • Organic growth (often 20% growth per year for 3-5 years)
  • Increased traffic due to the new site’s functionality or architecture (like new video streaming features or interactive polls or forums)

Factor of Safety

Back to our floor example, building codes typically require that the sum of the load elements be increased by some multiplier, or “factor of safety”.  This multiplier helps to make sure the floor doesn’t collapse, even if there are minor materials defects or unplanned usage like an office party where everyone is jumping in unison to House of Pain’s “Jump Around” (OK, that’s never happened in any office party I’ve been to). 

In the web world, we should also consider applying a factor of safety to protect against traffic spikes that can come from a variety of sources:

  • Ad campaign that goes viral
  • Media mention
  • Denial of Service (DoS) attack
  • Search engine indexing bots

The appropriate value for this factor of safety depends largely on the type of site and the perceived risk of traffic spikes.  For a typical corporate brochure site, a multiplier of 3x is usually adequate.  For a start-up, or a company actively engaged in marketing that is trying to create something viral, a multiplier of 5x might be more appropriate.

The Equation

Considering all of the ingredients listed above, the formula for calculating your Design Load might look something like the following:

DL = FS x (Current + New Features) x OG

Where:

  • DL = Design Load
  • FS = Factor of Safety (usually between 3 and 5)
  • Current = The site’s current traffic
  • New Features = Estimate of the new traffic that will result from new functionality or solution architecture
  • OG = Estimated organic growth (If planning for growth of 20% per year for 3 years, OG = 1.23)

Specifying Design Load

Design Load for a website can be specified using a variety of different units and measures.  For most sites, I recommend specifying the Design Load in Page Views per Second (a few exceptions are noted below).  This metric is meaningful for most business stakeholders and translates well to analytics data, which is often Page Views per Day.  The one challenge with this metric is that it doesn’t translate easily to most load testing tools, which are usually configured in terms of concurrent users.

Some alternative metrics that can be used to specify Design Load include:

  • Concurrent users - The number of users using the system at any one time.  This isn’t very meaningful unless you also specify the “think time”, which is the delay from one user request to another.  It also seems that every load testing tool treats concurrent users a little differently.  Be sure you REALLY understand what your tool means by “concurrent users”.  This metric often isn’t particularly meaningful to business types, and doesn’t translate well to analytics data.
  • Concurrent requests – The number of requests at any one time.  This is more meaningful than concurrent users, but still needs further clarification.  If the Design Load is five concurrent requests, does that mean five requests issued at exactly the same time, or does it mean five requests that are in various states of completion at any one time?  Again, make sure you REALLY understand your load testing tool.  Like concurrent users, concurrent requests isn’t meaningful in a business context and doesn’t translate well to analytics data.
  • Transactions per second – More appropriate for transactional systems where most of the systems resources are spent processing transactions rather than rendering pages.
  • MB per second (or GB per second) – More appropriate for systems that are heavy on video, audio, and/or images. 

Design Load and Load Testing

 As mentioned before, the Design Load typically helps define the success criteria for load testing.  The ultimate goal of load testing is to validate that the system can support the specified Design Load while still satisfying the performance criteria.  The statement this kind of technical requirement might look something like this:

“The site must be able to serve 50 page views per second, while maintaining a response time of no more than 2 seconds to serve a page’s HTML from the hosting environment to a client on the public internet.”

Load testing a site beyond the Design Load should NOT be a part of the success criteria or site acceptance, but in some cases it might provide some value for long term planning beyond the project’s planning horizon, or in case organic growth is faster than anticipated.