Top Ten Load Testing Tips
Tip 1 - Decide exactly why you want to test
Clearly define what value you expect from the load test. What question(s) are you trying to answer (see examples below)? Agree what level of success you need and this will define when you stop testing or making improvements. Make these relevant to your business.
See more.
Start simple and work towards your real world scenario. Understand how the load test affects your system and then apply that understanding to the final (more complex) tests. Understanding your results and being able to act on them is the key to success.
- Do you want to break your site, diagnose an existing problem, or plan for a specific event?
- Are you trying to replicate existing usage levels, or plan for the future?
- What makes a successful test?
- When do you stop testing?
- How many concurrent users?
- What mix of user journeys?
Hide this content.
Tip 2 - Give yourself time
Load testing doesn’t have to take ages. But you’ll get far more value from it if you give yourself time to plan, execute and complete corrective work. The three phases of a load testing project – before, during and after testing – each need a lot of thought and planning.
The rest of this guide expands on these phases.
Tip 3 - Involve the right people: in-house and suppliers
Your testing should provide detailed information about the performance of your infrastructure, content and network. Without this information you won’t have a complete view of performance. Do the people responsible for the testing have access to teams covering these areas? If the right people aren’t available, you won’t get the complete picture.
See more.
- How will they be involved and in which phases of the project?
- What will their responsibilities before, during and after the test?
- What information do you need to supply them with?
- What information do you need from them?
- Will the same people be available for further testing or remedial work?
- Can geographically dispersed project teams work together?
Hide this content.
Tip 4 - Be brave! Load test your live website if you can
Remember the value of the load test! If you load test a site that isn’t representative of live, will the results be meaningful? Testing your live website provides the ultimate assurance that it will perform under load. Make sure you understand the impacts of testing on a live site. Will you place orders as part of the test? Can you test your payment gateway? Will real users be affected in any way?
See more.
Don’t forget:
- If you’re testing on live, will you need to test out of hours?
- If you can’t test on live, then how will the test platform resemble the live one?
- Are you testing your failover systems and procedures? Can they achieve the same number of page impressions as live? Is failover and recovery seamless?
- Are there any batch processes running during the test that may skew results?
- When are backups, updates or re-indexes performed?
- Ensure you have notified any third party suppliers that support the website.
- If your site fails due to bandwidth issues, will your connectivity provider be able to respond? Will they charge for additional or burst bandwidth?
- What will happen if you run out of disk space?
- Don’t forget that if you are using 3rd party content providers, you will be putting load on their sites as well. A courtesy call to them is always appreciated.
Hide this content.
Tip 5 - Test what’s critical to your customers and your business
Leverage your analytics and highlight the most popular or business critical feature of your site. Ensure you understand what people are going to be doing on your site. Decide which user journeys touch on all parts of your system or infrastructure; how can you be sure that every aspect is tested? Time spent here designing your load test is invaluable.
See more.
- Find busy parts of the site
- Find parts of the site directly responsible for revenue
- Identify potential performance bottlenecks in advance
- Calculate think times between user journey steps (i.e. to replicate time to complete a form or read a page)
- Think about download thresholds – How long will users wait for a page before they move away?
- Understand how page size will affect your bandwidth
- There is less point in load testing elements of your site that users will rarely visit. Make it meaningful. Make it relevant. Make it count.
- Does 100% of the page need to be downloaded 100% of the time?
Hide this content.
Tip 6 - Get smart! Use various inputs when setting load levels
Understand how many page impressions you get at peak time and what you might expect in the future (your web analytics can help you here). What input data do you need to simulate usage (think user logins, search terms, credit card payment details and so on)?
See more.
- How many page impressions can my site handle now (or in the future)?
- How many actions are achieved during a period of peak and average activity? How many people are: - Browsing the site?
- Buying something?
- Logging in?
- Signing up?
- Searching?
- Downloading? - How many transactions fail (and where) under load?
- How does your new site compare to your old one with the same traffic levels (benchmarking)?
Hide this content.
Tip 7 - Understand your errors – don’t ignore them!
It sounds like such an obvious point, but it’s often not followed. Don’t just tick a box to say you’ve load tested! Dig into the errors to see where the weaknesses lie. You may need to understand performance in the future when usage patterns change. Understand what your errors are telling you as they’ll give you more understanding about your site.
See more.
- Think about why your site breaks at a particular tipping point (e.g. insufficient database memory, bandwidth, limited available connections, poorly configured load balancer, applications operating beyond their designed usage levels etc)
- Understand what level you might reach by fixing issues highlighted by reported errors
- Understand how exceeding comfortable levels of usage will impact everyone’s experience
- Understand the degradation in performance between what you want to achieve and what level of concurrency can actually be handled by the website at maximum capacity
- Has the testing methodology tainted the results? Can the application of load be improved?
Hide this content.
Tip 8 - Use iterative testing techniques: test, improve, re-test, etc.
Load testing always highlights vulnerabilities! It’s rare that you’ll only need one quick test and you will pass first time. Test to find your system’s weak points, and then look to make improvements there and then. Re-test to ensure you have increased performance as you expected and then repeat the process. Iterative testing leads to more rapid improvement.
See more.
Tests fail for all sorts of reasons:
- Bad test design
- Scheduled process in the back-end (common during out-of-hours testing)
- Bad data sets (user logins, search terms, stock availability, promotions)
- Technical limitations of hardware and software
- Limited number of database or web server connections
- Poorly configured load balancer
- Changing web application (currently in development or unstable)
- Bandwidth limitations
Ensure you build in time to discover these issues between tests. Also build in time to try out different system configurations (such as compression, persistent connections and caching). Test, tweak and retest!
Hide this content.
Tip 9 - You’re not quite done yet. Design your ongoing testing regime
Understanding when to load test will allow you to plan for the future. As you know from earlier tips, planning is the key to success. Think about what changes ahead may need to be tested or re-tested (more examples below). Make sure you consider load testing each time you do a significant release.
See more.
Load testing can be done at several stages:
- Before and after refreshing of infrastructure (i.e. how will the application perform with new hardware?)
- As part of your ongoing development/release cycle (i.e. have you planned testing time as part of your schedule?)
- New website ready to go live or just gone live (i.e. will the new code improve service delivery performance?)
- Marketing/email campaign about to launch (i.e. can the site handle a significant influx of new users in a short period of time)
Hide this content.
Tip 10 - Not just for Christmas…
OK, so we could only think of 9 tips. But the fact is that if you have followed the other nine, you not only get more from your load testing, you will also be way ahead of most of your competitors.
See more.
The bottom line is this:
- Load testing doesn’t have to be time-consuming or difficult. We’d love to say it was and that you must use people like us to help but it’s easy with the right tool and approach.
- Load testing can deliver great value by avoiding future disasters – if you do it right.
- Load testing should be part of everyday life, not just a last minute panic before the Christmas rush!
Hide this content.