Shift Left: Test Automation to Improve Site Outcomes

Do you want to respond faster to evolving business needs? Being able to make rapid changes to your website functionality — and other digital marketing technologies — is increasingly important. Marketers, and businesses as a whole, can can’t be slowed down by the traditional Software Development Lifecycle (SDLC). The traditional model of developing software for months (or years) and testing for weeks (or months) doesn’t give businesses the ability to compete in today’s world. We know software must be delivered more frequently and with quality.

We also know that the longer you wait to test your software, the more costly it's going to be.

The NIST estimates that fixing a defect in your software costs 5 to 30 times more to fix after a project has been launched. It costs less the earlier the defect is found.

What is ‘Shift Left’?

At Oshyn, we’ve put into practice something called “Shift Left”. We strive to test our software earlier in the cycle (i.e. to the ‘left’) to reduce the frequency and costliness of defects found later in the life cycle of a website. We do this through the use of test automation throughout the software-building process. We strive to minimize our manual testing and focus on test automation earlier in the development process.

Comparison of Testing Strategies

Manual testing — the process of having a human click through a piece of software or website to verify it behaves as expected — while tried and true, becomes costly if your aim is to fully regression test a website or piece of software for every release. This may have not been apparent when the software was released three to four times per month, but if the goal is to keep up with business and marketing, the requirement for frequent releases is in the order of weeks or days, not years or quarters. There is just no way to properly regression test a piece of software of any consequence in a two-week iteration. If you have a two-week Iteration to build and deploy new features for your application one of two things ends up happening; either:

1. Only a small percentage of the application ends up getting regression tested each release (many times known as the Critical Features Test). This leaves huge swaths of the application open for regressions every two weeks.
2. You end up spending 65% of your two-week iteration regression testing and only 35% building new features

Both situations are untenable and unacceptable for any business or marketing department.

Building test automation into our applications from the beginning increases the initial cost, but you quickly make up the cost further downstream during testing with reduced manual testing costs and reduced costs associated with unexpected defects in your production systems (i.e. rework costs AT BEST and lost business AT WORST).

No testing at all is really cheap in the beginning (it’s free!), but it gets exponentially more expensive to fix things as time goes on and the problems mount. Ultimately, it’s disastrous for everyone.

Manual testing doesn’t scale particularly well. As time goes on, and things get more complex, your costs rise linearly with each release and the burden only gets larger as more features are built.

Automated testing is the best solution: it requires a little more investment up front, but that investment pays off as time goes on, especially as your business demands more frequent releases of features.

Benefits of Shifting Left

Reduce Costs

• Investment in automation survives project phases, team turnover, and helps clarify requirements early. It is an asset that can keep your website working as expected through any tumultuous business events.
• Cost when bugs are found in production: doing work twice always costs your business more.
• Opportunity costs of lost customer perception may be the BIGGEST cost to your business. If you release a website that doesn’t work, your customers will quickly find an alternative in your competitors.

Increased Confidence

• Done right, tests at each level guarantee the test-target is performing as requested.
• Clear and objective proof of quality: you have metrics that can be clearly monitored by your entire business.

Higher Quality

• Fewer outages in production: don’t get phone calls when your customer service team gets angry calls from your customers.
• Fewer/no regressions: this is the website you expect. Not one that gives you “one step forward, two steps back”.

Increased Efficiency

• Automation can vastly outperform manual work in tests per second. Eventually, you can regression test your entire site in hours or less.
• You can re-allocate the time you gain in each sprint into new feature development.

Competitive Advantage

• Test automation matures into continuous delivery, enabling quick reactions to shifting business needs and market changes.
• Look good to customers.
• Your ability to react to your customer needs improves dramatically. You are no longer a slave to your software development timeline.

Shifting left with test automation can unlock these benefits for your business.

Oshyn can help you get started with automated testing. It's also a core component of our Uptime service.