Record & Playback testing: Dipping your toe into test automation

Testing is crucial to the success of any software project. Unfortunately, some stages of this process can be time-consuming, repetitive, and usually contain human errors made during the work. Automation of the testing process has become an important step to improve software quality.

It helps reduce risks not only in the current project, but also future projects, which means it can be used for longer periods of time with less maintenance. Unfortunately, learning an entirely new scripting language takes time and not all projects need a robust automating testing tool. In this case Record & Playback framework can be useful.

Record & Playback testing

Automated testing implicates automating a manual process that uses a defined testing process. This defined process must have:

• Detailed test cases. These commonly include a summary, description, preconditions and most importantly, expected results. These test cases must be based on the functional specification documentation.

• A dedicated test environment

In the Record & Playback methodology, the tester has to run each test case only once and each manual test is recorded. If the test case needs to be run again, the generated script needs to be loaded and run, then the results will be checked against the expected results. This methodology does not require special automation expertise. It works well in cases where:

• Scripts are created and used for a short period of time. Long periods mean increased scripts maintenance.
• The testing team does not have a major automation expertise and it is mandatory to use automated testing for the project.
• Test scripts will be executed during few release cycles, to be discarded later.

But knowing that this framework is easy to learn and use does not mean that the QA process should not be organized. For instance, it is important to understand the environment and business rules, including the unseen behaviors of the application. Determine if the test scripts will be reusable, flexible and robust. Create a physical structure where they will be saved and determine how they will be accessed. Finally, identify the different properties that need to be used. Based on how objects are recognized, changes will be made to identify break points and specific validations.

Limitations

Tests automated using this methodology will fail if there are minor changes in the interface or the system environment. There are some limitations that need to be taken into account when adopting this methodology:

Behavior sensitivity

Any change in the functional specifications or requirements will automatically affect the test cases, whether they are automated or not.

Interface sensitivity

Record & Playback testing is based on the user interface. Any small change made to it, like buttons or text fields name which are invisible to the user, will affect the test results.

Data sensitivity

Some preconditions for the test cases may change, like data that already exists in the system and can’t be modified. Additionally, the only data used is the one that was recorded.

Context sensitivity

This depends on the system configuration and the settings that may affect the system. For example, access to servers, date and time.

Conclusions

• Manual testing must be completed first before automating test cases. Create test cases first, add expected results and then analyze what to automate.
• It’s a good idea to automate most test cases that are going to be repeated. For test cases that are complex, it’s better to test manually (it would take too much time to automate them and get the desired results).
• The Record & Playback methodology works as a good starting point to introduce automated testing into the organization, but there are more robust methodologies to accomplish the QA goals more effectively.
• This methodology can be effective when test scripts are reusable and have a short life cycle.
• Organization in the QA process with the use of this technology is crucial because its maintenance demands a lot of time.

In part 2 of this series, I'll review automated testing methodologies.

References:

Keith Zambelich, Totally Data-Drive Automated testing, [ONLINE].

Six Tips to Get Started with Automated Testing

Dion Johnson, Record & Playback, You Have My Apologies.

Anil Jain, Test Automation — Record and Play or Automation Framework (Myth vs. Reality),