Data-driven or functional decomposition testing method: Which is right for you?

These are two main methods, among many, that have been proven most effective in the implementation of automated testing.

Data-driven method

To have better testing results, tests can be performed using different data. This is out of the scope of Record & Playback since it always uses the same information. Data-driven testing, also called test plan driven method, separates the data management from the test commands. A data-driven test works as follows:

1. A set of data is read from a file
2. Data is processed, actions are simulated
3. Real results are compared against the expected results
4. Next set of data is read and the process is repeated until all the data is consumed

Using an external tool to process data makes the tests reusable and easy to maintain. If new testing scenarios are added, the data files can be easily updated with new data without making any changes to the automated test.

Advantages

• Test data and verification data can be organized in just one file and it is separate from the test case logic.
• There are different file formats where the information can be stored: CSV, tab delimited, xml, Excel spreadsheets, etc.
• Depending on the tool, it is possible to have the test scripts in a single repository, which makes the scripts easy to maintain and manage.
• Actions and Functions can be reused in different tests.
• Some tools generate test data automatically. This is useful when large volumes of random test data are necessary, which helps save time.

Disadvantages

• High-level technical skills are required. A tester may have to learn an entire new scripting language.
• Learning a new tool consumes time and resources.
• There will be more documentation, mostly related to scripts management, tests infrastructure and testing results.

Functional decomposition method

The objective of this method is to break each business functionality into its most simple tasks: Log in, navigation, log out, etc. After that, common tasks are grouped into functions. These functions are managed by a driver in each test. Processing is managed with true/false conditions for each script. For instance, if a specific check box selection is necessary to enable other form options and the user does not select it, the script will return FALSE, the driver will call to a different script and continue with the test.

Advantages

• As in the data-driven methodology, it separates the data from the test scripts.
• There is a higher level of scripts reuse.
• Scripts can be prepared during the application development and in case the business rules change, only the specific script is changed.
• Test data can be stored in separate files.
• True/False results are better to work with instead of aborting the application when an expected result is not achieved.

Limitations

• Requires a high level of technical skills.
• Multiple data-files are required for each test case.
• Not only test functions and scripts must be updated, but test data files as well since this data might be re-entered in different cases.

Conclusions

• Functional decomposition framework reduces redundancy and repetition when creating test scripts, but is cost-effective.
• Data-driven testing is ideal for testing systems that are large, complex and have varied test environments.
• Data-driven testing is cost-effective because it requires minimum technical personnel.
• Good test data is useful for data-driven testing, it is easy to maintain and it helps creating automated tests efficiently.
• Test cases that are automated can be used for manual testing too.

References:

Nick Olivo (2012). Webinar: Become a Data-Driven Testing Ninja.

Keith Zambelich, Totally Data-Drive Automated testing, [ONLINE].

Data-Driven Testing - Basic Concepts.

Functional decomposition framework