Clarifying the Testing Tool Madness

I talk to clients and potential clients daily about our business analysis solution and its capabilities for managing requirements and associated testing and quality assurance activities. We have an excellent tool for performing these activities. However, I see a lot of confusion concerning testing and quality assurance with respect to requirements management tools.  In this blog, I will describe the various types of testing tools.

To perform successful testing usually requires a variety of tools.  There is no single tool on the market, including ours, that does it all – regardless of what vendors may claim.  There are usually many people and groups that participate in testing activities and each group may require a different type of tool, depending of course on what they are trying to do.  Below is a list of people that generally participate in testing activities.

• Users
• Developers
• Quality assurance teams
• Business analysts
• DBAs
• Product owners/sponsors
• Compliance and internal audit
• Performance testing engineers
• Penetration and security test consultants

There are many different types of tools available for testing. Testing tools can generally be categorized into following three main classes: Test Management Tools, Test Data Planning and Preparation Tools, and Automated Testing Tools. A description of each tool class is provided below.

Test Management Tools

Test management tools are focused on the overall testing cycle, as well as test management and reporting during complete project cycle. These management tools are further classified based on the purposes of different functionalities they perform or which testing phase they emphasis upon.  Test management tools can generally be categorized into three categories:

1. Requirements management tools
2. Test management tools
3. Review tools
4. Configuration management tools
5. Bug tracking tools

Test Data Planning and Preparation Tools

Test data planning and preparation tools help testing resources prepare test data, analyze risks, and plan test coverage.

1. Test data preparation
2. Modeling tools
3. Coverage management tools

Automated Testing Tools

Automated testing tools helps with actual execution of tests. They can often perform stress testing and simulate systems under load or even automatically playback prerecorded test scripts.  Automated testing tools can be further classified into the following categories.

1. Test execution tools
2. Performance measurement tools
3. Unit test tools
4. Security vulnerability tools

The table below shows key features and functions for each type of tools and an at least one example for each type of tool.

Test Management Tools
Requirements Management Tools	Storing requirements Identifying missing or incomplete requirements Traceability of requirements Interfacing with test management tools Requirements coverage Example – Enfocus Requirements SuiteTM
Test Management Tools	Management of tests Scheduling of tests Management of testing activities Interfaces to other testing tools Traceability Example: HP Quality Center
Review Tools	Sorting and storing review comments Communicating comments to relevant people Keeping track of review comments, including defects Traceability between review comments and review documents Monitoring review status (pass, pass with corrections, requires more changes) Example: Inview
Configuration Management Tools	Information about versions and builds of software and test ware Build and release management Access control (check in and check out) Example: SourceAnyWhere
Bug Tracking Tools	Track defects Capture feedback and status Classifies bugs by severity and priority Time tracking Example: Bugzilla, Mantis, Trac, Fogbugz
Modeling Tools	Identify Inconsistencies or defects in Models Help in prioritization of tests in accordance with the model in review Predicting system response under various levels of loads Using UML, it helps in understanding system functions and tests Example: Altova
Automated Testing Tools
Test Execution Tools	Storing an expected result in the form of a screen or GUI object and comparing it with run-time screen or object Executing tests from a stored scripts Logging test results Sending test summary to test management tools Access of data files for use as test data Example: QTP
Unit Test Tools	Supplying inputs or receiving outputs for the software under test Recording pass/fail status Storing tests Support for debugging Code coverage measurement Examples: Junit
Security Vulnerability Tools	Identify viruses Identify denial of service attacks Simulating various types of external attacks Identifying weaknesses in passwords for files and passwords Probing for open ports or externally visible points of attacks Example: Fortify
Performance Measurement Tool	Ability to simulate high user load on the application under test Ability to create diverse load conditions Support for majority of protocols Powerful analytical tools to interpret the performance logs generated Example: HP LoadRunner
Test Data Planning and Preparation Tools
Test Data Preparation	Extract Selected data records from files or databases Data anonymization Create new records populates with random data Create large number of similar records from a template Example: Accenture Clone and Test
Coverage Management Tools	Identifying coverage items Reporting coverage items which are not covered yet Identifying test inputs to exercise Generating stubs and drivers Examples: CodeCover
Static Analysis Tools	Calculate cyclomatic complexity Enforce coding standards Analyze structure and dependencies Help in understanding code Identify defects in code Example: PMD

[cta id=”7436″]