The Importance of Software Testing Documentation

Introduction

Software testing is performed to evaluate a product and determine whether it corresponds to its stated functionality.

Quality assurance procedures in software testing provide the owner of the product with information about its condition, and whether all requirements have been taken into account in the development process.

Overview

Here we will share the experience of testing a project with missing test documentation. We will consider the problems that may arise during testing and ways to solve them.

Industry

Testing at all development stages can significantly improve the quality, reliability, and performance of any system. During testing, the QA team makes sure a software product has all its documented functions and does not perform unnecessary operations.

It is crucial to include testing in the software development life cycle, to ensure a high quality final product. Introducing testing at the early stages of work on a project is of particular importance since this approach can significantly reduce the cost of bug elimination.

Main advantages of introducing testing into the software life cycle:

• Testing at early development stages dramatically reduces the cost of error correction.
• Given the characteristics of the modern market, only high-quality products are competitive. Therefore, when investing resources in software development, every effort should be made to ensure that the final product can compete with existing solutions. Software testing throughout the development cycle is a key factor in assuring a high-quality final product.
• The software development environment is different from the environment in which the final product will be used. Testing enables us to run software in conditions similar to actual use.

Main testing documents and their purposes.

There are various types of documents used in testing. Each of them plays an important role in achieving the general goal – to create a great product. Here is a list of the most common testing documents:

A test plan documents all the testing details, starting with the object’s description, strategy, schedule, criteria for starting and finishing testing, up to the equipment necessary for the system’s operation, special knowledge, and risk assessment, with options for resolving undesirable situations.

A checklist is a list of details to be checked.

• It helps developers plan deadlines for the future and compare them with actual operation periods. You can specify how much time is allocated for checking some details and how much time was actually spent.
• A checklist stores the history of tests passed. It is easy to recall which tests were passed with errors and double-check them.
• A checklist with results demonstrates the current state of the product being developed to any employee in the company and helps them determine the degree of this product’s readiness.

A test case is an artifact describing a set of steps, specific conditions, and parameters necessary to verify the implementation of the function being tested or its part.

• It displays what exactly was tested and how. We can make sure that certain functionality was tested, or will necessarily be tested and affected during testing.
• A test case helps to quickly orient new employees to the work. They do not have to study the subject matter for weeks. It is enough to open a saved test case and go through it in steps.

A testing report is a written or media report on the work done and its results. This document:

• Visually displays the output of the operations performed.
• Historically captures information. You can always return to certain stages and see what exactly was done and what result was obtained.
• Provides results to everyone who needs to know about them. For example, a support department staff could be notified about the release of a new version of the program being developed, as well as the most critical problems.
• Helps to decide on further actions (for example, whether the program can be released in its current state).

The Challenge

The project has been developed for more than 2 years. Several development teams and testers worked on the project during this time. Test documentation existed in the form of checklists (Excel spreadsheets) and project specifications for developers. The requirements specification was written on a more or less acceptable level.

But since the project was constantly growing, new functionality was added, some modules were slightly modified, and developers decided to implement the functionality in a somewhat different way. All this resulted in some parts of the specification becoming outdated. Checklist-based testing also did not give full confidence in the product’s quality, since the checklist was not sufficiently detailed and lost some relevance over time.

Without documentation, it was necessary to guess how various parts of the system influence each other. As a matter of urgency, full-fledged research testing had to be conducted in a short period of time, and bugs that are critical for users had to be found.

Over time, problems began to arise due to accelerated workflow:

1. There was no clear understanding of how long it would take to test the program.
2. It was difficult to determine the product’s quality.
3. After the regression, new bugs were constantly appearing in parts of the application that had previously worked correctly.
4. Introducing a new person to the project became very time-consuming.

The Solution

It was decided to build the testing process as follows:

1. Requirements analysis
   • Requirements visualization (flowchart diagrams, UML Use Cases, Mind Map)
   • Discussing the product with a project team and a customer team on a regular basis
2. Test planning
   • Using high-level checklists
   • Information from competing products
   • Using experience from past projects
3. Test design
   • Compiling a test plan
   • Writing test cases
4. Test execution
5. Analyzing the results obtained and writing reports
6. Product transfer to the custom

The Results

As a result of laborious work, most of the above-mentioned problems were corrected. Time spent on complete regression testing was reduced. All critical bugs were fixed. After the next few development iterations, the application was successfully deployed and delivered to consumers.

The following testing documents were compiled:

1. Test plan
2. Checklist
3. Test cases
4. Test summary reports
5. Weekly Status Reports
6. User Documents/ manuals
7. User Acceptance Reports
8. Bug reports
9. Test analysis

Conclusion

Software testing documents always play an important role in the project development/testing phase. It is important to keep things documented whenever possible. Do not rely on verbal communication.

Documentation will be of great use not only to you but also to your client. In the long run, it will help them save thousands of dollars on training and, more importantly, troubleshoot problems caused by lack of documentation for development and testing.

The habit of documenting everything will certainly result in a systematic approach to the testing process and leave ad hoc testing in the past.