Managing Quality Growing Teams

1. Overview
2. Industry
3. Services Provided
4. Company
5. Background
6. Grade and Quality
7. QA processes
8. The Challenge
9. The Solution
10. Quality Process Settlement
11. Determining the current status of the quality assurance process
12. The path of improvements
13. Testing And Documenting Quality
14. Planning Quality
15. As a result
16. Results/Achievements
17. Value Delivered
18. Lesson Learned
19. Conclusion

Overview

In this case study, we have considered the main issues of changing approaches to quality as software development teams grow. General approaches and metrics for analyzing the quality assurance process in projects have been determined. The key quality problems that teams may encounter as they grow from a startup to a large project have been outlined.

Industry

Blockchain is a constantly growing system of records (blocks) united by cryptography, an open distributed ledger efficiently and permanently recording transactions between several parties so that they can be reviewed whenever necessary. A hash, a timestamp, and information about a transaction (a Merkle tree) are available in each block.

Blockchain is highly valued for inter-node communication, safety, and resistance to data modification. If a user wants to alternate data in one block, they would change all subsequent blocks. The consensus of the network majority is needed. Therefore, blockchain has secure structure and provides tangible benefits to our platform.

Services Provided

1. Project Management
2. Quality Assurance & Control

Company

Opporty is a secure, transparent, blockchain-based platform for business owners and their clients. It consists of 3 layers: Proof of Expertise Protocol, a Business Scoring System, and a Marketplace Platform. All the processes taking place at Opporty are standardized and rigorously checked, so users can be sure that their personal information and money are duly protected.

The team of Opporty developers strives to reach international coverage since our platform can process intensive data flows and serve a great number of clients. This advantage is due to the use of Plasma Cash Protocol – an innovative technology nurtured in a traditional Ethereum environment. It was successfully applied while creating Opporty Enterprise Solution (OES) – a permission-based variation of the Opporty system.

OES can significantly improve and expand the functions of applications and third-party websites, contribute to the smooth operation of enclosed corporate environments, make them safer and more convenient, protect metadata, speed up transactions, ensure privacy, analyze transactions and assign scores to specific activities.

At the moment, Opporty has the following advantages: the team of top-notch IT specialists including React, JS, Python developers, QA, DevOps, BA, designer, and PM;

Once it was launched in 2018, a developer team faced challenges in many areas including security, integrations, growing load, and service delivery to several countries.

Background

The product quality is the basis for value delivery. The first and foremost task is to ensure the software are clear and stable after the final delivery. The second task is to make sure all required functionality is developed in the way the final user needs.

When the project is at a start-up stage, all the team members are focused on the value delivered to customers. This makes sense. After all, we should find the shortest way to the market and the first adopters.

The development requirements and approaches were changing in the process of project growth from an idea and a startup to a full-fledged product with offices in several countries. At the same time, it was necessary to revise the attitude towards teamwork organization in order to obtain timely and high-quality results.

Grade and Quality

In order to give you a comprehensive idea of our work, we should describe the two most important factors for product quality measurement: Grade and Quality.

Grade is the amount and completeness of value delivered e.g. features, fixes, improvements, etc.

Quality shows how value is delivered with regard to Stability, Reliability, UX, and Bugs.

For growing startups, the problem of insufficient Quality often arises at the intersection of these two factors.

The fact is that even a high-quality product would not satisfy the user if the UX and functionality completeness issues are not addressed. Such flaws generate so-called UX Bugs.

When working with a growing project, it is important to pay attention to both of the above-mentioned issues.

The issue of functionality completeness for an end user is often solved after the main development. As a result, some functionality parts are not included in architectural solutions. You either have to spend a lot of time refactoring and re-doing already completed work or generate a technical debt, that is, a bad problem-solving history.

In a hurry, refinements and corrections are added to already performed development tasks, which leads to low quality. Technical debt is formed, which reduces the speed of subsequent project development and increases the likelihood of non-systematic artifacts.

QA processes

The following practices should be used to make QA Process fast and reliable in the start-up mode:

• Manual testing,
• Manual Regressive testing,
• White-box testing.

The following testing types are rarely used:

• Automated Testing,
• Unit Testing,
• Load/Stress Testing,
• Reliability/Usability Testing

We are going to use these types for QA Process in the short run. Later, with the growth of the project’s functionality and data amount, the number of platform users would grow as well.

Each time we reject some types of testing we sacrifice the precious time needed for further development. It is wasted resolving integration, quality, and communication problems.

For a start-up, the main thing is to have simple and transparent development process. But you should be ready for the fact that this elementary process will become ineffective in the future.

Therefore, as the project and team grow, we have to introduce some changes to Quality Assurance & Control. This situation is similar to technical debt and the way to mitigate or avoid it.

Most young teams can’t predict and understand the issue of Quality while they grow and develop. This could be due to many different factors. For example, young teams often do not have extensive background and lessons learned in a particular area. Also, their system of practices and techniques may be rather weak or not shaped yet.

The Challenge

The project faced most of the common processes scaling, quality assurance, and project management problems as the team grew.

Unfortunately, practices suitable for several developers are inefficient for a project with many teams operating in different countries.

At the very beginning of the development process, we conduct testing manually. Functionality is small and it is easy to fully test it. But with the growth of the project and the functionality amount, it is necessary to introduce planned quality-related changes.

Otherwise, it may be challenging to ensure the quality of the functionality delivered to users. Also, the time required for development (according to test results) would grow exponentially. In fact, this results in:

• Delays in functionality delivery;
• Irrational load on developers since they have to spend time fixing bugs;
• Poor quality of the product or the availability of technical debt.

At some point, it is necessary to change quality planning and project workflow so as to ensure the timely involvement of QA engineers in the production process, as well as high-quality feedback from developers. This allows us to quickly identify and correct errors.

It is important to pay close attention to the following issues:

• Audit and analysis of quality processes;
• Quality requirements;
• Optimization and planning of quality processes;
• Documenting quality.

The Solution

Quality Process Settlement

As soon as we have an idea of problems related to the project and team growth, requirements for the desired quality should be formulated.

If we try to describe the quality expected for a particular project, many things should be discussed. Most of us want to have a beautiful, user-friendly, fast, and reliable application. But if we compile more detailed characteristics, a huge list of requirements in the following areas would be obtained:

• Speed and Reliability (Is it fast?)
• Usability (Is it user-friendly?)
• Errors (Are there any bugs?)
• Stability and Recovery (What would we do if the app breaks and we lose data?)
• Localisation / Compliance Testing (Is the application competently translated?)
• User Documentation Testing (Does user documentation reflect real-life functionality?)
• Service Layer (How fast should we respond to a support request?)

When formulating requirements, it may be determined that many nuances require constant attention at different work stages.

When creating quality requirements for a growing project, it is necessary to take into account its history and practices that have already been implemented. It is also necessary to determine which of already implemented practices require clarification, whether they work efficiently or need to be replaced.

Then you can create a plan for changing quality processes. This plan should include a list of consecutive actions or changes to be implemented. It is imperative for specialists working on the project to take an active part in the formation of this plan because agreed changes would influence their labor.

Work on the formation and optimization of quality processes may be performed in the following ways:

1. Analyze current QA flow.
2. Get an idea of a current Definition of Done for the development and Quality pipelines.
3. Define existing processes.
4. Define existing quality requirements.

It is necessary to find patterns within a company’s specifics in order to apply uniform quality standards to all projects.

It should also be borne in mind that the project quality requirements may vary depending on the type of product.

Determining the current status of the quality assurance process

The identification of existing requirements is usually reflected in the following documents (templates):

• Test Plan,
• Test Cases,
• Test Report,
• Bug Report,
• QA WorkFlow and DOD,
• Deployment QA Procedure.

It often happens that existing practices are not systematic or their use is associated with various difficulties. That is why, prior to the introduction of changes, it is necessary to determine the state of the testing processes and collect metrics.

If there are no metrics, they should be implemented and the indicator representativeness should be provided by statistical methods.

In addition, it is necessary to make a current optimization by following the Lean rules: Abolish Local Optimization and Make the Rules Explicit.

Determining the current state and using metrics, we contribute to more systematic work and find local problems. Their solution is obvious and subject to sound logic. This is due to the fact that similar problems have been solved by other company teams before. So the effectiveness of the decisions made is beyond doubt.

The following questions should be answered when assessing the system operation and processes unification:

• Is the testing process time-consuming?
• Is testing effective?
• What factors do lower quality and how to eliminate them?
• What factors do increase quality and how to improve or develop them?

You can apply the following metrics:

• The number of bugs with regard to the number of tasks.
• Labor costs for finding, identifying, and eliminating errors.
• Correlation of these costs with the labor expenses on development.

Necessary information is fairly easy to obtain statistically by analyzing the work of the team in a task tracker, namely:

• The laboriousness of the testing process is assessed by analyzing the time spent on certain tasks when testing, compiling documents and checking the bugs&fixes registry.
• The effectiveness of the testing process can be measured by analyzing the ratio of tasks performed and errors found in them. In this case, it is necessary to take into account both the number of errors and working hours spent by specialists on corrections.
• Quality barriers, as well as factors contributing to high quality,  can be determined by analyzing the sources of errors. For example, if the team adds information on the sources of eliminated bugs in a task tracker, it would be much easier to prevent subsequent errors.

In general, such teamwork analysis gives a comprehensive idea of the sources and elimination cost of errors.

As soon as we find barriers, we can improve the quality and consistency of work. To do this, we need a figure-out list of specific fixing actions.

Thanks to these actions, as a minimum, we would be able to stabilize and systematize the quality processes, and as a maximum, significantly improve the quality of the product and workflow while preserving previous labor costs.

The path of improvements

As soon as we systematize quality-related issues, we can consider the possibility of quality scaling along with project growth. It may turn out that manual regressive and functional testing takes too much time. Often this is due to the fact that the project gains high functionality and it is difficult for testers to quickly check everything manually. In a hurry, some checks are not carried out. Accordingly, some errors fall into public access, which reduces the satisfaction of app users.

Automated testing carried out by robots during app creation can come to the rescue and keep us assured that the key functionality works correctly. However, the implementation of auto-testing would be problematic if it is not used since the first day of development.

It is possible to use auto-tests and supply the whole project with them. But such a strategy may take months to implement and not correspond to the current needs of the project and its users.

Ultimately, we can systematize the requirements for quality, its processes, and the provision of resources. This will allow obtaining a stable quality level with predictable costs.

Testing And Documenting Quality

To get a clear picture of the project quality, we have to develop some quality requirements and deliver them to all the team members.

Let’s systemize our requirements as well as quality-related expectations:

When we know what we need, we can develop the process map for the quality throughout the project. Each process or action should be finished with a Definitions Of Done (DOD) list. It could be a simple checklist added to a task tracker.

Сompleting the checklist would mean “this is done” and “this could be accepted by the next team member”.

The main task is to have a clear and accurate criteria list for each particular process.

Let’s say we are working with DOD criteria for the Localisation Checking. The following checks may be done, reported, and reviewed by the development team:

• Compliance Check;
• Uniqueness Ratio Check;
• Recensing;
• Native Speaker Check.

These are simple actions to be performed each time we do something with the Localisation of the Product. They could be duly described and clearly explained to each team member.

This is the source of future optimization. We can get efficient project management and integration processes as a result of strict and organized checks. In other words, if each team member knows what and how should be done, it would be easier to increase quality in a specific time period.

Planning Quality

As soon as we determine What and When should happen in terms of Quality, we can plan its optimization:

• Formulate, assess, and schedule quality procedures.
• Choose a measurement unit for calculating the volume of resources per certain time period – an hour, a day, etc.
• Add all the measured activities to a list reflecting actual workflow.
• Register the resources needed for errors testing and correcting in performers’ work plans.

These stages may seem complex. But, after closer inspection, they turn out to be quite easy to implement.

Let’s consider an example. If we want to get a good application build, the following things should be done:

• 2 days before release, all the teams should stop new development and focus on quality.
• The QA team should start all the testing processes.
• During testing, the QA team should produce BugReports.
• A team lead should sort, delegate, and fix bugs.
• After the QA team eliminates all bugs, other teams should start working with the next functionality levels. At the same time, the build should be completed and delivered.
• As a result, we should get a new build with ready tests/bug reports.

If we work in this way, we would definitely know how much time any of these activities would take and what resources are needed to perform it.

Based on my example case, I am sure that the Delivery Team of 1-2 QA and several developers would spend 2 days to deliver the code for several teams’ 2 weeks sprint.

The following documents were developed, supplemented, or updated as a result of quality planning:

– Test Plan

– Test/Bug Report Template

– Test Cases

– Release Procedure

As a result

“You Can’t Manage What You Don’t Measure”

The truth is that we will get a stable and predictable process only after we organize ordered workflow, bring all the team members and their expectations to a single point. This should be the point of systematic quality improving, performance skyrocketing, and tracking project changes.

Systematic measurement makes it possible to implement changes and improvements that are effective in each particular case.

Results/Achievements

As a result of this implementation, Opporty has increased the management efficiency for projects supporting a company’s strategic plan, stimulated tangible improvements in such areas as internal processes, customer services, and the support of learning and growth.

Project achievements include the following positive changes:

• The time of work on changes posting has been reduced from two days to several hours.
• The ratio of the number of bugs to the number of tasks has decreased several times.
• Lead time of tasks has been reduced from 2-3 months to 2-3 weeks.
• Cycle time for criticals has been reduced from 2-3 days to 2-6 hours (with delivery to production).
• The number of Critical Bugs has decreased significantly.
• The effort to eliminate errors has decreased several times.

Value Delivered

The team now has a consistent delivery pattern allowing us to demonstrate efficiency and consistency of development.

It has been obtained as a result of the quality processes revision. The development team was able to focus on value delivery and compliance with the business needs of the project instead of struggling with bugs and problems.

Summary

As the project progresses and development teams grow, it is important to monitor labor resources being spent to ensure high quality. Depending on circumstances, different approaches, techniques, and documents may be used. It is important to determine and clarify the processes and requirements for them in a timely manner. As the workflow standardizes and metrics are being implemented, the quality assurance trends become predictable, while the labor costs become planned and expected.

It is important to detect the need for clarifying or optimizing the quality requirements in a timely manner and develop approaches to ensuring the necessary level of quality. This should prevent wasting time and resources.