The Importance of Utilizing a Software QA Team - Page 2
What is the Right Level of Quality?
Quality is not binary. It is not a question of whether you have it or not. The right question is what is the acceptable level of quality. Marketing application would require a different level of rigor than an air traffic control software or software powering medical devices. When thinking through the QA process and team structure, a good starting point is articulation and understanding of the acceptable QA criteria.
Measuring quality
Many QA teams use process oriented metrics such as a number of defects found which suffer from a fairly common metric ailment. They are easy to measure but not necessarily aligned with value. It is more insightful to measure effectiveness of QA by focusing on outcomes, i.e. assessing how good the software is after it has been deployed to production. Outcome oriented metrics answer questions such as:
- Is software uptime exceeding uptime commitment?
- Are response times below expected thresholds?
- Are there any unhandled exceptions reported by the software?
- What are the numbers and severity of defects found on production and reported by customers or employees?
My Practical Perspective
I have been working with agile methodologies for a number years. At my prior startup we had a small team of great QA engineers and they made a big impact on quality of our products. We had 99.99% uptime and very few issues reported by customers. At my current startup we are religious about following Test Driven Development and Continuous Integration practices. The process is working very well. It does not come for free. We spend roughly the same amount of time developing test cases as writing the actual code. It is definitely worth it. We also have a tiny but independent QA team.
Follow Technorati