Q7 and Code Coverage

Q7, 21.06.2012 by Andrey Platov View Comments

Sometimes our users are asking us about Code Coverage reports with Q7. So this article is to unveil our vision on this topic. First of all we believe it would be relatively easy to get code coverage reports for Q7 tests using third-party tools such as Emma or JaCoCo. However, we think any code coverage measurements are useless (or evil) when it comes to functional testing.

Unit Testing and Code Coverage

Code Coverage is definitively important metric when you are doing Unit Testing. Since the ultimate goal of Unit Testing is to ensure all your code (units, classes) is working properly (as designed) — Code Coverage level reflects your level of assurance – a percentage of your code, which is behave as expected. This is natural mapping between unit tests and corresponding pieces of code which is covered by tests. So having 10% of code covered with unit tests you may worry about your codebase quality, but having 90% of code covered with unit tests you may be much more relaxed knowing that major part of your codebase is working as expected.

Functional Testing and Functionality Coverage

Nevertheless functional testing looks quite similar to unit testing, they sits on another level. The goal of Functional Testing is to ensure all the application functionality delivered to the user, and those functions works as expected. Funny thing that any code coverage measurements would not help you to understand amount of planned functionality has been implemented and which amount is working properly. What you definitely need to know when working on functional tests is Functionality Coverage in some form (like User Stories or Software Requirements). There is natural mapping between functional tests and User Stories, which are covered by tests. So when it comes to functional testing the definitive goal of QA/Dev team should be 100% Functionality Coverage, which has no connection to Code Coverage.

Call for Comments

All of the above is just only our current opinion, and we’d be happy to hear any contras/ideas. Please share your thoughts. Kind Regards,
Q7 Development Team

blog comments powered by Disqus

Functional and UI Testing for Eclipse-based Applications

Test automation solution, handcrafted for Eclipse. Modern and easy to learn IDE.

Product Health Analytics and Support Automation Platform

End-user and product analytics solution that gives you rich insights into your application health & performance.

Eclipse-based IDE for the Fantom Programming Language

Full-featured Integrated Development Environment for emerging Fantom programming language.