A thought: the three layers of testing
After attending a talk about agile testing, I have come to the conclusion that in any project we have to take into account three layers of testing:
- Business specification testing
This is, we have to ensure the application is doing exactly what the customer wants. This represents the most important layer to be tested and it should be our priority number one. I always say it is better to spend hours, even days, writing specification and checking with customers that this is what they want than spending 5-10 days developing something wrong. Testers and developers write the stories and scenarios in BDD or tabular style focusing on the business point of view and I try to ignore technical aspects. Although testers can always test manually the behaviour of the application following the steps defined in the scenarios, automating the specification using a tool such as cucumber or fitnesse is a bonus in order to avoid mistakes or skip tests.
- Technical testing
This is tests of technical aspects such as exceptions (server down, using alphas instead of numbers, invalid dates…). This is mainly covered with unit testing.
- Visual testing
This tests are more difficult to automate (colors or element positions using CSS on different browsers) and most of them have to be done manually. Fortunately, this can be done first by developers themselves while developing; second by another developer if you are doing pair-programming or code reviews; third by the rest of the developers when the latest build is deployed on a development server; fourth by testers (QA); five by customers when it is showedcase; and sixth by QA again when it is tested for the release.
Another quote I heard once: “don’t try to test 100% of the application, it is almost impossible and you are going to get crazy”