My inspiration for this article has been a comment I received with regards to trying to organise an internal Rapid Software Testing course .
Someone made a comment that they felt that this ‘smacks of a gimmick’ but would be interested in finding out what people discover/get out of this in relation to the way we work.
The views expressed in this article are solely my own and are based upon my own experiences and knowledge of the testing profession.
Currently within the testing world there appears to be two schools of thought:
The traditional (standards) approach – driven by the ISTQB examination board (formerly the ISEB)
And there is the
This article is not about entering a debate to say one approach is better than the other and IMO I think a good balance is a mixture of the various approaches. There are many other approaches and concepts to testing than the two mentions above such as the Agile and the analytical approaches but the main discussions within the testing community appear to mainly refer to the two 'schools' listed above.
The principles and concept of context driven testing is that the emphasis is about thinking, experiencing and doing rather than assuming and making interpretation of what people believe the system should do. It is more about ‘hands-on’ and learning about the system as you test.
A common misconception is that there is no planning involved within rapid software testing and that it is based upon ‘free’ testing and it is without structure or discipline. In my experience and from using the material from the rapid software testing course there is more planning, structure and focused based testing than with any other approach. The introduction of session based testing in which testers have a mission and a goal to aim for during their testing session ensures that testing remains focused and on track.
The difference between the two approaches in that the standard approach is mainly used to define testing (test cases) before you have actually have access to the system under test. There is an inherent weakness in this approach in that assumptions are made that requirements, design and user needs are correct, accurate and not missing.
Once actual testing has started the majority of testers revert to working in a context driven way. They adapt the scripts they have written; they think of new ones and make decisions on what not to run. The context driven approach is to have some lightweight upfront planning which is ambiguous and allows the tester the freedom to approach the testing by using their logic and adapting as they learn more about the system. This allows the tester to build up knowledge of the system while executing, creating new tests and recording what they are testing. This is the basic definition of exploratory testing. Time is not wasted on creating tests that will never be run or maintaining a list of test scripts with incorrect test steps. It is about recording what is happening at the time testing happens and storing the results of that testing session. This then can if possible be automated and as a test never run manually again.
The difference between the approaches is that rapid software testing requires testers to think as they test and not just tick boxes. It forces the tester to question what they see and allows them to freely explore and discover new things about the system. It uses triggers (heuristics) to keep asking the question "Is there a problem here?"
It does this by comparing the product with similar product, looking at the history of the product or the claims made by the product. These are tests in which there is no yes or no answer it depends on the context and the thinking of the tester.
Is context driven testing a gimmick?
IMO it is not
It is a natural way to test products and it is the way testing has been done since it became a career choice. However people have not admitted (or will not admit) to following this approach or be aware that they are doing this.
The whole concept and approach of rapid software testing is to give it a name and provide useful skills and tools to improve this methodology which follows the thinking of context driven testing.
_______________________________________________________
FOOTNOTE:
After a lively discussion on twitter with James Bach I feel I need to clarify some misuse of definitions.
James gave a great description to show the differences between Rapid Software Testing and Context Driven:
Rapid Testing is a testing methodology that is context-driven.
But context-driven testing is not Rapid Testing.
After this 'revelation' I have made some minor changes to the original post.
John, this sounds like a great approach and you provided a decent high-level overview of both approaches and how they are different. Often there is not time to write test cases for every feature / module that is going into testing. This sounds like a good way to provide testing direction.
ReplyDeleteHi Steve,
ReplyDeleteSince you mentioned two schools of thought I figured I would share this great presentation by Bret Pettichord on Four Schools of Thought.
http://www.io.com/~wazmo/papers/four_schools.pdf
Cheers,
Lynn
"The difference between the two approaches in that the standard approach is mainly used to define testing (test cases) before you have actually have access to the system under test. There is an inherent weakness in this approach in that assumptions are made that requirements, design and user needs are correct, accurate and not missing. "
ReplyDeleteHi, John. In my experience, I've found that applications often have a longer lifespan the an employee's tenure, especially when dealing with off-the-shelf or third party developed software. I view documentation, whether it be requirements, designs, architectural diagrams, operations guides, etc. as valuable tools during times when you have high turnover rates among your employees.
I consider testing documentation as much a part of my job as testing software. We sometimes use test cases to accomplish this task.
I agree with you that you can't blindly use a process/pattern from a prior engagement because every application is unique. I try to discourage our testers from just writing down hundreds of scripts that have to be maintained just because that's what they have always done.