This is my response to an blog article written by Paul Gerrard.
http://gerrardconsulting.com/index.php?q=node/599
I was going to post this as a comment but thought it would be better as a separate blog article
.I am not sure if I agree with entirely what Paul was saying but that is the point of the good blog article. I will say I do entirely agree with his conclusion that we have to have our eyes open and your brain switched on. There are methods that can be used to prevent the ‘quitting’ process and the rambling around in the dark approach to exploratory testing but I think that would be an entirely different article.
However I would suggest people try searching for article on avoiding (or being aware of) bias, cognitive research methods, focusing and defocusing skills. Another one to look at is Air Traffic Control work patterns; they work in time boxed shifts is this similar to session based testing?The point I want to make is the issue Pauls makes about domain knowledge and the usefulness that scripts may bring is an important one.
I am not in the camp that we should abandoned scripts and a lot of the people I communicate with are not saying that either. I feel there are a lot of Chinese whispers with regards to views of some people on the use of scripted tests. I cannot recall anyone saying to me that we must abandon scripts in favour of just doing exploratory testing (Is that a bias and I am deliberately missing or not noticing information?) We also can train ourselves not to quit using a variety of cognitive processes especially the use of checklists and heuristics. These ‘tools’ enable us to counter the quitting instinct but triggering new paths, observations and comparisons.
Testing is not just about finding things it is about asking questions and forming theories based on the answers (evidence) given while experiencing the software. This may lead to more questions and further evaluation and even more re-evaluation of what you already thought debunking and disproving your theories. Finding bugs is a side effect of this approach, a very useful side effect but it is not the sole purpose of testing.
There is a term used within society (especially the social science community) which is ‘recipe’ knowledge, this is often devalued by academics since it is a step by step instruction for learning something. In the everyday world context recipes tell you what to use, what you will need (ingredients) and exactly what procedures to follow, this sound familiar to scripts in the testing world. These recipes can provide important foundations for acquiring or developing skills or as we would say in the software development world learning domain knowledge? People using a recipe as we know may not follow it exactly, they may taste the product and adjust it for their own personal taste so the will move away from the script. However we should not pretend that learning a recipe is the same as learning a skill.
If we for example look at baking, this requires a ‘knack’ which can only come from experience (if you have tried baking bread you will understand this) like qualitative analysis, baking also permits creativity and the development of your own styles.
The skilled tester at some point, like the experienced chef, may stop using the recipe book and start to experiment and explore different tastes and ways to discover more and hopeful improve their skills. At the same time the recipe (script) remains a useful tutorial for the newcomer to the art.
Some of the content used here is taken from the following book:
Qualitative Data Analysis: A User-friendly Guide for Social Scientists - Ian Dey
http://gerrardconsulting.com/index.php?q=node/599
I was going to post this as a comment but thought it would be better as a separate blog article
.I am not sure if I agree with entirely what Paul was saying but that is the point of the good blog article. I will say I do entirely agree with his conclusion that we have to have our eyes open and your brain switched on. There are methods that can be used to prevent the ‘quitting’ process and the rambling around in the dark approach to exploratory testing but I think that would be an entirely different article.
However I would suggest people try searching for article on avoiding (or being aware of) bias, cognitive research methods, focusing and defocusing skills. Another one to look at is Air Traffic Control work patterns; they work in time boxed shifts is this similar to session based testing?The point I want to make is the issue Pauls makes about domain knowledge and the usefulness that scripts may bring is an important one.
I am not in the camp that we should abandoned scripts and a lot of the people I communicate with are not saying that either. I feel there are a lot of Chinese whispers with regards to views of some people on the use of scripted tests. I cannot recall anyone saying to me that we must abandon scripts in favour of just doing exploratory testing (Is that a bias and I am deliberately missing or not noticing information?) We also can train ourselves not to quit using a variety of cognitive processes especially the use of checklists and heuristics. These ‘tools’ enable us to counter the quitting instinct but triggering new paths, observations and comparisons.
Testing is not just about finding things it is about asking questions and forming theories based on the answers (evidence) given while experiencing the software. This may lead to more questions and further evaluation and even more re-evaluation of what you already thought debunking and disproving your theories. Finding bugs is a side effect of this approach, a very useful side effect but it is not the sole purpose of testing.
There is a term used within society (especially the social science community) which is ‘recipe’ knowledge, this is often devalued by academics since it is a step by step instruction for learning something. In the everyday world context recipes tell you what to use, what you will need (ingredients) and exactly what procedures to follow, this sound familiar to scripts in the testing world. These recipes can provide important foundations for acquiring or developing skills or as we would say in the software development world learning domain knowledge? People using a recipe as we know may not follow it exactly, they may taste the product and adjust it for their own personal taste so the will move away from the script. However we should not pretend that learning a recipe is the same as learning a skill.
If we for example look at baking, this requires a ‘knack’ which can only come from experience (if you have tried baking bread you will understand this) like qualitative analysis, baking also permits creativity and the development of your own styles.
The skilled tester at some point, like the experienced chef, may stop using the recipe book and start to experiment and explore different tastes and ways to discover more and hopeful improve their skills. At the same time the recipe (script) remains a useful tutorial for the newcomer to the art.
Some of the content used here is taken from the following book:
Qualitative Data Analysis: A User-friendly Guide for Social Scientists - Ian Dey
I think Paul named the post incorrectly - the post is not about scripted vs exploratory AT ALL.
ReplyDeleteHe makes few good points our genetic and evolutionary pre-dispositions as humans to give up easily when asked to look for things.
He makes reference of "why we make mistakes" book and goes on a interesting route to explain his own experience in aircraft security.
But then takes an abrupt turn and goes on to talk about scripts vs exploratory.
Instead of words like experience or knowledge (domain) - I prefer the words like "training", "apprenticeship" to overcome human cognitive limitations.
Shrini
>>> Testing is not just about finding things it is about asking questions and forming theories based on the answers (evidence) given while experiencing the software.
ReplyDeleteTesting is ALL about finding things - various kinds of things. Why do you ask questions? because you are looking for things, information, patterns and models.
Looking for things - is a key activity - questioning, forming models, patterns - works like tool for this activity.
Looking for things LEADS to questioning (not other way around)
For the record, I've often advocated the elimination of all *click*by*click* scripting (which is what I first learned as a manual test script). If, however, you take a less stringent definition of "scripting" that includes, for example, jotting down a list of tasks, functions, paths, user objectives, or other "notes to self" then I quickly retreat from my advocacy of elimination of scripted testing and instead roll my eyes and mumble something about frequently ambiguous & often idiotic naming conventions in "testerland".
ReplyDeleteWhile I was reading both the initial article and this response, I had several "ranty responses" in mind, but all became rather irrelevant when I realized that I didn't know what definition of "scripted" was being used in the Paul's post.