The last blog post looked at what critical and creative thinking was and which may be most useful during different stages of testing. This post will start by giving an overview of each stage and then look in depth at each of the stages.
This is the often forgotten about stage of testing in which the tester takes a little time to reflect on what they tested, did they do the best they could within the constraints they had? Could they have done things differently? This time could be spent reviewing the session notes and investigating issues they uncovered during the execution phase, that they thought could be bugs. It could be used to check if a bug was repeatable and then raised in a defect tracking system with full evidence of how to recreate. It could be looking at what may be useful to automate from what information was discovered. It could be adding new ideas from the notes for future testing. It is also the time to debrief and talk to others about what you did.
Stages of testing.
Before we start to look at how we can use these types of thinking within testing we need to breakdown the testing life cycle into manageable stages. This is not an expansive list of the many stages but the labels given are to give a high level overall view and simplify the classification for which style of thinking is most suited for each phase. It not meant to state that these are the only things we need to thing about in stages nor does it mean that there is not any cross over between stages. They are meant as ‘guide’ lines only and not as best practice, process or otherwise.
Documentation Review (Static Testing)
- Requirements
- High Level Design (HLD)
- Feature documents
- Specifications
- Standards
- Regulations
- Code review
- Walkthroughs
- Inspections
- Oracles
Documentation review does not just mean reviewing requirement documents it could mean reviewing code, database models, country laws and standards. Requirements are only ONE source of information (Oracle) Instead of going into great detail about what role testing plays in the documentation review stage there are many resources available on-line that can do a far better job than me. I highly recommend that people look at these links if they wish to know more about static testing and documentation review.
Cem Kaner - Testing Computer Software book - has an excellent section on document review
Ron Patton - Software Testing - Chapter 4 - examining the specification
* Disclaimer - I may not agree with all the approaches; processes and methods being suggested in the above links but they provide information about the diversity of what documentation review is in the context of software testing
Test Planning
- Test Ideas
- Automation – feature files (cucumber)
- Missions/Charters
- Mind mapping
The test planning stage is one which is much maligned with people within the testing community. There are people making all sorts of statements about the purpose of test planning with some saying to do as little as possible (Agile style) with others saying it should be wrapped in best practice and process and standards. In my own world there is a important need for planning but there is a case that says do not do too much. I feel that with test planning you should do enough to enable you to start doing some actual testing. Once you start testing you can then update and maintain your test plan based upon what you experience and discover. I try to align myself to James Bach's approach to test planning which can be found here page 21.
Test Execution
Now we come to the key element of testing, the part where the tester should engage their brain and start to ask questions of the software. I am talking about manual testing and specifically exploratory testing. There are many articles on what is exploratory testing and how to do it and rather than repeat information I will provide just one link to the excellent resource by Michael Bolton here. This is one page I do have bookmarked and come back to time and time again.
Test Analysis
- Bug investigation
- Defect reporting
- Repeatability
- Questioning
- Future Automation
- Debrief
- Future missions/Charters
This is the often forgotten about stage of testing in which the tester takes a little time to reflect on what they tested, did they do the best they could within the constraints they had? Could they have done things differently? This time could be spent reviewing the session notes and investigating issues they uncovered during the execution phase, that they thought could be bugs. It could be used to check if a bug was repeatable and then raised in a defect tracking system with full evidence of how to recreate. It could be looking at what may be useful to automate from what information was discovered. It could be adding new ideas from the notes for future testing. It is also the time to debrief and talk to others about what you did.
Test Reporting
- Dashboard
- Wiki – sessions
- Updating plan
- Qualitative
- Quantitative
Test reporting is a highly controversial subject within the testing community with lots of discussion about the use of metrics. There are some great articles out there on this subject and I am not going to attempt to summaries it here. If you want to understand more about the metrics debate then can I suggest you start with this blog post of mine. Then have a look at some of the following articles from Michael Bolton's resources.
To me the best way to do test reporting is the telling of a story using numbers to help support the story. A good way to do this is by the use of dashboards, a great example of this can be found here.
How does this apply to testing?
So now we have our testing stages we can start to see which style of thinking would be most suited for this stage, not forgetting that the other style will and should still be used, but the focus should be on the primary style of thinking for that stage.
The next article in this series will look in depth at documentation review and the style of thinking that may be most suited for this phase
No comments:
Post a Comment