I was fortunate to be invited as a speaker at Testbash this year and had a wonderful yet tiring time. There were lots of community based activities alongside the usual presentations. The following is based upon some of my notes taken during the event and from my ever fallible memory.
This year Testbash made a change and decided to have a day of workshops however before that there was a pre-Testbash workshop meetup. Walking in there were many familiar faces and some new ones, it felt like meeting up with old friends. At these kinds of events I try to make an effort to sit with people I have never spoken to or met before. There is nothing worse than coming to a meetup for the first time and not knowing anyone or being too shy to talk to others. Sometimes it can be intimidating for those who are new and seeing everyone engaged in conversation as if they have known each other for years.
One great aspect of many testers is their ability to make friends and be quite social. I sat with one group of people who had traveled from Glasgow which consisted of Paul, Christine and sorry I have forgotten the other ladies name. The conversation centered on testing in financial organizations and trying to implement what they had learnt on the RST - course they had attended that day with Michael Bolton. We had an interesting discussion about starting small and expanding out to others and implementing by showing success. I talked about my experiences in doing this and we agreed that to implement RST learning in big organizations can be daunting but possible.
After this chat I brought out a few games I by chance had in my bag. We settled on playing Loonacy which is a simple matching game with no turns. This is a fun game and Christine was really good, Paul not so good. Then it was time to get my beauty sleep for the big day of workshops.
Building an Itinerary for Exploratory Testing - Karen Johnson
The first workshop of the day was by Karen Johnson called ‘Building an Itinerary for Exploratory Testing’. This workshop was packed with content and to really do it justice would require a lot more than two hours. I have attended previous workshops by Karen and always feel relaxed, her gentle style of teaching is amazing and she makes everyone feel inclusive. This as a presenter is a difficult task to achieve and Karen is one of the best I have experienced. In this workshop it was no different. The first part of the workshop involved learning about three different techniques that can help you build up an itinerary for exploratory testing. Karen started by explaining that sometimes we get stuck coming up with ideas for exploratory testing. Karen stated there are tools available that can help you generate exploratory testing ideas, such as personas, heuristics and tours.
The first tool/technique that Karen introduced was personas. This is where you place yourself into a certain type of person or user. Rob Lambert has a great article on using personas in testing. Personas are a useful way to place yourself in to the mindset of somebody else and can provide you with different perspectives on how to test the product. One key aspect of personas is that they should not just be a one line description; it should be a story about the person with some background information.
A couple of tools mentioned during this part of the workshop for creating personas included the following:
- Fake Name Generator - http://www.fakenamegenerator.com/
- Random User Generator - https://randomuser.me/
Next up from Karen was Heuristics, mainly ones using mnemonics to help trigger test ideas. I have in the past found many of these useful when testing. Karen mentioned she has a set of cards that contain a set of these heuristics and mnemonics.
These cards can be found on her website - http://karennicolejohnson.com/2012/07/testing-mnemonics-as-a-card-deck-v2/. I created an expanded set based upon the idea form Karen - http://steveo1967.blogspot.com/2013/06/test-ideas-cue-cards.html
Another resource that Karen mentioned was the use of The Quality Tree Cheat Sheet - by Elisabeth Hendrickson, James Lyndsay , and Dale Emery.
People found this one very interesting and useful
Others mentioned included:
During this section Karne mentioned Test charters and recommended the book by Elisabeth Hendrickson called Explore it - A review of the book can be found here: - http://steveo1967.blogspot.com/2013/11/book-review-explore-it-by-elizabeth.html.
In this book Elisabeth has a template for exploratory testing charters:
To discover (information)·
Target: Where are you exploring
Resources: What resources will you bring with you
Information: What kind of information are you hoping to find?
The final tool/technique that Karen introduced was testing tours. This is where you are given a scenario and then use this to guide your testing. First up was Michael Kellys FCC CUTS VIDS tour - http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html
- Feature tour: Move through the application and get familiar with all the controls and features you come across.
- Complexity tour: Find the five most complex things about the application.
- Claims tour: Find all the information in the product that tells you what the product does.
- Configuration tour: Attempt to find all the ways you can change settings in the product in a way that the application retains those settings.
- User tour: Imagine five users for the product and the information they would want from the product or the major features they would be interested in.
- Testability tour: Find all the features you can use as testability features and/or identify tools you have available that you can use to help in your testing.
- Scenario tour: Imagine five realistic scenarios for how the users identified in the user tour would use this product.
- Variability tour: Look for things you can change in the application - and then you try to change them.
- Interopeability tour: What does this application interact with?
- Data tour: Identify the major data elements of the application.
- Structure tour: Find everything you can about what comprises the physical product (code, interfaces, hardware, files, etc...).
This was followed by James Whittaker exploratory testing tours
- The Guidebook Tour
- Guidebooks for tourists identify the best hotels, the best bargains, and the top attractions, without going into too much detail or overwhelming a tourist with too many options. The analogous artifact for exploratory testing is the user manual, whether it is printed or implemented as online help (in which case, I often call this the F1 tour to denote the shortcut to most help systems). For this tour, we will follow the user manual’s advice just like the wary traveler, by never deviating from its lead.
- The Money Tour
- Every location that covets tourists must have some good reasons for them to come. For Las Vegas, it’s the casinos and the strip, and for Egypt it’s the pyramids. For exploratory testers finding the money features leads directly to the sales force. Sales folk spend a great deal of time giving demos of applications and are a fantastic source of information for the Money tour. To execute the tour, simply run through the demos yourself and look for problems. As the product code is modified for bug fixes and new features, it may be that the demo breaks and you’ve not only found a great bug, but you’ve saved your sales force from some pretty serious embarrassment.
- The Landmark Tour
- As a boy growing up in the fields, meadows, and woods of Kentucky, I learned to use a compass by watching my older brother. The process was simple. Use the compass to locate a landmark (a tree, rock, cliff face, and so forth) in the direction you want to go, make your way to that landmark, and then locate the next landmark, and so on and so forth. As long as the landmarks were all in the same direction, you could get yourself through a patch of dense Kentucky woods.
- The Landmark tour for exploratory testers is similar in that we will choose landmarks and perform the same landmark hopping through the software that we would through a forest. Choose a set of landmarks, decide on an ordering for them, and then explore the application going from landmark to landmark until you’ve visited all of them in your list. Keep track of which landmarks you’ve used and create a landmark coverage map to track your progress.
- The Intellectual Tour
- I was once on a walking tour of London in which the guide was a gentleman in his fifties who claimed at the outset to have lived in London all his life. A fellow tourist happened to be a scholar who was knowledgeable in English history and was constantly asking hard questions of the guide. He didn’t mean to be a jerk, but he was curious, and that combined with his knowledge ended up being a dangerous combination … at least to the guide. When applied to exploratory testing, this tour takes on the approach of asking the software hard questions. How do we make the software work as hard as possible? Which features will stretch it to its limits? What inputs and data will cause it to perform the most processing? Which inputs might fool its error-checking routines? Which inputs and internal data will stress its capability to produce any specific output?
- The FedEx Tour
- FedEx is an icon in the package-delivery world. They pick up packages, move them around their various distribution centers, and send them to their final destination. For this tour, instead of packages moving around the planet through the FedEx system, think of data moving through the software. During this tour, a tester must concentrate on this data. Try to identify inputs that are stored and “follow” them around the software. For example, when an address is entered into a shopping site, where does it gets displayed? What features consume it? If it is used as a billing address, make sure you exercise that feature. If it is used as a shipping address, make sure you use that feature. If can be updated, update it. Does it ever get printed or purged or processed? Try to find every feature that touches the data so that, just as FedEx handles their packages, you are involved in every stage of the data’s life cycle.
- The Garbage Collector’s Tour
- Those who collect curbside garbage often know neighborhoods better than even residents and police because they go street by street, house by house, and become familiar with every bump in the road. However, because they are in a hurry, they don’t stay in one place very long. For software, this is like a methodical spot check. We can decide to spot check the interface where we go screen by screen, dialog by dialog (favoring, like the garbage collector, the shortest route), and not stopping to test in detail, but checking the obvious things (perhaps like the Supermodel tour). We could also use this tour to go feature by feature, module by module, or any other landmark that makes sense for our specific application.
- The Bad-Neighborhood Tour
- Every city worth visiting has bad neighborhoods and areas that a tourist is well advised to avoid. Software also has bad neighborhoods—those sections of the code populated by bugs. Clearly, we do not know in advance which features are likely to represent bad neighborhoods. But as bugs are found and reported, we can connect certain features with bug counts and can track where bugs are occurring on our product. Because bugs tend to congregate, revisiting buggy sections of the product is a tour worth taking. Indeed, once a buggy section of code is identified, it is recommended to take a Garbage Collector’s tour through nearby features to verify that the fixes didn’t introduce any new bugs.
- The Museum Tour
- Museums that display antiquities are a favorite of tourists. Antiquities within a code base deserve the same kind of attention from testers. In this case, software’s antiquities are legacy code. Older code files that undergo revision or that are put into a new environment tend to be failure prone. With the original developers long gone and documentation often poor, legacy code is hard to modify, hard to review, and evades the unit testing net of developers (who usually write such tests only for new code). During this tour, testers should identify older code and executable artifacts and ensure they receive a fair share of testing attention.
- The Back Alley Tour
- In many peoples’ eye, a good tour is one in which you visit popular places. The opposite of these tours would be one in which you visited places no one else was likely to go. In exploratory testing terms, these are the least likely features to be used and the ones that are the least attractive to users. If your organization tracks feature usage, this tour will direct you to test the ones at the bottom of the list. If your organization tracks code coverage, this tour implores you to find ways to test the code yet to be covered.
- The All-Nighter Tour
- Also known as the Clubbing tour, this one is for those folks who stay out late and hit the nightspots. The key here is all night. Exploratory testers on the All-Nighter tour will keep their application running without closing it. They will open files and not close them. Often, they don’t even bother saving them so as to avoid any potential resetting effect that might occur at save time. They connect to remote resources and never disconnect. And while all these resources are in constant use, they may even run tests using other tours to keep the software working and moving data around. If they do this long enough, they may find bugs that other testers will not find because the software is denied that clean reset that occurs when it is restarted.
- The Supermodel Tour
- For this tour, I want you to think superficially. Whatever you do, don’t go beyond skin deep. This tour is not about function or substance; it’s about looks and first impressions. During the Supermodel tour, the focus is not on functionality or real interaction. It’s only on the interface. Take the tour and watch the interface elements. Do they look good? Do they render properly, and is the performance good? As you make changes, does the GUI refresh properly? Does it do so correctly or are there unsightly artifacts left on the screen? If the software is using color in a way to convey some meaning, is this done consistently? Are the GUI panels internally consistent with buttons and controls where you would expect them to be? Does the interface violate any conventions or standards?
- The Couch Potato Tour
- There’s always one person on a group tour who just doesn’t participate. He stands in the back with his arms folded. He’s bored, unenergetic, and makes one wonder exactly why he bothered paying for the tour in the first place. A Coach Potato tour means doing as little actual work as possible. This means accepting all default values (values prepopulated by the application), leaving input fields blank, filling in as little form data as possible, never clicking on an advertisement, paging through screens without clicking any buttons or entering any data, and so forth. If there is any choice to go one way in the application or another, the coach potato always takes the path of least resistance.
- The Obsessive-Compulsive Tour
- OCD testers will enter the same input over and over. They will perform the same action over and over. They will repeat, redo, copy, paste, borrow, and then do all that some more. Mostly, the name of the game is repetition. Order an item on a shopping site and then order it again to check if a multiple purchase discount applies. Enter some data on a screen, then return immediately to enter it again. These are actions developers often don’t program error cases for. They can wreak significant havoc.
Once this was finished Karen then got everyone into groups and asked them to come up with test ideas using these techniques and tools to test a website or a mobile application.
Observing the groups it was interesting to see that some used mind maps to document their ideas, others used lists, and some created test charters using the template by Elisabeth Hendrickson and one group story to create user stories. Whilst observing it became clear that many of the groups found the cheat sheet a useful tool for coming up with ideas whilst others found the tours a good way to come up with novel ideas to test the application.
At the end Karen brought all the groups together to do a debrief. During this part of the workshop many came to the conclusion that each of the techniques that had been introduced provided them with far more test ideas that they could have thought of without the techniques or tools. Some of the discussions focused on which technique people preferred with many having a presence for either tours or heuristics. It was asked why people did not like the personas as much and the majority stated it was hard to come up with people for the sessions; however in their own contexts it may be easier.
To conclude this was a fantastic introduction to some great exploratory testing techniques given by an experienced and knowledgeable presenter. The style of training that Karen used in the workshop was a calm and unassuming one which made everyone warm to her and the session. If you get an opportunity to attend a workshop by Karen I would highly recommend you do so.
Creative and Critical thinking workshop
The next workshop I attended was the Games and Tools to Encourage Creative and Critical Thinking within Testing one. When I say attended I meant presented. I cannot say much about the workshop in terms of attending it; however I do hope people enjoyed it.
For more details of some of the content I wrote a series of article (seven) the first of which can be found here: - http://steveo1967.blogspot.com/2013/03/creative-and-critical-thinking-and.html
Some of the tools and techniques used during the workshop:
- Disruptus - http://steveo1967.blogspot.com/2014/01/using-games-to-aid-tester-creativity.html
- SCAMPER - http://www.mindtools.com/pages/article/newCT_02.htm
- FiLCHeRS - http://www.csicop.org/si/show/field_guide_to_critical_thinking/
- Huh, Really, So – Video - http://www.youtube.com/v/8TX6rzz60xQ
- The 5 Whys - http://en.wikipedia.org/wiki/5_Whys
- Rule of 3 – Weinberg - http://s0ftwaretesting.livejournal.com/1430.html
- Thinkpak - http://www.amazon.com/Thinkpak-A-Brainstorming-Card-Deck/dp/1580087728
- Disney method - http://www.idea-sandbox.com/blog/disney-brainstorming-method-dreamer-realist-and-spoiler/
- Technique for producing ideas - http://www.scribd.com/doc/242701702/A-Technique-For-Producing-Ideas-Poster-pdf#scribd
- Story cubes - https://www.storycubes.com/
After this session I needed to relax a little and gather my thoughts so I did not attended the next workshop
Next there was an open session where we played some games.
I introduced the following games:
- Zendo http://www.looneylabs.com/rules/zendo using Ice Dice - http://www.looneylabs.com/games/icedice
- Loonacy - http://www.looneylabs.com/games/loonacy
- Dobble - http://www.amazon.co.uk/Asmodee-Dobble-Card-Game/dp/B00NELITFK
- SET - http://www.setgame.com/set
If you want to learn more about these games then if you see me at testing events let me know.
In the evening there was another meetup “Pre Testbash Meetup” where over 60 testers got together and chatted. The best part of Testbash is not the workshops or presentation but the community feeling of these meetups. I met old friend and found new ones. Guna (https://twitter.com/alt_lv) was a shining star and someone who is fun to talk about anything not just testing. We end up talking about my tee-shirt whichwas based upon the work by the artist Jenny Parks - http://www.jennyparks.com/. In fact Guna tweeted afterwards about the artist, what a great memory! I bumped into many testers I had not seen in a while including Vernon ‘tutu’ Richards (https://twitter.com/TesterfromLeic), Danny Dainton (https://twitter.com/DannyDainton) Dan Billing (https://twitter.com/thetestdoctor) and many others sorry if I have not mentioned you.
One interesting person I chatting with who I feel is a future testing star was Emma Keaveny (https://twitter.com/EmJayKay80). Her story into testing is a wonderful and emotional journey. I do hope one day Emma will start a blog post and tell others her inspiring journey. I am sure it will be an interesting read and contain colorful language.
I did not expect this post to become so big so it has become two parts, the workshops and the key presentations. Watch out for the key presentations post soon.