A list of games taken to Testbash

Testbash in Brighton was yet again a highly organised and community based testing conference.

Signup for the Ministry of Testing Dojo to see the talks, which should be available soon.

At a pre-testbash meetup in Brighton I was asked by Rosie if I would not mind bringing some of my collection of games along. That may have had something to do with having a big interest in games and how they can be useful for testers to improve creative and critical thinking skills.

After the conference I had a few requests for a list of the games that I brought along to the Testbash gaming evening.  To provide a reference I have created this page to list the games I brought along and some I did not due to limited luggage space.  All the links are for the UK version of Amazon unless otherwise stated.

Dixit

A wonderful abstract communication game with highly detailed artwork.  The purpose of the game is to describe the card you have to enable some of the players to have a good idea of what your card is.  There is a twist, you want to communicate so only a few people get it, but provide enough information to make sure that no-one does not get it. If everyone guesses your card or no-one guess your card then no points are awarded. A useful game about communication and providing enough information. It comes with many expansions which can be seen when looking at the main link.

Quirkle

Quirkle is a cross between dominoes and the card game SET (See below).  You place tiles that meet a set of rules.  The tiles placed in a single line have to be either same colour, different shape OR same shape different colour.   You score double points for getting a set of six, a quirkle!  A great game but needs good lighting and a lot of space!  One of my current go to games for a bit of fun and lateral thinking skills.

Quirkle Cubes

Quirkle cubes takes that concepts of quirkle and adds a random element and the ability to use a different strategy.  Unlike the original quirkle you can now see other players hands and can choose on your turn to roll some of your cubes to change the hand you have. This version require a lot more thought about tactics.

SET

This seems to be the 'go to' testers game of choice.  A game of visual perception, where you have to find a set of three that matches the rules of the game.  A set is three cards where either the feature is the same or the feature is different on all cards. A feature is either colour, shape number or shade.

Fluxx & Star Flux

A card game of ever changing rules where the object is to have cards, keepers, that meet the goals.  However the goals can and do change along with the rules as well. I do have the board game version of this which is fun since the board changes as well as the rules and goals.

Fluxx Dice

Fluxx dice is an expansion for the fluxx card game which adds an extra dimemsion. By rolling the dice you change the draw and play rules every single hand. Adds a great new random dynamic to the game.

Ice Dice

Ice dice or ice pyramids is a set of dice and pyramids which can be used for a variety of games. The most common one for testers is to play Zendo. This game is where a master sets a rule and shows an example of the rule and an example that does not meet the rule.  The student has to try and work out the rule by building their own pyramids and ask if it meets the rule or not.  A wonderful deductive game to challenge the mind and expand the investigative skills of testers.

Colt Express

One of my favorite games at the moment.  Each player has a bandit trying to take loot from a train whilst avoid the marsh and other bandits.  Each player has aset of action cards that they select at random.  These actions cards such as punch, fire, move, steal and so on are player during a round.  A round has a series of turns.  Some of the actions are played face up for all to see and some are placed face down.  At the end of the turn section, the action cards are played one at a time and the action carried out for that player.  It is fun to watch as some thought goes into what actions to play depending on where your bandit is, however it never quite works out as your expect.  This to me has alot in common with coding, we create what we think works only to find it ends up in a big mess.  Highly recommend this game for all testers!

Blink

A fast paced game where there are no turns.  You try to place your cards down that match the play decks by matching either colour, shape or number.   The first to get rid of there cards wins.  An ideal game for improving your pattern spotting skills.  One of the reasons I got this game was based upon an article by James Bach - Quick Oracle - Blink Testing.

Frenzi

This card game needs a lot of space to play and can become very hectic.  All the cards are placed on the table and then they are turned to the other side, they are double sided cards. Each player gets a rule card which shows what they need to have facing up at the end of the round.  They could be looking for a color, number or shape.  The first round is colour, then shape and finally number.  A timer is set and everyone starts to turn cards over using one hand.  The aim is to have as many of the cards facing up that matches your own rule. This is a quick and face paced game where it quickly becomes manic! You need lots of room to play this game and it is based on quick visual perception.

Rory Story Cubes

Rory story cubes are little dice with pictures on each face.  The dice are rolled then the players one at a time pick up a cube and using the picture that is face up start to tell a story, they leave it on a cliff hanger for the next player to pick a cube and continue the story.  Once the last cube is selected the person who picks this provides an ending to the story. These cubes are great for creative play and improving story telling skills.

Story Wars

A wonderful game in which two teams battle it out to convince an impartial judge why their character(s) should win. They is magical lands, characters and special weapons to use.  This game is useful for those wanting to improve their influencing and convincing communication skills.

This is quite an expensive game at the moment, I picked it up in the USA for $15.   You can obtain a PDF to printout for free from the manufacturers website see here.

Disruptus

I have previously done an in depth review of this game on this blog spot. See here.

Dobble

A quick game of spotting the same item on a different card.  You place two cards face up and the first to spot the same image on both cards wins those cards.  A clever game where every card has an image that is on another card. What makes it tricky is the images could be bigger or smaller or rotated.  Another game that improves the visual perception of the players.

Loonacy

This is another visual percetion game where the players try to match one of the two characters in their hand with the game cards.  There are no turns and the first to get a match keeps goig until they have no cards left and are declared the winner.

Cubu

Cubu is an intense game of following a sequence of colours and numbered squares with the aim to get rid of all your cards before the other player does.  This game requires a lot of concentration to be able to workout which way would be best for your sequence and to throw a proverbial spanner in the works there are action cards which can force you to miss turns, pick up more cards and other actions that inhibit your chances of winning.

Interviews and Videos

I thought I would start the new year with a brief post about what I have been up to over the holiday period.

I was approached by the editors of Teatime with Testers magazine seeking permission to publish my current series of posts on testing skills - the first one has been published and you can download the magazine for FREE from here - http://www.teatimewithtesters.com/ .

During December Matt Young got in touch with me from the company Functionize to see if I would be willing to do a podcast with their CEO Ray Grieselhuber about "Why automation will not replace manual testing."  This ended up being a really good discussion in which everything from why automation can be useful, to a philosophical debate about what being human means.  We ended up chatting for so long that the article had to be split into two parts.  The podcast and articles can be found at the links below.

• http://www.functionize.com/blog/John-Stevenson-Test-Automation
• http://www.functionize.com/blog/Automated-Testing-John-Stevenson-Ray-Grieselhuber
• https://soundcloud.com/functionize/john-stevenson-on-test-automation - Podcast

During this period Richard Bradshaw started a great new concept of Whiteboard testing.  Considering I do not live that far from Richard I decided to pop over and we recorded a couple of sessions.  Links to these session can be found below:

A look at the test automation pyramid
Test Execution Model

I intend to record a few more whiteboard sessions in the near future.  If you have any great ideas for whiteboard sessions then please do record them and help support this wonderful initiative.

*EDIT*

Oh and I will be speaking in Brighton on March 11th at Testbash.  Hope to see you there.

Test Execution Model - updated

This is a follow on to the post I recently did about the automation pyramid that Richard Bradshaw gave at MEWT.

Following on from this some in the testing Twitterverse asked if Richard and I would do a video for the new initiative created by Richard called Whiteboard testing.  Instead of one video we decided to do two.  The first looked at the history of the test automation pyramid and you can watch it here: A look at the test automation pyramid.  The second video was a chance for me to present my own model which I have called the test execution model and can be seen here: - John Stevenson Test Execution Model.


After the video was posted I had a message from Dan Ashby who wanted to discuss more with me about the  test execution model and what he felt was missing.  We arranged a chat and discussed the model and broke down a few assumptions we were both making.  The outcome from this chat was the need to add a few extra parts to complete the test execution cycle.  With thanks to Dan I would like to present the next generation of the model.

The changes that Dan suggested for the model are as follows:

• As you test you turn some of your tacit knowledge into explicit knowledge which then can be added to your checks and become known known information.  This is represented by the flow from the testing arrow to the checking arrow at the top of the diagram.
• As you create more checks these in turn become oracles which you can use as heuristics for your future tests.  This is represented by the flow from the checking arrow to the testing arrow.

This then gives a cycle of test execution were the testing feeds into the checking which in turn can feed back into the testing.  It should be noted that is model is for test execution which does not only apply to only testing the product deliverable.  Testing is carried throughout the development cycle and as such testing activities can occur during artitecture & design discussion and during coding. Testing activities can and should occur anywhere during the development and deployment of the product and this model should be used in that context with regards to test execution.

Testing Skills #9 - Differentiating between wants and needs

One common discussion that keeps cropping up from testers saying that their managers are telling them that they 'want' this and that they 'want' that.  Normally in connection with 100% test automation or some other new shiny testing discussion point. A crucial skill a tester can have is the ability to separate what some one wants and what they really need.

It is common for people to not see the huge difference between these two words but having the ability to do so can improve your testing skill set.

The difference between the two is easy to explain with the following scenario.

Imagine that you have decided that you to be fitter and to do so you decide you want to be able to swim ten miles in one session.

Now ten miles is a lot and since the average human swims at about  two mph that is going to take about five hours to complete. You may never get around to meeting this want but you may come close.

Now one day you decide that you want to go sailing off the coast of Britain, you sail out to ten miles from the coast when suddenly the boat is hit by a huge wave and destroys it.  You are ten miles from the coast and if you do not make it back within five hours you will die due to the exposure to the cold water.  You now need to swim 10 miles in a single session.

This is the difference between wants and needs.  What people want is not necessarily what they need.

As a tester you should try and uncover what people need when they say they want this or that.  In the test automation example  it is useful to probe deeper and ask what is the problem they are trying to resolve.  By doing this you can get to the needs rather than the wants.

When the manager says "They want to automate all the testing" delve deeper to understand the problem the manager is trying to solve.  Ask "Why do they want to try and automate all of the testing?", "Uncover the needs so that you can understand what is being asked.

** The story about swimming is not my own and I cannot recollect where I first heard this.  If any of the readers can let me know where it is from I will add the correct attribution. 

Testing Skills # 8 – Being a Skeptic

 “Science is the organized skepticism in the reliability of expert opinion.”  Richard Feynman

The reason for this article is based upon a recent twitter conversation regarding how someone sometimes avoids skepticism due to its negative nature.  I fully understand what they mean when skepticism is used in way to attack a person or their character (ad-hominem argument).  The purpose of this article is to provide information about how useful having a healthy dose of skepticism is for those involved in software testing.

Many people have a belief that the purpose of testing is to prove that the software works as expected.  This is really a fallacy since testing cannot prove that the software will always work the way you expect. There is always an element of doubt in proving that it works due to the nature of software and how it is used. This is the same as in the field of scientific research where someone comes up with a theory based upon their experiments and then their peers using a fair bit of skepticism try to prove the theory wrong.  This is the crux of any scientific based research method.  It is not about producing your theory it is about applying critical thinking to your theory as to why it can be wrong.   Testing software is similar in its approach it is very difficult to inform someone, who matters that the software is working. We can provide useful information about its behavior and what it actually doing.

As a tester you need to apply critical thinking to your testing and to the evidence you produce.   This is where being able to look at what you are doing and what information you find will a fair degree of skeptical thought.  The scientific method is a useful skeptic tool to apply to your testing and to any other information that people provide to you as ‘fact’.  One approach is the use of FiLCHers:

If you look at each of these headings you can see that it is about trying to find evidence where your theory and experiment could be wrong, acting as a skeptic.

This is a vital skill for testers to possess to ensure that their testing is unbiased and factually correct. If you feel the skeptic is lacking then look at ways you can improve it.

Here are some suggestions to help you become a better skeptic:

• Learn about logical arguments and fallacies
• This page has a substantial list of them - http://changingminds.org/disciplines/argument/fallacies/fallacies_alpha.htm
• This book is a wonderful illustrated book proving examples of fallacies  and bad arguments - http://www.amazon.co.uk/Illustrated-Book-Bad-Arguments/dp/1615192255/

• Practice using skepticism – apply it to your work and the work of others.
• Use James Bach’s’ , Huh, Really, So technique - http://itknowledgeexchange.techtarget.com/software-quality/james-bach-on-critical-thinking-huh-really-so/

• Use reflection to help improve your skeptic skills.
• See my previous post on reflection - http://www.steveo1967.blogspot.com/2015/10/testing-skills-7-reflection.html 

If you have some suggestions of your own to help others in their journey to being a great skeptic please let me know by using the comments section.

The Laws of Sport and Automation

I have had this idea in my mind and on my backlog for quite a while.  It was only after speaking at MEWT in Nottingham that I felt I really should get around to writing it. 

There are many debates in the software development world about ‘test automation’ and how we can ‘automate all of the testing’.  I am in the context of this article ignoring the difference between testing and checking, more details of this discussion can be found here - Testing and Checking Refined . However some of my ideas and concepts will touch on the difference between checking and testing. 

Many have put forward arguments about automating what we know and if we have defined requirements up front then it should be possible to automate these.  My counter to this is that in many sports there are well defined upfront requirements (laws) of how the game should be regulated.  For example the laws of football (Soccer to those outside of Europe) can be found online here: FIFA Laws of the Game.  If this is the case and these requirements are defined upfront then why do we not have automated referees? I asked this question on twitter and some of the responses gave reasons due to the psychical limitations, such as battery power unable to run and so forth.  My line of thought is on how these requirements can, and are, interpreted. 

Looking deeper in to the football laws of the game it can be seen there are many ambiguous statements which given the current state of AI, at the time of the publication of this article, I feel are impossible to automate. For example on page 39 it states the following as a reason for a player to be cautioned.

“unsporting behavior”

What does this mean?  Page 125 attempts to define this with a list of what constitutes unsporting behavior.  One of this in particular I found interesting, it is one based on human nature of trying to con or cheat

“attempts to deceive the referee by feigning injury or pretending to have been fouled (simulation)”

This I feel would be a common sense decision made by the referee. How could an automated system know if is fake or not?  Then again how would the ref know?  It it is a common sense decision, being made depending on a multitude of factors and contexts.

How about this one? 

“acts in a manner which shows a lack of respect for the game”

What would count as lack of respect?  A player who in the last second of the game lets in a goal that allows the opposition to win the title. The player shows human emotion and frustration, there is a fine line between emotion and respect or the lack of it?  

My issue with this automation debate is that at this time it is not possible to automate common sense and multiple contexts in the decision making process that a referee has to go though in their thinking process. 

For example a team is winning 20 – 0 a machine would continue to officiate the game in accordance to the strict letter of the law.  Whereas a human referee would allow some flexibility in the interpretation of the rules.   They will allow some aspects of empathy to be applied to the game.  Is it yet possible to automate empathy? 

James Christie made a valid point on twitter that the reason in the majority of sports they are called laws and not rules is that:

“rules are detailed and specific whilst laws can be based on vague principles, which require interpretation and judgment. “

This makes sense since most countries have courts where lawyers debate how the laws of the land can or should be interpreted.  Then a jury, judge or set of judges make a decision based upon the arguments presented. Another case of were the requirements are listed and known but given current AI limitations would be impossible to fully automate. Even though we know that human beings are flawed in the judgement that are made, would using an automated judgement machine be any less flawed, if at all possible to produce?

Returning back to the laws of sport and how ambiguous those laws are we can look at the laws of Rugby Union

Looking at the beginning of the laws on page 21 there is guidance on how the laws should be applied:

The Laws must be applied in such a way as to ensure that the Game is played according to the principles of play. The referee and touch judges can achieve this through fairness, consistency, sensitivity and, at the highest levels, management.”

How would you automate sensitivity in this context?

According to the Oxford English Dictionary this in this context is defined as:

“A persons feelings which might be easily offended or hurt”

Add into that equation “fairness”, we are now journeying down the automation rabbit hole.

Looking at the laws regarding fair play and the guidance that the document provides for foul play (Law 10) section m gives the following guidance.

“Acts contrary to good sportsmanship. A player must not do anything that is against the spirit of good sportsmanship in the playing enclosure”

What constitutes “the sprint of good sportsmanship”? How do you clarify between intentional and unintentional behavior?   Again I am uncertain if this kind of decision could be automated.

If we look at the laws of Rugby League we can see similar issues in how difficult it can be for the laws to be interpreted. Rugby league was one of the early adopters of video technology to help assist the referee in the game.  This is what Michael and James in their article would define as tool assisted testing.  In this case a video referee can review certain decisions via the use of video technology. 

Looking at the definition of a forward pass.

“is a throw towards the opponents’ dead ball line”

How do you define this in the context of a fast moving game?  Under the section 10 which offers some guidance there is a distinction between deliberate and accidental forward passes.  How do you make a distinction between these two actions? Also would an automated system be able to deal with factors such as the momentum of the player and the wind moving the ball.  Yes they could process information quicker than a human could but would it be right?

This is not to say that referees are not fallible and there are many instances in sport of them making mistakes; however people are aware of this and can accept that fact.  Would people be so willing to accept a machine making similar mistakes based upon our biases that machine are not fallible?

Many sports are implementing some level of automated systems which are used to aid the referees.  

• Tennis has been using Hawkeye since 2002 
• Football has started to implement goal-line technology
• Cricket uses the Umpire Decision Review System 

It is interesting to note that each of these automated systems have had some controversy regarding their accuracy and success especially with the cricket system.

To conclude when people discuss test automation and attempt to automate as much as possible there is a need to step back and think critically. Automation in software development has a place and is a useful tool to use, however, it should not be thought of as an alternative to testing as applied by a human being.  Even when you think you have the requirements nailed down they are words and as such are open to a multitude of interpretations.  Using a mixture of automation, tool assisted testing and human testing in a ratio that adds value to the quality of the product being delivered is a more thoughtful approach rather than the mantra of we can “automate all the testing effort.” Going forward we need to be thoughtful of what machines can do and what they cannot do.  This may change as technology progresses but as of the publication of this article there are big limitations in automation.

Testing Skills #7 - Reflection

What type of learning do you need to engage in today?

Do you need to learn facts?

Or

Do you need to analyse and critically think about what you need to learn or have learned?
Neither of these ways of learning are wrong and each has its place in your learning journey.  Knowing which to use and when is a great tool for a tester to have.

Learning facts is defined as a 'surface approach ' style of learning and writing and thinking critically about what your have learned can be defined as a 'deep approach'.

Reflection is important in software testing and you need to be able to communicate what you have found and uncovered when testing.  If you have no opportunities,  in your organization, to be able to communicate with others then you may find you have a lot on tacit knowledge but little in the way of explicit.  It is vital to set aside time for members of the team to discuss what new information they have uncovered or learned when testing.  This allows them to use reflective reasoning to turn tacit into explicit and gives the person providing the information a sense of purpose and achievement in what they have learned.

To improve your own self-reflection produce a reflective action plan. By doing this you are committing to your self what you intend to do and engage critical thinking skills in doing so. Make sure your plan has goals and deadlines that you can commit to.  Make this deadlines and goals public it encourages you to keep to them.  Using a personal Kanban board can help you achieve this.

"First, the deep and surface approaches are not personality traits or fixed learning styles.  Students adopt an approach which is related to their perceptions of the task."

Marton and Säljö - Student Approaches to Learning - University of Oxford

When you need to think deeply about what you have been studying then writing down you own thoughts and understanding is a good way for you to be able to see what you have remembered.  If you only need to learn information or facts then other models would be more suited.

Writing down you thoughts on what you have been attempting to learn enables you to better remember or apply what has been studied.  Reflection is about doing your own internal self-assessment  of what you have been learning.  This assessment enables you make what you know internally, tacit knowledge, and attempting to make it clear and detailed explicit knowledge.  To do this you need opportunities to talk to others verbally or by writing down what you have studied to invoke critical thinking.

"In general, writing appears to be suitable for tasks where the aim is fostering understanding, changing students' conceptions and developing their thinking skills, but less suitable if the goal is the simple accumulation of factual information"

Conceptualizing and Measuring Knowledge Change due to writing

Schumacher & Gradwohl Nash (1991)

One way in which can improve your learning is to use reflection:

"Reflection is an active process whereby the professional can gain an understanding of how historical, social, cultural and personal experiences have contributed to professional knowledge and practice "(Wilkinson, 1996).  

You learn from your experiences and to make this happen you need to engage in reflection.  If you think about what you are doing or what you have experienced you are engaging in self-reflection. This approach helps you to turn learning into knowledge and it is vital in improving your own knowledge and skills.  Reflection is really about looking back at a situation, thinking about it, critically and learning from the experience, then using that new knowledge to help in future similar situations.  This in many ways is similar to the approach you will taking when engaging in testing activities.

"The process of reflective writing leads to more than just a gain in your knowledge; it should also challenge the concepts and theories by which you make sense of knowledge. When you reflect on a situation, you do not simply see more, you see differently. This different way of viewing a situation is reflected in statements about a commitment to action. Action is the final stage of reflection. "

Learning through reflection(Word doc)?