The Emotional Tester (Part 2)

The first part of this blog looked at how our emotions could affect how we test. This second part will look at how we could capture our feelings when testing and could this provide us with any useful information about the product we are testing. Could it prove to be a useful oracle when testing?

On twitter @testsidestory said the following:

That is done regularly in usability labs: capture emotions and facial expressions of the users as they use the s/w

This was in response to a question that I posted on twitter:

…. - what I am thinking is that we need to capture our mood when #testing it could indicate a problem in the s/w…

The concern with this is that it would be very expensive to implement for the majority of people. I thought how we could implement a system that could capture emotional state and be effective and inexpensive.

One idea I had was to use a concept from the book Blink by Malcolm Gladwell, in which Malcolm talks about how important our initial emotion/reaction is when we first encounter something. There is a discussion about how often our ‘gut reaction’ proves to be correct and he uses an example of a statue that a gallery had bought after a lot of scientific experts, who had tested the statue, had said the statue was genuine. A couple of art experts who got to see the statue before it was unveiled in private viewings had a ‘feeling; that there was something wrong about the statue, their initial gut reaction was telling them it was a fake. Several months latter it was discovered to be a fake.

The above is a concise retelling of the story within the book, however why did the scientific experts get it so wrong? Could it be that conformation bias played a part? The scientific experts wanted so much to believe that it was real and not fake they caused bias in the results or avoided obvious facts that pointed to it being a fake. I think confirmation bias is a great subject and one I will look at from a testing perspective sometime in the future.

• So can we use this ‘gut reaction’ concept in testing?
• Would it be of any value?

I should state that I have not tried any the following ideas and that if anyone would love to volunteer within their organizations to ‘trial’ the ideas out I would be most interested. Due to circumstances I currently do not have the ability to try this out on a large scale.

The first problem we face is how we capture out initial reaction to what we are testing. The requirements for this are that it is:

• Easy to capture
• Simple
• Quick

My thought is to use different smiley’s which are simple and quick to create and capture thus covering all the requirements.

My idea would be to use three different smiley’s:

• Happy
• Neutral
• Unhappy

Why use smiley’s?

The idea as to why use smiley’s is that anyone can draw them no matter how artistic and from the perspective of measurements it is very easy to recognize and see pasterns when using such well known symbols. The other longer term thought was that it is easy to extend to add sad, angry, and extremely happy if you wish to improve the range of emotions and feelings.

Capturing the initial feeling/emotion.

If you are working in an environment in which you are carrying out exploratory testing and following mission statements (Session based testing) then this is very simple to implement. The idea is that when the tester starts their mission (session) they should within the first couple of minutes (5 at a max) record their emotion/feeling of the software by the use of the smiley’s.

If this was done for every session being run and captured in such a way that it would be easy to see at a glance which areas (test charters) testers are unhappy with it could provide some useful information.

So you now have a whole set of data with regards to the testers initial feeling about the software there are testing, what does this information tell you?

For example a certain test focus area shows that all the testers are unhappy in that area would this indicate a problem? I feel it could indicate something wrong in that area but you would need to talk to the testers and gather more information (obtain context) I think the great thing about capturing initial feelings towards the software could help the development teams to focus on areas where there could be implied problems based upon initial feeling.

This approach could be taken a step further and get the testers to add another smiley when they have finished the session to see how they feel about the software after they have finished their session. You now have two sets of data and can compare any discrepancies with the two.

What would you think if the majority of testers were happy about a certain test focus area but at the end of the session they were unhappy?

Does this indicate a problem?

Or what if it was the opposite mostly unhappy and at end of session they were happy?

Also if they were unhappy at the beginning and at the end, their gut reaction proves to be correct, does this give an indicator that there are some major issues within that area?

Could this indicate frustration with the system, lack of knowledge maybe?

In my opinion this approach could provide to be a very useful oracle to the quality of the software.

What do think?

Could this prove to be useful?

I would love some feedback on this idea - good or bad.