It meets the business requirements

Recently I came across an interesting software requirements issue whilst eating out at a UK restaurant chain.

They had one of those common deals you can come across in similar chains.  You choose a main meal from a limited, but tasty, menu.  You get unlimited drinks of either tea/coffee or fizzy drinks. (soda) and you can have unlimited access to the salad bar .  To end your meal you get an ice cream sundae.  All of this for the sum of £9.99!

I was with my wife and we both ordered a main meal and a drink.  Whilst our meal was being prepared we went and selected our bowl of salad from the salad bar.  We set about eating our salad, during this time the drinks arrived followed by our main meal.  We only just managed to finish our main meal and was feeling rather full by now.  So when the waitress came over and asked if there was anything else we both said no can we skip the ice cream sundae and just get the bill.

The table was cleared away and the bill arrived......

Can you guess how much the bill came to?

If you are thinking the same as my wife and I you would think the bill would be £19.98.   The actual bill came to £20.13.  I queried this with the waitress and she replied....

"Oh, that is because you did not have the ice cream sundae!"

Our reply was "WHAT", We have less but it will cost us more?

The waitress was very confused and went to see her line manager. Who came over and explained that what they will need to do is order us the ice cream sundaes and then go and let the kitchens know that there is no need to make it.  They did say we could take it with us!  Ice cream, in car for hours & warm day  not a good combination.

All of the orders were taken on a mobile phone app which fed back to the main system and the kitchens.  There appears to be no option to be able to mark an order as one of the deals. Each item is entered individually.  Once you have the four items it now knows which are part of the deal and works out the price.  Now the question to those reading this..... "Is this a defect?"

From the business requirements perspective this is, I guess, what they wanted from the software.

Since entering each item one at a time and then working out if it is part of the deal allows them to manage stock control and keep business costs under control.  It has one less step for the operator to carry out rather than if they had to press a button to indicate it was a deal,

From the end user experience perspective it is flawed.  It appears they did not expect customers to decline something that was in reality free. From the staff working there they had to override the system and implement manual processes to enable the correct price to be calculated. At the same time this will upset the stock control reporting that they have two less ice cream sundaes than they actually do.

When we look at software requirements and we start testing these requirements, it is important we look at the implementation from all perspectives.  This is especially true for systems that businesses use to help facilitate customer service.  Get this wrong and it can leave a bad impression on the business customers.

Believing in the Requirements

Traditionally in testing there has been a large amount of emphasis placed upon ‘testing’ ‘checking’ the requirements.  An article by Paul Holland on Functional specification blinders  and my currently reading of Thomas Gilovich excellent book on How we know what isn’t so has made me re-think this strategy from a psychological perspective. I feel Paul was on the right track with his suggestions of not using the requirements/specification to guide your creative test idea generation but looking at alternatives.  However even these alternatives could cause limitations in your thinking and creative ideas due to the way we think.

The problem we have is that once we have been presented with any information our inbuilt beliefs start to play their part and look at any information with a bias slant.  We at built to look for confirmations that match our beliefs in other words we look for things we want to believe in.  So if believe the implementation is poor or the system under test has been badly designed we will look for things that confirm this and provide evidence that what we believe is true.  We get a ‘buzz’ when we get a ‘yes’ that matches our beliefs.  The same could apply when looking through the requirements we start to find things that matches our beliefs and at the same time the requirements (especially if ambiguous) start to influence our beliefs so that we, as Paul discovered, only look for confirmations of what is being said.  Once we have enough information to satisfy our beliefs we then stop and feel that we have done enough.

The other side of this is that any information that goes against our beliefs makes us dig deeper and look for ways to discount the evidence that is against what we believe.  When faced with evidence that is against what we believe we want to find ways to discount this information and find flaws in it.  The issue is that if we are looking at requirements or specification then normally there is not much that goes against our initial beliefs due to the historic influence that these documents can have.  So we normally do not get to the stage of digging deeper into the meaning of these documents.

As Thomas Gilovich stated

People’s preferences influence not only the kind of information they consider, but also the amount they examine.

If we find enough evidence to support our views then normally we are satisfied and stop.  This limits our scope for testing and being creative. My thoughts on how to get around this apart from following the advice Paul gives is one of being self-critical and questioning oneself.

When we are in a confirming our beliefs mode we are internally asking ourselves the following question

 “Can I believe this?”

Alternatively when we find information that does not match or confirm our beliefs we internally ask ourselves the following question

“Must I believe this?”

These questions are taken from the book by Thomas Gilovich referenced earlier and in this Gilovich states

The evidence required for affirmative answers to these two questions are enormously different.

Gilovich mentions that this is a type of internally framing we do at a psychological level, after reading this it reminded me to go back and read the article by Michael Bolton on Test Framing in which I attended a tutorial at the Eurostar Test Conference . I noted within the article by Michael that there appeared, IMO, a lot of proving the persons beliefs rather than disproving.  In other words many of the examples were answering the “Can I believe this” question.  This is not wrong and is a vital part of testing and I use the methods described by Michael a great deal in my day to day work.  I wonder if this topic could be expanded a little by looking at the opposite and trying to disprove your beliefs, in other words asking the “Must I believe this?” questions.

So moving forward I believe that we can utilize our biases here to our advantage to become more creative in our test ideas.  To do this we need to look at ways to go against what we belief is right and think more negatively.  The next time you look at a requirements or specification document ask yourself the following:

“MUST I BELIEVE THIS”

And see where this leads you.

---

PS – this article is a double edged sword – if you read this article you should now be asking “Must I believe this?”