Information Overload and Bad Decisions

This blog article is based upon my conference talk  at the Lets Test conference in Sweden.

Before we start to look at why too much information  is bad it might be worth defining what Information Overload actually means.

According to  D. Allen and T. D. Wilson, “Information overload: Context and causes,” - The New Review of Information Behaviour Research Volume 4, Issue 1, 2003

Information overload occurs when the information available exceeds  the user's ability to process it 

Research has shown that as human beings we have a limited amount of capacity for storing information within our brains.

The human brain is quite remarkable.  It can store perhaps three terabytes of information.
http://www.sizes.com/people/brain.htm

And yet this is only about one millionth of the information that IBM say is now being produced in the world each day (and growing)

The signal and the Noise – Nate Silver

The problem is that too much information can have a serious affect on human beings.  The term 'Information Overload' appears to be credited to Alvin Toffler in his book 'Future Shock' In which he says the following about the problems of overloading.

It has been seen that Overloading leads to a serious breakdown of performance (sometimes with dangerous results). We are, in other words forcing them to process information at a far more rapid pace than was necessary in slowly-evolving societies. There can be little doubt that we are subjecting at least some of them to cognitive over stimulation.  What consequences this may have for mental health in the techno-societies has yet to be determined.

You may think that this is a 'new' problem, Alvin wrote the above passage of text in 1972!

Look at the image above what do you see?  Butterfly?  Tree?  Stars? It is a randomly created piece of art.

We are good at seeing patterns and really good at making patterns from things where no patterns exists.  We do the same with numbers and patterns of numbers, lucky streak.  This within testing can cause us problems and unless we use critically thinking our own ancient survival skills will let us down.

 .. your brain hates ambiguity and is willing to take shortcuts to remove it from any situation.  If there is nothing else to go on, you will use what is available.  When pattern recognition fails, you create patterns of your own 
You Are Not So Smart - David McRaney

Statistics Joke:

Did you hear about the statistician who drowned whilst crossing a river which had an average depth of 3 foot. 

We get mislead and enticed within testing by the use of numbers, especially the use of pass/fail metrics and many others without thinking that numbers need a story.  Once you have as story you can use numbers to help backup the story.  Telling the story of numbers is vital. Using only numbers to measure will ensure that you end up measuring the wrong thing!

I know Alan Page loves the Gorilla so I included it in my talk.

If you already seen the Gorilla video or know about it I still recommend you watch the following video.

http://www.youtube.com/watch?v=IGQmdoK_ZfY

Did it still mange to fool you?  Even when people have seen it they still miss the unexpected.  We find it difficult to do two things at once.  When we try to do this we lose some valuable learning - multi-thinking is a myth.  More information on this can be found in this interesting article. http://blogs.kqed.org/mindshift/2013/05/how-does-multitasking-change-the-way-kids-learn/

The next challenge I set was to ask the following question - now try to do t his quickly before moving on to the next paragraph:

How many animals did Moses take into the Ark?

Did you try and find an answer?  Or did you question the question (think critically).  If you still see nothing wrong with the question read it again.

There is a danger as testers that we can easily be primed and anchoring into thinking that we need to give an answer or solve a problem.  This leads into a discussion about what is the role of testers.  To find or solve problems? I have slightly touch on this subject previously.

The other way in which we get anchoring into thinking in one way is by an over reliance on requirement documentation. Instead of playing and using the software we try to second guess and base our testing on a set of requirements that then controls and leads our testing effort.  Rikard Edgren talks more about this in his article on the search of the potato.  Do not only rely on requirement documents we need to ask what more is there?  We need to think about what the requirement may not be saying

So how does all of the above relate to testing?

We base our testing decisions on our biases and “quick” judgement , what we think we believe, we do not take the time to think We create patterns from our tests where no patterns may exist – we are built to spot patterns and sometimes this is useful other times it can mislead and waste time and lead to wrong choices being made. We by design follow the path of least resistance with our thinking – we are lazy. We make irrational quick choices that we are unaware we are making.  Sometimes we do things without thinking, this is useful for driving etc but not when testing we need to reflect, refocus and re-frame.

When we are faced with too much information we may not be able to make decisions or even make the wrong decision. For example soldiers freezing in a conflict situation or panic on plane which has crashed and on fire.  We can become overwhelmed and cannot make a choice so we make no decision and await our fate

Information: The very thing that makes it possible to be an engineer is threatening our ability to do our work. – IEEE Spectrum

We constantly fail to apply critical thinking to our testing, we are so overwhelmed that we forget to ask questions such as:

• Why are you doing what you are doing?  
• Could you be doing something better?  
• Is this the most important thing you could be doing?

This leads on to the James Bach approach to critical thinking in testing

• Huh? – Do I really understand?
• Really? How do I know what you say is true?
• So? Is that the only solution?

You do not have to use this example be creative and create your own that is thinking!

“Rule of Three” – If you haven't thought of at least three plausible explanations, you’re not thinking critically enough - Gerald Weinberg - Quality Software Management Volume 2

So what can we do to help this overloading of information?

• Slow down and think, we are going far to fast. We need time to pause and  to reflect I have written about slowing down previously.
• We need to remember that creative thinking is just as important and we sometimes need to take a step back
• We need to learn that it is ok to make mistakes and get your assumptions wrong that is the best way to learn.  That is the important part, we must remember to learn from them.
• If you are aware you are human and can easily be fooled then that can help you improve your thinking and question your own beliefs and motives for what you are doing.
• Stop doing too much planning far far ahead. We learn a lot more by doing, tinkering and playing, Discover by accident, be creative .  To remove your fallacies and assumptions you need to play with the system and see what it does.
• Have a passion for testing  and your job (not just testing) passion drives your knowledge and thirst to learn more.

Nothing great has been and nothing great can be accomplished without passion
GWF Hegel