Information Overload and Bad Decisions

This blog article is based upon my conference talk  at the Lets Test conference in Sweden.

Before we start to look at why too much information  is bad it might be worth defining what Information Overload actually means.

According to  D. Allen and T. D. Wilson, “Information overload: Context and causes,” - The New Review of Information Behaviour Research Volume 4, Issue 1, 2003

Information overload occurs when the information available exceeds  the user's ability to process it 

Research has shown that as human beings we have a limited amount of capacity for storing information within our brains.

The human brain is quite remarkable.  It can store perhaps three terabytes of information.
http://www.sizes.com/people/brain.htm

And yet this is only about one millionth of the information that IBM say is now being produced in the world each day (and growing)

The signal and the Noise – Nate Silver

The problem is that too much information can have a serious affect on human beings.  The term 'Information Overload' appears to be credited to Alvin Toffler in his book 'Future Shock' In which he says the following about the problems of overloading.

It has been seen that Overloading leads to a serious breakdown of performance (sometimes with dangerous results). We are, in other words forcing them to process information at a far more rapid pace than was necessary in slowly-evolving societies. There can be little doubt that we are subjecting at least some of them to cognitive over stimulation.  What consequences this may have for mental health in the techno-societies has yet to be determined.

You may think that this is a 'new' problem, Alvin wrote the above passage of text in 1972!

Look at the image above what do you see?  Butterfly?  Tree?  Stars? It is a randomly created piece of art.

We are good at seeing patterns and really good at making patterns from things where no patterns exists.  We do the same with numbers and patterns of numbers, lucky streak.  This within testing can cause us problems and unless we use critically thinking our own ancient survival skills will let us down.

 .. your brain hates ambiguity and is willing to take shortcuts to remove it from any situation.  If there is nothing else to go on, you will use what is available.  When pattern recognition fails, you create patterns of your own 
You Are Not So Smart - David McRaney

Statistics Joke:

Did you hear about the statistician who drowned whilst crossing a river which had an average depth of 3 foot. 

We get mislead and enticed within testing by the use of numbers, especially the use of pass/fail metrics and many others without thinking that numbers need a story.  Once you have as story you can use numbers to help backup the story.  Telling the story of numbers is vital. Using only numbers to measure will ensure that you end up measuring the wrong thing!

I know Alan Page loves the Gorilla so I included it in my talk.

If you already seen the Gorilla video or know about it I still recommend you watch the following video.

http://www.youtube.com/watch?v=IGQmdoK_ZfY

Did it still mange to fool you?  Even when people have seen it they still miss the unexpected.  We find it difficult to do two things at once.  When we try to do this we lose some valuable learning - multi-thinking is a myth.  More information on this can be found in this interesting article. http://blogs.kqed.org/mindshift/2013/05/how-does-multitasking-change-the-way-kids-learn/

The next challenge I set was to ask the following question - now try to do t his quickly before moving on to the next paragraph:

How many animals did Moses take into the Ark?

Did you try and find an answer?  Or did you question the question (think critically).  If you still see nothing wrong with the question read it again.

There is a danger as testers that we can easily be primed and anchoring into thinking that we need to give an answer or solve a problem.  This leads into a discussion about what is the role of testers.  To find or solve problems? I have slightly touch on this subject previously.

The other way in which we get anchoring into thinking in one way is by an over reliance on requirement documentation. Instead of playing and using the software we try to second guess and base our testing on a set of requirements that then controls and leads our testing effort.  Rikard Edgren talks more about this in his article on the search of the potato.  Do not only rely on requirement documents we need to ask what more is there?  We need to think about what the requirement may not be saying

So how does all of the above relate to testing?

We base our testing decisions on our biases and “quick” judgement , what we think we believe, we do not take the time to think We create patterns from our tests where no patterns may exist – we are built to spot patterns and sometimes this is useful other times it can mislead and waste time and lead to wrong choices being made. We by design follow the path of least resistance with our thinking – we are lazy. We make irrational quick choices that we are unaware we are making.  Sometimes we do things without thinking, this is useful for driving etc but not when testing we need to reflect, refocus and re-frame.

When we are faced with too much information we may not be able to make decisions or even make the wrong decision. For example soldiers freezing in a conflict situation or panic on plane which has crashed and on fire.  We can become overwhelmed and cannot make a choice so we make no decision and await our fate

Information: The very thing that makes it possible to be an engineer is threatening our ability to do our work. – IEEE Spectrum

We constantly fail to apply critical thinking to our testing, we are so overwhelmed that we forget to ask questions such as:

• Why are you doing what you are doing?  
• Could you be doing something better?  
• Is this the most important thing you could be doing?

This leads on to the James Bach approach to critical thinking in testing

• Huh? – Do I really understand?
• Really? How do I know what you say is true?
• So? Is that the only solution?

You do not have to use this example be creative and create your own that is thinking!

“Rule of Three” – If you haven't thought of at least three plausible explanations, you’re not thinking critically enough - Gerald Weinberg - Quality Software Management Volume 2

So what can we do to help this overloading of information?

• Slow down and think, we are going far to fast. We need time to pause and  to reflect I have written about slowing down previously.
• We need to remember that creative thinking is just as important and we sometimes need to take a step back
• We need to learn that it is ok to make mistakes and get your assumptions wrong that is the best way to learn.  That is the important part, we must remember to learn from them.
• If you are aware you are human and can easily be fooled then that can help you improve your thinking and question your own beliefs and motives for what you are doing.
• Stop doing too much planning far far ahead. We learn a lot more by doing, tinkering and playing, Discover by accident, be creative .  To remove your fallacies and assumptions you need to play with the system and see what it does.
• Have a passion for testing  and your job (not just testing) passion drives your knowledge and thirst to learn more.

Nothing great has been and nothing great can be accomplished without passion
GWF Hegel

Lets Test Conference Talk 2013 - Resources

I recently had the good fortune to be able to be a speaker at the Lets Test conference in Sweden. - .  My topic for the conference talk was called Information Overload and Bad Decisions  - More details here

Anyone interested in viewing my presentation can use the following link for the SVG version:
https://docs.google.com/file/d/0B8jc_cHKwbNoSE80SnRLdDVzbXM/edit?usp=sharing

I also have a traditional PowerPoint version for those more used to this kind of presentation
https://docs.google.com/file/d/0B8jc_cHKwbNoNzNJVXl0SnBYcVU/edit?usp=sharing

My notes:
https://docs.google.com/file/d/0B8jc_cHKwbNoUktpSjBmcU1JZGc/edit?usp=sharing

For those interested in learning how I created the SVG presentation you can find out more by clicking on the links below.

I used two tools - Inkscape (http://inkscape.org/) and Sozi - http://sozi.baierouge.fr/wiki/en:welcome

My next blog article will be a short summary of the talk.

Time to slow down.

This short blog post has been inspired by some of my reading over the holiday period including the following

• Future Minds: How the Digital Age is Changing Our Minds, Why This Matters and What We Can Do About It by Richard Watson 
• Future Shock - Alvin Toffler

We appear to be living in a constantly connected world where we are being bombarded with Terabytes of information each and every day and we could be approaching information overload and the dangers this bring.  Since we are receiving so much information our brains are taking the easy route and mainly accepting what it is been fed without questioning and you start believing things you would not normally accept.

 In A Mind of Its Own, Cordelia Fine makes the point that the brain’s default setting is to believe, largely because the brain is lazy and this is the easier, or more economical, position. However, when the brain is especially busy, it takes this to extremes and starts to believe things that it would ordinarily question or distrust.

Richard Watson in Future Minds states the following:

One of the consequences of rapid information transmission is that we increasingly fail to think properly about the validity of incoming or outgoing information; we are too busy and there is too much of it.

There is so much pressure on us to be doing stuff and looking up stuff and being always available no matter where we or what we are doing, there are very few places where you can escape to think.  The traditional places such as the pub (bar)  or the local park have been taken over by constant ringing and people with their faces bent over a tiny screen.

Alvin Toffler talks about Information Overload in his book Future Shock and talks about how we freeze when we get overloaded with information.  He talks about being overstimulated in war situations and how people will just shut down.  This appears to becoming more common with people turning into mobile phone zombies craving for their next information fix and ignoring all possibilities of serendipity moments from looking at the world around them.

We are using our memory less and less since we can now “Google” it, so we have less storage in our own heads to be radically creative and generate ideas.  Life is being run in the fast lane and we are in danger of doing less and less serious thinking.  We are being told we need a decision NOW, so we skim, scan or ignore and then make a (what could be a wrong) choice.

I feel very much spilt on this subject since I am a techno geek, I have a passion for gadgets and anything technological but at the same time I am starting to realise that I have less time to myself to do nothing and gather my thoughts and do some SLOW thinking.

Richard Watson suggests going for a walk or just starting out of the window.  How many people do you see today in the office day dreaming or just starting into space and really thinking?  What would happen if you did this in your office?  He suggests that we have a day in which we plan and do nothing and allow ourselves to be immersed in our own thoughts.

My own view is that we need to step back a little sometimes and slow down to allow our minds to think and to think deeply.  I have a concern that in the future there will not be any deep thinking and people will just be looking at what they see on the surface and believe that to be true.  We need to start carrying or using notebooks to capture our ideas especially when in a deep thinking moment.  We may have many ideas during the day, some great, some good and some bad but we need to start capturing this and help to provide situations which are conducive to the generation of ideas and deep thinking.

Richard gives the following helpful hint:

If it helps, create three physical notepads, files, or boxes marked “no,” “yes,” and “maybe” and place a note of your thinking in the appropriate one. 

This is a method I use for my ideas for a blog article I have currently about 400 ideas marked 1-3 with one being likely and 3 not likely.  I review these about once a month and change the rating of the chances that it will appear as an article.  This is important since my views and thoughts over time will change and something I felt was relevant at a certain time is no longer relevant.

We can also apply this to exploratory testing There is a still a strong contingent of people in the testing world who measure by number of tasks (test cases*) being completed is a useful way to measure progress and know when we are done testing.  However I feel we need to take a step back and slow down a little to allow ourselves sometime to think.

Exploratory testing is a very human thinking activity and it is easy to start measuring progress by number of missions/sessions completed.  Instead we should allow time to thinking deeply about what we are testing and what information are we uncovering since this could lead to moments of serendipity and that eureka moment.  So the next time you see a tester staring into nothing it may be that they are deeply thinking about what they are doing.  TE Lawerence is quoted as saying the following about people who daydream:

“Those who dream by night in the dusty recesses of their minds wake in the day to find that it was vanity: but the dreamers of the day are dangerous men, for they may act on their dreams with open eyes, to make it possible.”

(*This blog is not going to get into a metrics debate here others have done a much better job than myself see Michael Bolton links here)

So to conclude

• We need to get off the information highway sometimes and reduce the information we are receiving.  
• We need to slow down and allocate some time to thinking
• We need to do something different so we get more experiences that can help generate more serendipity ideas. 
• Do some gardening, take a bath, go for a walk in the countryside with no destination, lose yourself in your own mind.
• Take vacations(holidays) and remain unconnected from the office/work
• Do something you have a passion for and enjoy
• Do not be afraid of making mistakes with the ideas your generate this is valuable experiences

“I may not have gone where I intended to go, but I think I have ended up where I intended to be.” Douglas Adams