Emergent Strategy

How often have you been asked why do we do exploratory testing rather than planned and predicted scripts.  Recently I have been reading some material on corporate planning strategies and how some become successful and other do not and looking at how this links into software development especially from a testing perspective. 

Given that software can be very dynamic and react in some unpredictable ways no matter how much planning we do.  It surprises us and more importantly it surprises the person who created it.  This goes against the commonly held notation that software is predictable since we planned with great care and detail what was going to be coded.  The problem comes is that we are human and may not act in a rational way and this is reflected in our creations.  

So what is the best way to do software testing?  

The purpose of testing is to learn things about it and the best way of doing this is to experience it and learn by do it.  This is best summed up by Nassim Nicholas Taleb

 We are better at doing than learning. Our capacity for knowledge is vastly inferior to our capacity for doing things – our ability to tinker, play, discover by accident.

http://www.fooledbyrandomness.com/notebook.htm

Within the corporate strategy world I have discovered that this appears  to have a  name  'emergent strategy'  (or realized strategy).  Looking more into this I found the following link http://planningskills.com/glossary/154.php

One of the most interesting part of the above link to me that I noticed was the following sentence:

Emergent strategy implies that an organization is learning what works in practice

Is this not similar to what we attempt to do when doing exploratory testing?  We try to learn about the product and what is working or not working by experiencing  it?

An interesting point made in the article is the following statement

Mixing the deliberate and the emergent strategies in some way will help the organization to control its course while encouraging the learning process.

This appears to link back to an  article I wrote about  hybrid testing and having a mixture of scripted checks and  exploratory tests.

So to go back to my first statement about the purpose of exploratory testing.  To me since we cannot predict everything that the software will do the only way to understand what it will do and to learn is to explore it and that is the purpose of exploratory testing.  To uncover information that may prove to be of value to someone who matters.

 I may need to look more into emergent strategy a little more.

Place your bets

Having recently completed reading the excellent book “The Click Moment” by  Franz Johansson (@Frans_Johansson).  I was amazed by how much of the material written in the book appears to relate to software testing.  The principles of the book are about creating opportunities in an unpredictable world and not about putting in hours and hours of practice.  The author explains that if there are fixed rules and these rules do not change too much then the 10000 hours rule of practice works  .  However the author points out we are living in a world in which the rules are always changing and unexpected (random) things can and do happen.

Some of the statements made in the book appear to have a correlation to the current state of software testing and the various “schools”.  (The scripted vs the exploratory debate).  The first thing that caused me to think was some of the comments on planning  and how this stifles opportunities for random events and for uncovering new and exciting things.  

For example Franz stated the following

…In fact, it might mean that the plan is outdated before you even start to execute it….”

I have often experienced this within companies that believe that we can plan upfront and know all we need to know to write scripts before we actually use the product.  I have seen test plans which when I start doing some testing are hopelessly out of date and then spend unnecessary time trying to retrofit what I am experiencing when testing with what the plan is saying.  Doing this makes me take my eye of the ball and miss chances to find out what could be important information.

Franz then makes a statement which could be taken directly from why we need to do exploratory testing.

“..As ironic as it may sound, it actually pays to schedule time to do something unscripted and unplanned. We need to leave enough room in our day to explore things that are not connected to our immediate goals. We need to free ourselves up to become aware of hidden opportunities and expose ourselves to significant click moments. Leave some flexibility in your schedule. Then, make sure you use the flexibility to explore something unrelated to what you are doing or to follow up on a curious idea you have been considering”

This offers so much potential for uncovering new and valuable information without the restrictions of following someone else’s thinking.  This way of testing in my world can lead to many serendipity moments.

So how can we help to make this happen in software testing?  Is there anything we can do to help create more of these moments of randomness?

Franz within the book gives 5 great tips which may encourage more serendipity.  I have listed the tips below and give a description of how this could apply to exploratory testing.

1. Place Many Bets 

Having a single exploratory testing mission which can consist of an infinite number of tests (bets) is surely much better than having a single scripted test in which you are only placing one bet.

2. Minimize the Size of the Bets

Instead of spending lots of time creating a test script based upon assumptions do the minimum required to do some actual testing and time box your testing sessions.

3. Take the Smallest Executable Step

Do the minimal amount of planning you can do to enable you to do some exploratory testing.  We need to stop thinking if we write detailed test scripts and plans before we really know anything that this will lead to us uncovering lots of information about the system.

4. Calculate Affordable Loss, Not ROI

We still believe that there is a justifiable, measurable cost in planning ahead and creating detailed test plans and scripts.  Which we then discover are outdated and very costly to maintain but we insist they are useful because someone else may use them in the future.  Instead look at creating lots of test ideals using test models and heuristics  which are cheap to create and if of no use can easily be discarded once we uncover more information when testing.  We should be looking at testing and its cost effectiveness from what can we afford to throw away if our assumptions are wrong.

5. Use Passion as Fuel

This is so important people with a passion for what they are doing are the drivers of opportunities.  This type of person is one where if they get stuck or falter they pick themselves back up, dust themselves off and look for ways around the problems.   These are innovators, the people who can radically change the market and improve what is already there.  There is a need to employ more of these passionate types of people in the world of software testing.  I am getting fed up of the 9-5 testers, the ones who have no desire to learn or improve themselves, the ones not reading this blog.

I do recommend that anyone involved in software development read this book it gives some great advice of how we can adapt what we do and how we think to improve our chances of delivering successful projects. 

PS No I am not being paid by Franz for writing this 

Testers should learn code

At the Eurostar conference in Amsterdam  Simon Stewart (@shs96c) presented a keynote on Selenium over the years

During the key note Simon made the following comment

"If you are testing the web you absolutely need to be able to code"

Now I am sure that out of context this could be taken in many ways and Rob Lambert has produced a very good discussion on this same subject here which takes on both sides.

I will add is that Simon did follow this up with the following line.

"If not become a specialist so you can add value"

This led to some interesting exchanges on Twitter (search for #esconfs) in which people came down on either side. I had concerns that people would only hear the first bit and this could cause barriers to some great people being able to be involved in testing just because they have no interest in coding or even wanting to learn to code.

IMO I am not sure about this statement and it caused much debate after the key note.  If you have an interest in learning code then do so otherwise do something that can add value. The discussions continued during lunch and the rest of this article is my own thoughts on this subject.

After talking to Simon afterwards it appears his message had got taken the wrong way.  He said it is helpful to code and that if all you do is test (check) scripts then you may not have a job.

My concern is forcing people to code if they have no interest could be a block from great people wanting to enter the world of software testing.

Dot Graham stated the following "Lose a good tester but gain a poor programmer".

I am not convinced that everyone needs to code, it can have its advantages but there is another perspective. If you do not understand the code you may test in a different non-confirmatory way.  You may be able to ask the difficult questions of why did you do it this way and made it complex?  You may not have a bias built up from your coding experiences and knowledge.  I think for some it can be useful but insisting on it is a very dangerous path to follow.

Some of the discussions that followed went along the lines that if testers refused to learn we should not employ them.  This is where I had a WTF moment.....  I have not said anything about testers not wishing to learn what I was saying was some people may not have an interest or a knack for coding or find it impossible for whatever reason to grasp.  However they make one hell of a tester and show a great thirst for learning new ways to exercise the software that are novel, unique and valued.  This was the second point that Simon was making and sadly appeared to have been missed.  As long as you can find ways to add value then you can be a tester.

I am afraid that a statement of this sort can be used as a filter to prevent people entering this great world of software testing.  There are many other things that IMO testers could learn about such as grounded theory, anthropology, social sciences, humanities, creative arts and the list goes on.  There are some great testers who have learnt these things and should we prevent them for working as software testers because they have no desire to learn code?

I will finish this on a positive note and that it was great to chat with Simon even if our views are slightly different and that he is a very thoughtful and  passionate person.  I look forward to meeting up again sometime  in the future and finding another topic to discuss.

PS Thanks to Rob Lambert for being the referee!!

(edited some of the grammar :o( )

Some good ideas to aid testing in 7 sentences

I was asked to do a small talk in the community hub at Eurostar 2012 test conference on the following topic:

The best test technique in seven sentences

which I changed to the following

Some good ideas to aid testing in 7 sentences

Since it might be best just for now.....

Some people have asked I post the presentation somewhere so here it is.

DISCOVER

...all the information you can, both verbal, non-verbal and written.

EXPLORE

....the system to learn about what it actually does rather than what you think it does.

TALKING

....and communication is important, you need to talk to everyone.

EXAMINE

..all information provided or uncovered by exploring this is your evidence.

CONTEXT

...is crucial, all your testing should be driven by the context at that time.

THINK

...and engage your brain, testing is all about thinking.

D.E.T.E.C.T

Act like a detective and DETECT your bugs