A test is...

On twitter Santhosh Tuppad posted the following tweet:

It made me think about what 'a test' really is.

I made half an attempt to improve the statement Santhosh made and suggested the following:

This would make the statement read as following:

A Test is an experiment that tests to see if the testing yields observational information.

I have thought about this and it still does not appear quite right for what I would say is a test.

There have been many discussions on what testing is and how 'a test' relates to the testing.

• Testing vs Checking - Bolton 2009
• Testing vs Checking refined - Bach/Bolton 2013
• What is Software Testing - ISQTB
• Exploratory Testing - Kaner 2006

A test is a part of testing as some of the links above state, therefore the definitions used for testing could apply to a test.  However there is a gap if you are communicating with someone and they ask you 'What is a test?'  What is your reply?  'Oh, it is a part of testing'  Hence thanks to Santhosh I wanted to look a little more into the definition of a test.

My thoughts is that for a definition of a test we should borrow from the scientific method.

You start with a theory, you run a series of experiments against that theory, you observe what happens, you analyse what you saw, you report the findings and adjust your theory based upon your new knowledge.

There is no mention of finding bugs within that process, looking at a dictionary definition of test it gives the following:

a critical examination, observation, or evaluation :  trial; specifically :  the procedure of submitting a statement to such conditions or operations as will lead to its proof or disproof or to its acceptance or rejection a test of a statistical hypothesis (2) :  a basis for evaluation :  criterion 

 an ordeal or oath required as proof of conformity with a set of beliefs 

a means of testing: such as (1) :  a procedure, reaction, or reagent used to identify or characterize a substance or constituent (2) :  something (such as a series of questions or exercises) for measuring the skill, knowledge, intelligence, capacities, or aptitudes of an individual or group 

 a positive result in such a test 

:a result or value determined by testing

https://www.merriam-webster.com/dictionary/test

Wow so many different meaning for a simple word!  

To aid the definition it needs context.

We are focusing on a software test, this enables us to narrow the definition of 'a test'. Note we are not using the term 'test case'.  Using 'test case' I feel encourages limitations in the scope of what a test means.

To conclude the following is my definition of 'a test' within the context of software testing.

A test is a series of experiments performed against a theory that evolves based upon observational and behavioral information uncovered by the test.

The software testing part is the reporting of these evolving theories and the observation made.

What about you?  Could you improve this?  Do you have your own definition?

The Dark side of Continuous Deployment

A short post!

In the software development world there is a drive towards continuous deployment of software which in the majority of cases in a good thing.  The good side is the reduction in costs, faster feedback from users and quicker release of features and bug fixes.

However my experiences of using software in which there are constant and frequent updates provides me with some concerns.  Have you ever downloaded an app for your device that crashes upon starting or has new updates each and every day?  How does this make you feel?  I for one get very frustrated when I get up in the morning and my phone says x number of apps have been updated.  Or that the same app I have updated 3 times previously in the last week!  Suffice to say this app have been deleted and I no longer use it.

There is a common phrase that appears to have been forgotten in the rush to continuous deployment and using users as testers and that is "First impressions count"  If you come across a piece of software which you have purchased and it is buggy or does not work in the way you expect you may feel cheated.  All software has bugs that is to be expected, my worry is that we are paying for software which has not been sufficiently tested by skilled testers.

There is a major movement to automate 'testing' within continuous deployment practices.  This implies to me that those who promote and suggest this may not have an understanding of what testing is and the value it provides to the quality of software.  This is a big assumption for me to make but I am experiencing more and more examples of software being released to users which is not good enough and with a little bit of testing could have provided a much better user experience.

I am not saying to stop doing continuous deployment but within the development cycle there should be some level of human testing to provide information about how the software is behaving.  This is not meant to say have a dedicated testing phase but ensure that testing activities are added into each and every release. Continuous deployment is a wonderful tool to enable quick and easy releases however it should not be used to release poor or sub-standard software that relies on users to provide details of issues, especially software that a user has purchased.

The key problem to me as someone who works in business is that if you release poorly tested software and it is noticed then the chances are they will not come back and you will lose their customer.  If this happens again and again you may find you do not have a business, social media is powerful in allowing those who experience bad software to make it know very quickly to a huge amount of people.  Can your business take that risk?

RANT over.....

Over to you the reader, what are your thoughts on this?

Further reading:

Testing vs Checking refined - James Bach
Continuous Deployment - Agile Alliance
Why continuous deployment - Eric Ries

If Testers were Paranormal Investigators

Image: Witthaya Phonsawat / FreeDigitalPhotos.net
http://www.freedigitalphotos.net/images/view_photog.php?photogid=3116

I thought considering it is getting close to the spooky time of year (All Hallows Eve) I would put together a tongue in cheek article about what would happen if exploratory testers were paranormal investigators.

It was a dark moonlit night as the certified and exploratory tester approached the dark imposing building that their project manager had asked them to look at. The project manager wanted them to report back on if the building was suitable for him to move into and that there were no hidden surprises. They had heard stories that the building was full of bugs and other scary stuff.
The testers used the (pass) keys to enter the building and slowly walked into the main hallway of the building. As they started to walk around the room suddenly got cold as the temperature dropped.

The certified tester says it must death being down the temperature.

The exploratory tester looks around the room and notices that there appeared to be a draft coming from just outside the room. They go to explore the draft since it has now interested them as something that could answer a question. Once outside the room they notice that one of the windows has come open due to a broken catch. They close the window and make a note to contact a handy man in the morning. The temperature of the room returns back to normal.
The testers slowly move towards the kitchen when all of sudden an overwhelming disgusting smell overpowers their senses.

The certified tester is certain that this is the smell of death coming to get them.

The exploratory tester is not sure and starts to use their sense of smell to see if they locate where the smell is coming from. They notice that the smell appears to get stronger in the direction of the fridge. They open the fridge door and note that the fridge does not appear to be working (even noticing that it is plugged in) inside the fridge there is a bottle of gone off milk which appears to be the source of the smell. They make another note to contact a fridge repair person in the morning.

The testers now move to the next floor in the building when suddenly they appear to see something move on the stairwell.

The certified tester is sure that this is a sign of sprits from beyond the grave.

The exploratory tester takes a moment to think about possible reasons for the movement before realising that the window has come open again causing the light fitting to swing and cast different light patterns on the stairwell giving the impression of movement.

They now move towards the main bedroom on the first floor and suddenly they hear unnatural sounds and what appears to be a creature from another world.
The certified tester is certain that this is souls from the other side warning them to leave and now starts to panic.

The certified tester is not so sure and even though they are starting to get a little scared they open the door to the main bedroom and the noise gets louder and louder. With their heart pounding they enter the room and see a large irregular shaped mass on the bed from where the noises are coming from. Slowly they move forward get closer, closer still and even closer……..


Suddenly the mass moves and the exploratory tester notice that it is their project manager fast asleep and snoring, the snoring which is causing the supernatural noises.

THE END.

Loosely based upon the following article: http://www.wired.com/magazine/2011/09/pl_screenghosthunters

Disclaimers:
• All characters appearing in this work are fictitious. Any resemblance to real persons, living or dead is purely coincidental.
• I wish to stress that this blog in no way endorses a belief or non-belief in the occult or anything of a supernatural nature.