Wednesday, 26 April 2017

A test is...

On twitter Santhosh Tuppad posted the following tweet:



It made me think about what 'a test' really is.

I made half an attempt to improve the statement Santhosh made and suggested the following:


This would make the statement read as following:
A Test is an experiment that tests to see if the testing yields observational information.
I have thought about this and it still does not appear quite right for what I would say is a test.

There have been many discussions on what testing is and how 'a test' relates to the testing.

A test is a part of testing as some of the links above state, therefore the definitions used for testing could apply to a test.  However there is a gap if you are communicating with someone and they ask you 'What is a test?'  What is your reply?  'Oh, it is a part of testing'  Hence thanks to Santhosh I wanted to look a little more into the definition of a test.

My thoughts is that for a definition of a test we should borrow from the scientific method.

You start with a theory, you run a series of experiments against that theory, you observe what happens, you analyse what you saw, you report the findings and adjust your theory based upon your new knowledge.

There is no mention of finding bugs within that process, looking at a dictionary definition of test it gives the following:


a critical examination, observation, or evaluation :  trial; specifically :  the procedure of submitting a statement to such conditions or operations as will lead to its proof or disproof or to its acceptance or rejection a test of a statistical hypothesis (2) :  a basis for evaluation :  criterion 
 an ordeal or oath required as proof of conformity with a set of beliefs 
a means of testing: such as (1) :  a procedure, reaction, or reagent used to identify or characterize a substance or constituent (2) :  something (such as a series of questions or exercises) for measuring the skill, knowledge, intelligence, capacities, or aptitudes of an individual or group 
 a positive result in such a test 
:a result or value determined by testing


Wow so many different meaning for a simple word!  

To aid the definition it needs context.

We are focusing on a software test, this enables us to narrow the definition of 'a test'. Note we are not using the term 'test case'.  Using 'test case' I feel encourages limitations in the scope of what a test means.

To conclude the following is my definition of 'a test' within the context of software testing.

A test is a series of experiments performed against a theory that evolves based upon observational and behavioral information uncovered by the test.
The software testing part is the reporting of these evolving theories and the observation made.

What about you?  Could you improve this?  Do you have your own definition?



Posted by at No comments:
Labels: , , , , , , ,

Monday, 24 April 2017

Fake News!



It has been a long time since my last post and I apologize, there have been many factors as to why this has been the case some to do with work load and others to do with my health.  Suffice to say I may not publish as often but I still want to get out into the public domain information that others may found useful.

The topic for this post is to do with the current wave of 'fake news' articles and how this may have influenced people and persuaded them to make decisions different from what they may have made if they had not seen such articles.

The concern is that there appears to be little critical thinking done around these stories by those who read them or watch them on the 'news' channels. These articles appeal to peoples biases either via 'confirmation bias'  (What is confirmation bias? - https://www.psychologytoday.com/blog/science-choice/201504/what-is-confirmation-bias) or appeal to authority.  (What is appeal to authority - https://yourlogicalfallacyis.com/appeal-to-authority)  Humans are easily duped or mislead and it requires conscious effort to overcome these fallacies and others.  There are many fallacies that we fall prey to and the following has a good list and description of them - http://www.webpages.uidaho.edu/eng207-td/Logic%20and%20Analysis/most_common_logical_fallacies.htm.

There are many guidelines and techniques that can be used to overcome such fallacies and clarify what is truthful or not.

One easy method is to look at the source of the information.

  • Is the source reliable?  
  • Does it come from multiple sources?  
  • Does the source have an undisclosed agenda? 
  • Can the information be verified independently?
Critically analyzing the information presented can help you make better judgement on  what is being said.

The following are a couple of techniques for critical thinking that I came across and are included in my book The Psychology of Software Testing

The 5 W's and H

Another technique, often used in journalism, is the five W's and one H.  The five W's are Who, What, When, Where and Why, The H is How.

The five Ws and one H have been immortalized in the poem 'I Keep Six Honest Serving Men' by Rudyard Kipling.
I KEEP six honest serving-men
They taught me all I knew; 
Their names are What and Why and When And How and Where and Who.
I send them over land and sea,
I send them east and west;
But after they have worked for me,I give them all a rest.    
I let them rest from nine till five,
For I am busy then,As well as breakfast, lunch, and tea,
For they are hungry men.
But different folk have different views; know a person small
She keeps ten million serving-men,
Who get no rest at all!  
She sends'em abroad on her own affairs,
From the second she opens her eyes
One million Hows, two million Wheres,And seven million Whys!    

The five W's and One H are a series of questions used to get the complete story, hence its use in journalism.  

Reporting on the 3 Little pigs story
  • Who was involved? 
    • The three little pigs (the first pig, the second pig and the third pig) and The Big Bad Wolf (a.k.a. Wolf).
  • What happened?
    •  Each pig constructed a house out of different materials (straw, sticks and bricks). Wolf (allegedly) threatened to blow over their houses and is believed to have destroyed both the straw and stick homes at this time. Pig one and two were able to flee to the brick house, where they remain at the moment. We’re still waiting to hear from local authorities, but it looks like the Wolf may have been injured while attempting to enter the brick house.
  • Where did it take place?
    • Outside a straw house, a stick house and a brick house.
  • When did it take place? 
    • At various times throughout the day.
  • Why did it happen? 
    • Apparently the Big Bad Wolf was trying to eat the pigs. Several eyewitnesses recall the Wolf taunting the pigs before he destroyed the straw and stick homes by chanting, “Little pigs, little pigs, let me in.” The pigs apparently scoffed at the Wolf’s idle treats, saying “Not by the hair of our chinny, chin chins.” It’s believed this angered the Wolf and led to him blowing the houses down.
  • How did it happen? 
    • It would appear the first two homes were not built to withstand the Wolf’s powerful breath. The incident inside the brick house is still being investigated, but early indications suggest the Wolf fell into a boiling pot of water when trying to enter the house through the chimney.


If you read any articles and it does not appear to follow this journalistic technique would be cause for concern as to its truthfulness.  You may want to delve deeper and see if the article is accurate and independent in its reporting.  

16 Steps to become a critical thinker

The following set of steps are based upon the article 'Intro to Logic: Techniques of Critical Thinking'.  
It is a useful critical thinking exercise to examine each of these steps and rewrite them to form your own set of steps to enable critical thinking.
  • Clarify
    • Ask questions to clarify what is being said.  Simplify to aid clarity.
  • Be accurate.
    • Facts can only be in the past.  Is anything in the statement making future predictions?  If so this is not fact.  Are the facts correct? Is there any factual evidence to back up the statement?
  • Be precise.
    • Make sure what is being said is accurate, try to avoid ambiguity.
  • Be relevant.
    • Make sure to stick to the issue under discussion, avoid falling for strawman or other fallacies.
  • Know your purpose.
    • Figure out what the most important thing is in the discussion.  Try to remove any related but not relevant information (see 'be relevant' above).
  • Identify assumptions.
    • When involved in critical thinking it is important to be aware that all thinking is based upon some level of assumption. Try to identify these assumptions.
  • Check your emotions.
    • Emotion can directly affect our critical thinking.  Try to keep emotions under control when discussing issues.  Ask yourself are my emotions influencing my judgement?
  • Empathize.
    • Look at what a person is saying from their viewpoint.  Try to put yourself in their shoes, how would you feel if someone spoke to you in the way in which you are speaking to them.
  • Know your own ignorance.
    • Know your level of knowledge. You do not know everything and what you do know may be wrong. Be gracious when someone proves you wrong, learn from being wrong.
  • Be independent.
    • Do not follow the crowd.  Verify information with dependent thought.  Do your own research to verify what is being claimed. 
    • Laurent Bossavit has a wonderful book on this subject called The Leprechauns of Software Engineering
  • Think through implications.
    • Look at what is being claimed and see what the implications of this claim could be.  Look for alternative implications, both negative and positive.
  • Know your own biases.
    • Being aware of your biases is crucial when involved in critical thinking.  How are they affecting your judgement? Are they affecting your judgement of others?
  • Suspend judgement.
    • Do not arrive at a conclusion and then try to find reasons that support your conclusion.  Use the scientific method as discussed earlier in this chapter. Form a theory on how it should work and then attempt to find ways to disprove your theory. 
  • Consider the opposition
    • Look for alternative and opposite perspectives.   Do not base your conclusion from one source.  Look for sources that disagree with the first source.
  • Recognize cultural assumptions.
    • Be conscious of stereotyping and cultural bias.  It does not mean if someone is from a different culture or period in time that their views are any less greater than your own.
  • Be fair, not selfish.
    • We are naturally selfish creatures and find it hard to be wrong and admit our mistakes.  Be fair with yourself and others, look for selfish traits in yourself and others.

Some links:

Speaking events.

I am due to speak at a couple of events this year.


Posted by at No comments:
Labels: , , , , , ,
Subscribe to: Posts (Atom)