An Open Letter to Professional Tester Magazine

Dear editors of professional tester magazine

It is not often that I get a strong enough reason to become political and reply to some editorial pieces within a published article.   It is against my better nature to enter into debates which one side attacks another side but unfortunately your recent articles have been on mind for a few days and I feel the need to reply to the following articles on your website.

• Book burners threaten (old) new testing standard
• PT’s independence is questioned

I feel the only way to express my disappointed is by use of social media, I 'had' a great deal of respect for your magazine and your in-depth reports and wide variety of articles, sadly that has diminished.

My concerns were raised upon the publication of the article about 'Book burners' in relation to the campaign instigated at the CAST conference to 'suspend' the publication of the ISO/IEC/IEEE 29119 software testing standards by means of a petition and enable some open debate on the validity of these standards too the software testing profession. NOTE the wording it says suspend it does not say as your article states 'suppress' which indicates that these people signing the petition wants to "forcibly put an end to." That is not the purpose of the petition. Its' purpose is to allow those who will be impacted by these standards a voice and  input.  That I feel is fair enough for any democratic process and hence why I proud to sign this petition.

Now let me get to the part that really got me annoyed.  The use of the term "Book burners".

• Did the author of this article understand what this term means?  
• Did they understand how much of an offensive term this is?
• Was this term deliberately chosen to ensure controversy?
• Was the author naive in choosing this term without understanding it meaning?

I have been attempting over the past couple of days to attempt to answers these questions and at one point I felt it was just me being over sensitive. Then the second article was published about 'PT independence is questioned' in which the author appears, to me, to take the higher moral ground and attack some within the testing community for making some assumptions about linkedin and comments being deleted, which they quickly retracted and apologized for once it was understood as a bug within linkedin.  This did not stop the author of the article stating that the comments made were deliberately false and meant to damage the reputation of the magazine.  I can understand how the magazine editors must have felt,  however this was an opportunity for the editors of the magazine to also apologize for their attack on some within the testing community by the use of the term 'book burners', sadly none was forthcoming.

To clarify the term 'book burner' is used against those who are attempting to suppress freedom of speech.

 Book burning can be emblematic of a harsh and oppressive regime which is seeking to censor or silence an aspect of a nation's culture - Wikipedia

Can you as authors of the magazine see the irony of using this term against those who are attempting to make something open and debatable rather than hidden behind closed doors?

Another ironic aspect is that those within the testing community who I engage with are amongst those who read a wide variety of books even those books that they may disagree with.  They are also writers, publishers and authors themselves to so use such a term is so derogatory and offensive to many of these people.

All I ask from this open letter is to acknowledge that using the term 'book burners' just may have been over the top, unjustified and offensive to those with the software testing profession who care about testing. 

To end this on a positive note I do intend to continue reading your magazine and will not encourage anyone to not read it or avoid it, since that serves no purpose.  I look forward to many more great articles from your magazine and hopefully some will be edgy and promote debate so all of those within the testing profession can be encouraged to learn from all the diverse views that this profession has. 

your sincerely

John Stevenson

A professional tester

If you care about the future of software testing please look at signing the petition to get ISO 29119 suspended and enabling a debate on its content by going here - http://www.ipetitions.com/petition/stop29119

Also have a look at the professional tester manifesto here - http://www.professionaltestersmanifesto.org/ and if you agree look at signing this too.

Why Testers do not add Value

This is an article I published internally last year when there was a debate going on about the value of testers within the testing community so I thought would re-publish it here for people to have a read.

___________________

This may seem like a strange statement coming from a tester; however the purpose of this article is an attempt at justifying the need for testers on a project. Financial people might use traditional cost/benefit comparison models when recruiting people for a team or project which may not necessarily be applicable when resourcing testers.

What made me think about writing this article was something I happened to read on the Software Testing Zone website by Debasis Pradhan 1 and from Michael Bolton at the Developsense website .2

There are two comments within these articles that appear to come from a developer’s point of view about the role of testers within software projects.

The first is from Debasis blog:

Software Testing is a worthless process. It does not add any value to the project!

And one from Michael’s blog

But testers don't add value to a project;

These comments made me think about the way testers are perceived on projects and whether or not they really do add any value to a project. This led to researching how accurate these statements were and the production of this article.

The problem I have found when carrying out the research for this article is that the term ‘value’ can have different meanings depending on the context in which it is used.

If I were to ask you what is the value of a gift a loved one gave you who is no longer here?

What value would you put on that item?

In money terms it might not be worth that much but in emotional or sentimental terms, to you, it could be priceless. It is very important when reading this to use the correct context for the meaning of value. For the purpose of this article value will be measured in terms of monetary value otherwise your emotional viewpoint may get in the way of what is being said.

So the definition of value for this article will be:

Value implies monetary value.

If we take a look at a typical upfront project and see where costs are attributed and where value is added it may give a better understanding of the statement.

The first part of a project is the initial design and scope where the architect captures and translates the requirements of the customer in to technical solutions and business requirements. The value of an architect is very clear since they are the people who provide solutions to the questions raised by the business. They ensure that the requirements for the customer are met and what the customer asks for is what will be delivered. Those who work within software development may not agree with the final statement since sometimes features requested are not delivered or changes are made that affect part of the delivery. However in the majority of cases what the customer requires is delivered by the architect.

The project manager will then look at these requirements and sees which has the highest business value at the same time assessing the risk of adding the features before making the decision as to what to develop which will give the maximum value.

These technical solutions, risks and business value are then interpreted by the developers and turned into an actual project so their value can be measured in that they build the project. The developers also fix any problems and solve any issues that may occur either from the customer or the test team.

Marketing then can sell that product or may already have sold the product adding value to the company.

Some may say that testers must add value since they find problems before a product is released and as such stop a project costing more in terms of company reputation and actual monetary cost. This may appear true but testers do not actually provide solutions and fix the problems, this is the role of the architect and the developer.

So what does a tester do within a software project?

Testers look at the project being developed and raise questions and report observations as to what may not be working correctly. They look and report on areas which could cause the most risk and most expense to fix. They give priority to the main problems (defects) within the product.

However the people who fix these problems, act upon the observations and answer the tester’s questions are the ones who add value to a project.

This view is based upon a typical style of software development where all requirements are gathered upfront so what about agile style projects?

It is rather interesting that there are very differing viewpoints on this within the testing world, one side saying that if the testers are working directly with the developers and writing the test cases before the code is written then they are adding value to the project. The other side is saying they may write the test cases but they do not actually do any creative design work and as such do not add any real value to the project. My viewpoint is of the latter if the testers are not actually creating code that will lead to a function that will be used by the user in the live system then they are not adding any real value to the project.

The way in which testers could add actual value to the project when using agile is when they have close working arrangements with the actual customer since they can then suggest or inform the customer that if they did this a different way it would have more benefit. This would still not be intrinsic value unless the tester managed to get the customer to sign up for more work at an additional cost, therefore adding to the value of the project.

To conclude testers do not add intrinsic value to software development project but they prevent deprecation of the project value. If you have a project that a customer is willing to pay five million dollars for and you ship that project with x amount of high level problems. The customer then quite rightly demands the problems be fixed free of charge. Then the value of your project could go down from the initial five million to say four million, losing the company both real money and another value that cannot be measured: ‘your company reputation’.

It should be noted that there is the factor of costs of resource on projects and to keep things very simple I have excluded these other costs when making the above statement. Otherwise you would need to include the cost of equipment, building, utilities and so forth.

With testers on the project the majority of the high level problems would have been caught before being shipped to the customer and as such the value of the project would have been maintained.

Therefore testers do not add value but they certainly prevent value from being lost. It should be noted that no tester can guarantee one hundred percent bug free software. Even companies such as NASA who on the Space Shuttle program had a ratio of 10 testers to 1 developer 4 cannot give that guarantee as demonstrated by the tragic accidents that have occurred! The average ratio in the majority of projects is 1 tester to 3 developers5; therefore value will always be lost in some way from a project.

So remember those whose job it is to resource software development projects never skimp or cut costs by using less testing resources and think that you have saved yourself some money. You might just find that the 1 million dollar project you are working on becomes worthless due to lack of testing resource to ensure the value is maintained.

I will leave you with a view from James Bach, who is a software testing author and one of the main contributors in the software testing field.

‘James Bach suggests, testers help to defend the value that's already there, or help to identify ways in which value may be lacking. Testers raise questions and make observations; the people who make decisions based on those observations are the ones who add value. We help them do that, but we don't do it intrinsically on our own.’ (3)

References:

1 http://software-testing-zone.blogspot.com/2008/10/software-testing-add-value-to-project.html

2 http://www.developsense.com/2008/03/breaking-code.html

3 http://www.developsense.com/2008/10/while-back-i-wrote-post-on-breaking.html

4 http://codeidol.com/other/Software-Estimation/Estimating-Planning-Parameters/21.1-Estimating-Activity-Breakdown-on-a-Project/ (table 21-7)

5 http://www.infoq.com/news/2009/01/tester-to-developer-ratio

______________

Hopefully if I get myself sorted my next article will be about how we can learn to improve how we use exploratory testing from watching how children play and learn.