What do we mean by testers learning to be technical?

I recently read a post by Justin Rohrman entitled "Do testers need to code".  The following line gave me some concerns:

"So, I think that testers do need to learn some programming skills now."

Now I wish to begin by stating I am not against testers learning to code  if that is something they are interested in and they want to learn.  On the other hand it may not be the best value skill you could bring to a team, especially if you already have 5-6 exceptional coders on your team.   Justin within the article tries to align to this thought with the following:

"My new feeling is that testers need to learn to program, or at least become more technical –write SQL, use developer tools effectively, read code — for any sort of longevity in the field. "

I have blogged about this in the past- 'A discussion on do tester really need to code.' in which I talk about adding value by learning the syntax of code.  I was asked on twitter by Marcel Gehlen to provide an example of what I meant by this.  I replied with the following:

'I understand the syntax of the french language but I am terrible at speaking it.'

The main point of this update to the 'testers need to code' debate is around what we mean by being 'more technical'?  To the majority of people and from the many articles I come across this implies learning to code.  I feel this is too narrow and at the same time limits the potential and opportunities testers may have.  As an example if you now have a team in which everyone can code and there is a need to reduce the team due to financial and business pressures.   If the decision being made is based upon coding skills then I feel testers could be at risk.

Where does this learning to code end?    In the modern development world of 'devops' are we now going to state that operational people and marketing people should now learn to code?  we could end up with an over saturated market of half decent coders.  What skills are going to make testers stand out and be seen to be adding both team and business value?

Taking a step back what do we mean by coding?

I can writing scripts in bash which to me is similar to writing test script in the sense of logical following steps.  At the same time I understand the command line interface to enable me to deploy a test environment.  Going forward into the devops model I can write dockerfiles to create containers..

#
# Super simple example of a Dockerfile
#
FROM ubuntu:latest
MAINTAINER Andrew Odewahn "odewahn@oreilly.com"

RUN apt-get update
RUN apt-get install -y python python-pip wget
RUN pip install Flask

ADD hello.py /home/hello.py

WORKDIR /home

Example from http://odewahn.github.io/docker-jumpstart/building-images-with-dockerfiles.html

I understand how to use ansible to deploy an environment to enable me to test and run automated checks.

These are, what many would class as, highly technical skills.  However as Jerry Weinberg just posted about 'Becoming a Better programmer; ' there are other skills that are of value to a team and to the business that many would not class as technical.  The ability to get people to work well together and collaborate and communicate is a highly desirable skill to have.  Since it is classed as a soft skill not a lot of emphasis is placed upon this as a skill testers should learn.

What about analytical skills, as more and more companies move to a devops model for software delivery, the skill of being able to determine patterns within the software from all the data and collate that data to help improve try and improve the quality is a highly demanding and technically challenging skill.  I can see this skill becoming highly desirable in the future.  Maybe a tester could invest their time learning this skill?

Then we have the other areas such as an understanding of human behavior to help influence the design .  Learning about social science and systems thinking to help see the big picture and overcome obstacles for the team.   I have been running a series of blog posts on skills testers should learn away from the traditional 'must code' school of thought.  Using your time wisely to practice and improve these skills can help you become a better tester and more importantly show your value to the business.

A few examples are given below (feel free to browse my blog for other examples)

• http://steveo1967.blogspot.com.co/2016/06/testing-skills-10-persuasion-and-selling.html
• http://steveo1967.blogspot.com.co/2015/11/testing-skills-8-being-skeptic.html
• http://steveo1967.blogspot.com.co/2015/10/testing-skills-7-reflection.html

Also with your testing skill set you could mentor, coach and teach others about testing and how they can use that to help improve the software.  Set yourself a goal of coaching the programmers in simple test techniques that they could look to add to their frameworks to enable you to focus more on the difficult to find issues using exploratory testing .

Again I wish to state I am not against tester learning and if they feel they have a passion towards coding then go and do it.   At the same time think about what else they could learn that adds both team and business value.

This article was first publish in Jan 2015 on the Ministry of Testing Website - http://www.ministryoftesting.com/2015/01/qualitative-research-coding-software-testing/

Qualitative Research Coding and Software Testing

Coding is used extensively in the field of social science for qualitative analysis.  Coding is not defined in the same ways as it is for software development.  The following quote by Saldaña provides a useful definition of coding within the social science world:

A code in qualitative inquiry is most often a word or short phrase that symbolically assigns a summative, salient, essence capturing, and/or evocative attribute for a portion of language-based or visual data  [1]

“By “language-based or visual data”, Saldaña means video, audio recordings or written notes.   Once the information has been gathered the researcher then assigns ‘a word or a short phrase’ to represent sections of the data. Then the researcher examines and categorises the codes to see what patterns and theories emerge.  These activities together are known as qualitative research coding.   For example if a researcher were studying the morals of teenagers, the researcher would record conversations and then use coding activities to see what patterns emerge.  Based upon these patterns the researcher would form theories about the morals of teenagers.

Testers using this type of coding may find it helps to label; organise and classify their testing observations.  It provides structure not only to the testers’ observations, but also to the process of observing.  Observation and gathering information are important aspects of testing.

The changing minds website describes qualitative coding as:

Coding is an important technique in qualitative research such as anthropology [2), ethnography [3] and other observer and participant-observer methods. [4]

Strauss stressed the importance of coding when carrying out qualitative analysis:

Any researcher who wishes to become proficient at doing qualitative analysis must learn to code well and easily. The excellence of the research rests in large part on the excellence of the coding. [5]

The same can be applied by testers when analysing their testing effort, there is a need as a tester to be able to ‘code well and easily’. Testers should examine the testing evidence that has been gathered and use coding to formulate flexible theories and ideas of the behaviour of the software.

There appears to be some comparisons between qualitative research coding and exploratory testing [6]; the table below describes some of these comparisons.

Social Science	Exploratory Testing
Evidence based upon immersion in the culture under investigation	Evidence based upon exploring the software under test
Theories formed based upon the evidence gathered during immersion	Theories formed based upon the evidence gathered during exploration
Theories adjusted and altered as new evidence is gathered	Theories adjusted and altered as new evidence is gathered
Further investigations take place to uncover evidence that may support or disprove current theories	Further exploration takes place to uncover evidence that supports or disproves current thinking about the behavior of the software under test
Coding and classifications of evidence to identify patterns	Coding of evidence from exploratory sessions to identify patterns and risks

Table of comparisons between social science research methods and exploratory testing

Testers can utilise social science coding to analyse the testing evidence gathered to help form theories about the behaviour of the software. These theories can be presented to stakeholders to provide valuable information and support decision-making.

When people first start to use social science coding they can find it complex and daunting.  To help simplify coding Chris Hann created the following diagram:

Coding pyramid [7]

The diagram shows the different levels of coding that social scientists go through to form and reform theories.

Testers can use social science coding not only for test execution, but also for other testing activities such as test planning, testing discussion and test reporting.

Coding in action

The following is an example of using level 1 coding for user stories.

User Story

As a user

I want to login in securely

So that my private information is kept private

A tester using coding uses the following codes for this user story. (Each code is separated by the | symbol)

Codes

Security | Login | Operations |Function

The tester then looks at more user stories and codes:

User Story

As a user

I want to record a currently playing live show

So that I can watch the show at a later time

Codes

Recording | Live TV | Device Remote | Operations | Data |Time | Platform | Functions

User Story

As a user

I want to playback a currently recorded show

So that I can watch the recorded show now

Codes

Recording | Playback | Device Remote | Operations |Time | Data | Structure | Interface

The examples above make use of the SFDPOT [8] heuristic.  This heuristic is a useful way of utilising social science coding to identify testing coverage gaps.   When coding it is acceptable to assign multiple codes to each piece of information or evidence gathered.

1. The process of coding can be defined in the following steps: (taken from [9])
2. Decide which types of coding are most relevant
3. Start coding
4. Create a start list of codes
5. Generate categories (pattern codes)
6. Test these categories against new data (start with contrasting data early on!)
7. Write about categories/pattern codes in a memo to explain their significance

The following is an example of a tester using these steps:

The testers’ initial testing effort showed that a particular API appeared to work correctly with the customer data set used.  The tester coded this as ‘API interface: customer data set ingests’.  The tester formed a theory that the API had been implemented and appeared to work for that data set. The tester then tested their theory using a different customer data set and the behaviour was inconsistent, this led to a change in the testers’ theory based upon the evidence gathered. This is level 2 and 3 on the coding pyramid diagram.

Memos and questioning

Testers when carrying out coding can use ethnographic research methods [10] by asking themselves the following questions.

• What is the software doing?
• What is the software trying to accomplish?
• How does the software accomplish this?
• Does the user understand what is being accomplished?
• What assumptions does the software make about the user?
• What surprises you about how the software is behaving?
• What do I see going on here? (To track your assumptions)
• What did I learn from the notes I have taken?
• What intrigued me? (To track your positionality [11] – where do you stand, what biases could be at play?)
• What disturbed me? (To track your tensions, beliefs, attitude)

These questions have been adapted based upon work by (Sunstein and Chiseri-Strater, 2007: 106) Field Working: Reading and Writing Research. [12]

Categorisation, level 4 on the coding pyramid, is the concept of tying together a number of observations and seeing if they have any common characteristics. These characteristics or connections in social science coding terms are called memoing.

Glaser defines memoing as:

(A memo is) the theorising write-up of ideas about codes and their relationships as they strike the analyst while coding… it can be a sentence, a paragraph or a few pages… it exhausts the analyst’s momentary ideation based on data with perhaps a little conceptual elaboration (Glaser, 1978: 83) [13]

The following gives a testing example of this:

After testing a User Interface, the tester defines the following codes from the gathered testing evidence:

UI inconsistent | error messages unclear | undermined feature behavior

The tester memos these codes as ‘unwarranted user behaviour.’ Ideally though, the tester should write more detailed memos than the example given.  The purpose of a memo is to write down your thoughts and reflections on the information you have gathered.

The more extensive the testers’ memos are the more they can form concrete theories.  This aspect of coding can prove to be valuable for testers, since the completed memos can be used as evidence of what the system they are testing is really doing.

Coding can be used when analysing the evidence gathered during testing; the tester can group together and code similar behaviours and observations to see if there are any patterns.  These patterns can be used by the tester to guide their testing effort and provide focus for areas that they may find useful to explore.

Ian Day in his book Qualitative Data Analysis – A user friendly guide for social scientists [14] makes the following statement about coding.

The term ‘coding’ has a rather mechanical overtone quite at odds with the conceptual tasks involved in categorising data. This arises from the association of coding with a consistent and complete set of rules governing the assignment of codes to data, thereby eliminating error and of course allowing recovery of the original data simply by reversing the process (i.e. decoding). Qualitative analysis, in contrast, requires the analyst to create or adapt concepts relevant to the data rather than to apply a set of pre-established rules.

Since coding is a conceptual approach, having pre-defined rules or processes before the tester has the evidence is not something that fits easily in to a qualitative research approach.   Codes are relevant to the actual evidence that has been gathered rather than based upon assumptions of what that evidence may be.

Summary

The use of qualitative coding in testing provides many benefits that testers can utilise across all testing activities, some of these benefits include:

• Pattern Recognition
• Theory Forming
• Critical Thinking
• System Thinking
• Testing Gap Analysis
• Meaningful reporting
• Evidence gathering

Learning about social science coding and applying it in practice can be a powerful tool in a testers testing skills portfolio.  Those who promote that testers should learn to code are correct; nevertheless they may want to explore alternative coding approaches.

For those who wish to learn more about the connection between social science and software testing I recommend the following articles and books as useful starting points:

• A complete beginner guide to qualitative analysis – Taylor and Graham R. Gibbs University of Huddersfield (web site)
• Analysis of Qualitative Data for Beginners – Demola Akinyoade – Department of Peace and Conflict Studies Afe Babalola University, 30th June 2005 (Web PDF)
• Software Testing as a Social Science – Cem Kaner – STEP 2008 (Slides)
• What Testers can learn from Social Sciences – Huib Schoots – TestBash 2.0 – 2013 (Video)
• What Testers can Learn from Social Sciences – Part 1 – Huib Schoots – 2013 (Series of Web Articles)
• Looking to Social Science for help with Metrics – Justin Rohrman – CAST 2014 (Video)

With special thanks and appreciation for their help and patience in making this a much more complete article than it was to begin with:

• Michael Bolton: http://www.developsense.com/ Twitter: @michaelbolton
• Huib Schoots: http://www.huibschoots.nl/wordpress/ Twiiter: @huibschoots
• Duncan Nisbit: http://www.duncannisbet.co.uk/ Twitter: @DuncNisbet

References

1. The Coding Manual for Qualitative Researchers – Saldaña, 2013 –http://www.amazon.com/The-Coding-Manual-Qualitative-Researchers/dp/1446247376
2. What is Anthropology – American Anthropological Association (website – Last accessed (Aug 2014) – http://www.aaanet.org/about/whatisanthropology.cfm
3. Brian A. Hoey. “A Simple Introduction to the Practice of Ethnography and Guide to Ethnographic Fieldnotes” Marshall University Digital Scholar (2014): 1-10. Available at:http://works.bepress.com/brian_hoey/12
4. Changing Minds – Ethnographic coding –http://changingminds.org/explanations/research/analysis/ethnographic_coding.htm
5. Qualitative Analysis for Social Scientists, 1987, p. 27 – Anselm L. Strauss –http://www.amazon.com/Qualitative-Analysis-Social-Scientists-Strauss/dp/0521338069
6. What is Exploratory Testing – James Bach –http://www.satisfice.com/articles/what_is_et.shtml
7. Techniques and Tips for Qualitative Researchers – Chris Hann –http://qrtips.com/faq/FAQ–code%20terms.htm
8. How Models Change – Michael Bolton –http://www.developsense.com/blog/2014/07/how-models-change/
9. 8 Qualitative codes and coding (2014) – Heather Fordhttp://www.slideshare.net/hfordsa/qualitative-codes-and-coding
10. Are testers’ ethnographic researchers? Stevenson (2011) –http://steveo1967.blogspot.com/2011/01/are-testers-ethnographic-researchers.html
11. What is positionality in practitioner research? – Dissertation Scholar (website Last accessed August 2014) http://dissertationscholar.blogspot.mx/2013/04/what-is-positionality-in-practitioner.html
12. Field working Reading and writing research – Sunstein and Chiseri-Strater (2007: 106) – http://www.amazon.com/FieldWorking-Reading-Writing-Research-Edition/dp/0312622759
13. Theoretical Sensitivity: Advances in the Methodology of Grounded Theory – Glaser (1978: 83) http://www.amazon.com/Theoretical-Sensitivity-Advances-Methodology-Grounded/dp/1884156010
14. Qualitative Data Analysis: A User Friendly Guide for Social Scientists – Ian Day (1993)http://www.amazon.com/Qualitative-Data-Analysis-Friendly-Scientists/dp/041505852
    X

A discussion on do testers really need to code.

Recently there have been a few posts talking about testers learning to code.

• Rob Lambert: - Why Testers Really Should learn to Code
• Fiona Charles - Why I believe its good for testers to learn to code
• Anand Ramdeo - Do Software Testers need to learn coding
• Maaret Pyhäjärvi  - Learning to Code - Just Learning

I find these discussions interesting and at the same time very worrying.  I have seen many posts that say testers ‘should’ or ‘need’ to code as shown from some of the examples above, excluding the one from Maaret. To make it clear none of the posts dictate that testers MUST code, but they do appear to give an impression that for testers to make themselves employable they sure should learn how to code.

My temptation was to reply to each of these posts as comments but in the end felt that would not be a great platform to get across my concerns and views on this topic, a topic that keeps coming around again and again.  I am not against testers learning to code but feel that going down this path could have a significant impact on the testing profession in the future.

What do people mean when we say "Learn to Code."?

To the majority of people learning to code means learning a programming language and being able to write an application/script in that language. I can see,  to some, in certain situations where that can be very useful, however do we really need to learn coding or would just having an understanding of the syntax of code be just as useful?

What does this mean when I say syntax?

The Oxford dictionary defines Syntax as:

The arrangement of words and phrases to create well-formed sentences in a language:

Or

The structure of statements in a computer language.

http://www.oxforddictionaries.com/us/definition/english/syntax

E.g.: the syntax of Java or Python.

Having an understanding of the structure of a programming language does not necessary mean you can code an application in that language but you should be able to read and follow lines of code and have a fair idea of what it means and what it does.  The first key point in this article is that we need to move away from creating generic testers who all code to ones that have a fair grasp of the syntax of languages.  This way we will not excluding, what could be, great testers, by turning them away from testing by insisting they learn coding, which is something they may have little or no interest in.  An English major for example could make a wonderful tester by understanding the subtle written words given by a customer to the project team and providing a multitude of creative testing ideas.  Do we really want to exclude these types of people from the testing profession?

This leads on to my next concern, if we insist that all testers need to learn to code we miss out on building diverse teams with many skills that compliment and sometimes conflict each other.  This enables much deeper critical thinking processes and ideas to emerge.  It is a concern that if we insist on testers all learning to code or need to code, then becoming an entry barrier into testing then we could lose a great deal of diversity among software development teams. We instead get a set of ‘cookie cutter’ clones who all code, think and work in similar ways.  This will diminish the testing talent pool, as those who would love to become a tester get set aside for an ex developer who can code but now wants to try their hands at testing. This is not to say that ex developers do not make great testers, we just to need to welcome people with a diverse range of skills and expertise into this great career rather than limit it to those with a coding background or skill set.  Also what do we mean by coding?  I touched on this earlier, within software development coding normally indicates being able to program.  However in social science and especially ethnographic research coding has a different meaning.

Coding is an important technique in qualitative research such as anthropology, ethnography and other observer and participant-observer methods.

http://changingminds.org/explanations/research/analysis/ethnographic_coding.htm

If we insist that testers need to learn code, how about we twist this around and say that testers need to learn social science methods and use that in our testing activities.  To quote the title of the article by Rob Lambert:

“Why testers should really learn to code”

If the code Rob was referring to above was the social science technique and he stated in his article that all testers ‘should really’ learn to do social science coding.  How much debate would that have caused?  If others insisted that every tester should really have this skill and that without it you have a far less chance of being employed as a tester, how would you react? Does a tester having this analytical skill make them any less important to a development team than someone who can do programming? I am unsure if this is the case, since the cognitive thinking skills needed to do programming and to do testing are different as mentioned extremely well in the article by Maaret.

One argument put forward is that testers who can code can create their own tools to do tasks such as data generation, performance, load or stress. I do not see a problem with this but I would add that there are many FREE tools available that may or could do the job without the need to re-invent the wheel. For example for data generation I use Excel  and PerlClip both of which suit my needs.  I have a tool inventory that for the majority of my tool requirements perfectly meets my needs.  I have seen many cases of people writing tools or scripts for the sake of doing so instead of researching or asking advice on tools that are already available.  This is not to say there are situations where creating or writing a tool could be more useful and time saving.  I have in the past done this and my programming skills are not great and I would rate as rather mediocre.

We have to be careful about the barriers we put in place for those who show an interest in having a career in testing.  Sure they need to have certain characteristics such as:

•  A willingness to learn.
• Self-improvement
•  Critical and creative skills
• Plus many more.

Classifying this to one particular skill such as programming rather than a characteristic, such as analytical thinking or logical thinking in my opinion is very narrow minded.

One other aspect that I have not seen mentioned in any of the articles is the issue of bias caused by thinking in a similar way.  If there is a continuing trend to insist that testers learn to code then there could be a convergence towards similar thinking for development and testing.  This may lead to bias in the approach to testing where the effort and focus is on the logical way in which the code works rather than an outward view of how the user may, or may not interact with the application.  Humans are not logical and will not follow the path you expect them to do so.  This is what I feel distinguishes good from great testers, the ability to think critically and apply creative ideas in a way that exposes flaws or issues within the system under test.  If we  continue to insist that testers should really code we could end up with a set of people developing software in which everyone thinks in a similar way and there is no movement from the well-trodden path.

If people have no interest in learning about coding and instead have a passion for testing and how to improve testing then we need to be careful about the message we are sending out from this profession.  I would rather have people working with me who are passionate, driven to learn and improve themselves and others than an excellent coder who want to do just a 9 to 5 job and no more.

I will finish this post with the following quote:

“We are not supposed to all be the same, feel the same, think the same, and believe the same. The key to continued expansion of our Universe lies in diversity, not in conformity and coercion. Conventionality is the death of creation.” Anthon St. Maarten - Divine Living: The Essential Guide To Your True Destiny

EDIT : Corrected some typos.