Volunteers needed for a FREE remote testing workshop concept

I am looking for two or three people to act as guinea pigs for a remote testing workshop concept I am working on.

The workshop will be FREE but require a couple hours of your time each week for about seven weeks.  I am hoping to start this workshop be the end of August 2015.

• Ideally the participants will be fairly new to testing with less than two years experience. 
• Able to work within UK timezone (Early evenings)
• Willing to provide feedback about the workshop.

Why am I doing this?

I want to give back to the testing community, I understand it can be difficult for those new to testing to develop testing skills and techniques that they can apply directly to their work.  The concept of the workshop is that there will be series of self learning exercises and some challenges that they can apply to their testing in their current role.   Each week the group will get together using video conferencing to discuss what they learnt and share it with others in the group.

I currently have one volunteer and I am looking for 2 or 3 more, if you are interested then please reach out to me via twitter @steveo1967 and we can discuss in more detail.

**UPDATE - it is now full - please contact me via twitter if you are still interested and I can create a waiting list.**

Using Aspects of Lean Coffee to Drive a Retrospective

Those who have been following my blog may have noticed that I have not updated it as frequently as I normally would, or would like to.  There are been a variety of reasons for this, I am writing a book which is taking a lot of my spare writing time, I have not had much of interest to write about and finally over the past year my role within testing has changed to one of being a scrum master for agile teams.  It has been an exciting and challenging role which over time I may blog about.  It has been strange since people have commented that I appear to be a natural in this role and part of me thinks this could be due to my skills as a tester that helps in this role.  I am still trying to get involved in some testing and hopefully there will be opportunities for me to do this.

This post is about an experiment I attempted with a scrum team recently for the retrospective.  I am unsure if I obtained this approach from somewhere, or it was just an idea I came up with.  I have decided to share it  via my blog for others to see if they find it useful.

A common approach to retrospectives is to get the team together and discuss what they felt went well, what could they improve and to have some actions for the team.   The way I have run this in the past is we take each of these statements and get some feedback from each members individual perspective.   Some of the questions that can be used can be found here - http://www.benlinders.com/2013/which-questions-do-you-ask-in-retrospectives/

What I found was these retrospectives did not appear to be that engaging and after awhile became a little stale.  So I had an idea to change the dynamics by introducing a lean coffee (http://leancoffee.org/) format.  Lean Coffee is a structured but agenda less meeting, which consists of three steps:

(1) Set up a Personal Kanban

This basically means create a series of post-its to represent

• What we are currently discussing, "In Progress"
• What we have discussed "Done"
• What actions have comes from the discussions "Actions"

(2)What to Discuss

This can be any topic  that you wish to discuss, for the first retrospective I ran in this way I used the following:

• Shout outs
• What was good.
• What was Bad
• Improvement Ideas

The first four were placed upon the wall and then I gave the team members a set of post-its and pens to write down their thoughts for each of these titles. One post-it per comment

Shout Outs:

Who do you know who went beyond their normal day to day work to help the team.  This could be someone within the team or outside our team who helped support the team.

What was good

What do you feel went well in the sprint.

What made you feel good about what the team did,

What was bad

What do you feel went wrong in the sprint

What made you feel bad about something in the sprint

Improvement Ideas

How can we improve what we do?

What ideas do you have to make the team better?

(3) Vote and Talk

Once we had done this everyone was encouraged to go up and look what others had written and at the same time we decided to group similar comments together. After this we had some cookies, always bring cookies to a retrospective, I moved the In progress post-it over to the 'shout outs' column and people started to explain what they have written,  It was interesting that it was someone outside our team that people felt had added value to the team.  From this we as a team sent an internal gift and a recognition for their work.

Once we had finished with the 'shout outs' we moved this column to 'done' and move the 'in progress 'sticky to the next column  'What was good', we followed the same approach and everyone had opportunities to discuss what had been written.  At the same time any actions that came from the discussions was added to the Actions post-it.  This approach was then done for the rest of the columns 'What was bad' and 'Improvement ideas'.

The dynamics of the team during these discussions was far greater than at any of the other retrospectives and I felt as a scrum master it worked really well to encourage the different members of the team to participate.  Part of being a scrum master is to keep the various ceremonies you have for the team 'fresh' and 'interesting' and in this case it appeared to work well.  Will it have the same affect the next time, I am unsure however it is one more tool I can use to help the team. Let me know of your approaches to keep the team motivated and encouraged.  Also if you try this approach let me know how it works for you and your teams.

In Lean coffee you would normally VOTE on the topics you are interested in using dot voting,

"Each participant gets two votes. You can vote twice for the same thing or for two different topics. Simple put a dot on the sticky you are interested in. Tally the dots. "

Update: I have run this approach a couple of times since the first trial and  it appears to work very well.  I have tried to prevent it from being too routine and have changed the questions that are posted.  I have used titles such as "What was positive about the sprint" , "What was negative" "If we could improve on one aspect what would it be".  It is important to create varieity in the retrospective even if you are using the same approach.  It helps to keep the team interested and motivated.  Cakes and cookies help too!

What drives us? Extrinsic and intrinsic motivation.

After recently presenting a workshop at the Lets Test conference on Self-Learning one of the concepts that people found difficult to grasp was the difference between extrinsic and intrinsic motivation.  It may be due to the time constraints of the workshop or I did not explain clearly enough.   Therefore I decided to put together this article to give a little more detail about extrinsic and intrinsic motivational factors. 

One psychology aspect of motivation is to work out what motivation factors are intrinsic and which are extrinsic.  To begin with it is useful to define what is meant by extrinsic and intrinsic.

"Extrinsic motivation is ‘external’: people – in this case athletes – are driven to succeed by factors from outside i.e. money, prizes, acclaim, status, praise." 

"Intrinsic motivation comes from within i.e. an athlete driven by a need to succeed because they want to be the best and are not overly concerned by financial or ego boosts."

The Sports Mind - Extrinsic vs Intrinsic motivation

Many people and organizations mistakenly assume that people are motivated and driven by financial rewards and to some extent they are.  People do want to be financially rewarded for doing work. In the majority of cases money works as a motivation factor for people to get out of bed to go to work and do the normal everyday tasks.

As Kamenica points out:

"It is helpful to distinguish those tasks that people certainly do not want to do unless they are paid for them from those that people may or may not engage in.” 

Behavioral Economics and Psychology of Incentives -Emir Kamenica - 2012

However there are studies which show that rewarding someone with money for something they have a passion for can demotivate and make them less effective. 

“...tangible rewards tend to have a substantially negative effect on intrinsic motivation (…) Even when tangible rewards are offered as indicators of good performance, they typically decrease intrinsic motivation for interesting activities.” 

A meta-analytic review of experiments examining the effects of extrinsic rewardson intrinsic motivation. Deci EL1, Koestner R, Ryan RM

Since extrinsic rewards form only a small part of what motivates people it is important to find out what makes people 'tick'  How can you set an environment that encourages peoples passion and motivates them to be the best. Providing people with opportunities to pursue their passion be it time to study or learn can have a positive impact on a team as long as others in the team are given similar opportunities.  The psychology of motivation is complex and what may motivate someone may not motivate someone else.

If you find something that is of interest you and you want to become passionate about it or if someone on your team is showing a passion for a certain activity it is worth focusing on the intrinsic motivation rather than the extrinsic.  It is also important to be aware of the over-justification effect

"The catch-22 of extrinsic motivation. The over-justification effect occurs when someone naturally has a passion (intrinsic motivation) to see something through, but is offered a reward for its completion. Thus rendering them less effective. For instance, if an employee loves writing on your corporate blog but you decide to financially compensate them for each post. There is a chance they will find the writing less enjoyable. Since they have to be bribed into writing, then the task must not be worth doing for its own sake." 

12 Psychology Concepts for Improving Employee Motivation -Bradley Gauthier - August 17,2011

One way to inspire individuals is by using unexpected rewards. Unexpected rewards can inspire and motivate people; the key is to not expect a reward. For example if someone has done something that you feel was outstanding offering to take them for lunch and paying or giving a small gift of appreciation can go a long way to keep them motivated.  One approach that can be useful when showing your appreciation for someone is to say how much you appreciate their hard work rather than how clever they were.  This makes people value the effort more than anything else. You can use this kind of reward system to encourage the right behavior but it is important to realize that there is a thin line between unexpected and expected rewards.

“Yes, sometimes rewards do work, especially if people really don’t want to do something. But when tasks are inherently interesting to us rewards can damage our motivation by undermining our natural talent for self-regulation."

How rewards can backfire and reduce motivation -Psyblog

When thinking about the testing you are performing it is worthwhile investigating the motivating factors.  If the testing you are carrying out a scripted approach then your motivation could be linked to extrinsic rewards rather than intrinsic rewards.   It is worth asking yourself the following about the testing you are performing:

"Is the task at hand routine?  That is, does accomplishing it require following a prescribed set of rules to a specified end?" 

Daniel Pink -Drive

For these types of tasks extrinsic rewards can work.  As a tester you should question if testing is really this type of task? Read the following questions:

• -When you are performing testing activities what is it that drives you? 
• What gives you the most joy and value to yourself in the testing you are doing? 
• Is it the satisfaction you get internally from uncovering how the system is working or not working? 
• Is it the ability to be autonomous in your exploring of the software?
• Or is it something else that drives you to carry on with your investigations?

If you are nodding to any of these then maybe the testing you are performing is linked to your intrinsic motivation.   This is different from the feelings you may get if carrying out step by step test scripts.  

Daniel Pink sums up what intrinsic rewards means to the individual.

"It concerns itself less with the external rewards to which an activity leads and more with the inherent satisfaction of the activity itself."  

Daniel Pink -Drive 

As an added complication Carles Malet described three motivational forces in his article "Motivation from Maslow to PerezLopez".

• Extrinsic motivation: when individuals act prompted by an external reward (or punishment), such as wages or improvements in the labor conditions.
• Intrinsic motivation: linked to the satisfaction that individuals obtain when performing certain tasks. The intrinsic motivation is linked to the human need of learning.
• Transcendent motivation: when the action is directed towards satisfying needs of other human beings. The transcendent motivation is linked to human generosity and the inner call for serving other human beings. Parents will recognize transcendent motivation patterns in their acting with their children, and so will do senior supervisors when empowering employees and charting their career plans.

Adding the third motivation factor is an interesting one since it plays on our human nature to want to help and support others.  This as the example explains is apparent in our nurturing instinct where we get satisfaction for helping our offspring.  In the software testing industry I have seen many examples of this with people providing their time freely to support and help others to learn, rather than being inwards and looking for their own learning opportunities.   People depending on the context will apply different weighting to each of these motivational forces and being able to understand and know which has more significance to individuals and to yourself can help drive your and others passion.

Some of this material has been taken from the next chapter of my book – The Psychology of Software Testing – Building passion, due for publication July 2015.

Technical and Non-Technical Testing Skills

One of the sessions at the Romanian testing conference in Cluj last week was a debate on do testers need to be technical or non-technical. It was an interesting debate and one which causes some great discussions.  One of the first observations was many of the people present were willing to come forward and present the case that testers need to be technical.  The framing of the definitions to me made it difficult to choose which side to present for, since no one had come forward for the non-technical side I put myself forward along with Richard Bradshaw (@friendly tester)  and a couple of other people,  unfortunately their names have escaped me.

The debate was set in the style of one side presented their argument the other side had a chance to reply and then present their side to the various statements that were shown on the screen.   It was a lively and interesting debate and I will keep to the end of this article to let you know which side was voted the most persuasive.

The reason for this article is the issue I have with labeling people as technical or non-technical as if one is better than the other.  I have come across disparaging remarks made against those who see themselves as being non-technical especially if they do not use a computer programming language.   There are the endless debates at conferences, on twitter, in testing forums in which people say you cannot be a tester without knowing how to write code in a programming language.  It is as if you are seen as a second class tester if you do not code or wish to code. Worryingly is this also being reflects in career prospects for testers with many roles requiring testers to become programmers and code. 

The’ testers should code’ debate has been discussed many times and on my blog I have written many articles about this very subject.  (http://steveo1967.blogspot.co.uk/2014/06/a-discussion-on-do-tester-really-need.html, http://steveo1967.blogspot.co.uk/2012/11/testers-should-learn-code.html) However this to me detracts from the issue at hand and what is meant by being technical.  During the debate the argument I presented started with the concept that the word technical has different meanings and that it is dependent on context.   This during the debate was missing from the technical and non-technical statements presented on screen for the debate.  (I did not manage to capture the statements due to being involved in the debate).  One of the assumptions I reached during the debate was that to many a technical tester has one of the following skills:

• SQL
• Programming
• Performance
• Security
• In depth knowledge of the system under test

With these skills the conclusion made was this enabled them to be a better, faster tester who could test the system far better than someone who was non-technical.  These could be good skills for a tester to have and I am sure that in some situations they have value, however they are not the only technical skills a tester could have.

I questioned this conclusion and asked what about those who understand the business, user behavior, the domain – for example finance and the ability to communicate and sell the benefit of testing.  I asked if these were classed as technical or non-technical.   Many said there were non-technical; I disagreed and stated in the right context each of these areas could be technical.  Understanding human behavior and being able to  work out how humans interact and what is their motivation can be a very technical skill. Understanding the value of something to the business and the reasoning why it needs to be done for the benefit of the company is a technical skill.  

In summary the reason for this article is that we in the testing profession need to resist being labeled as either technical or non-technical and simply state we are testers driven by context who apply the relevant skills for that context. 

For those wanting to know the debate was won by the non-technical side.

I would also like to thank the organizers of this conference for a wonderful and welcoming experience.  If you have never been to the Romanian Testing Conference try to come along and attend,  the participates were some of the most engaged testers I have come across at conferences. 

What Interests you? - A Short post.

One of the topics I am currently researching is motivation and how we become passionate about certain activities and ideas.  Part of this research is for my workshop at Lets Test.  One of the most interesting techniques  I have found for uncovering your passion is to use the following simple question:

"What interests YOU?"

What thoughts occur in your mind when you ask yourself that question?

Now ask yourself what YOU can do explore this further? The key to uncovering your passions is to find out what you like and pursue it. This does not mean it has to be work related, it could be doing charity work, travelling or some other activity which interests you and gives you a warm happy feeling.  Once you have found something that interests you, it is a good idea to find ways to write about either privately or publicly.

There are ways you can use the answer to that question to motivate you to do more. Look at talking to others with similar interests by joining clubs or organizations that have similar passions and interests as yourself.  As a tester it is also good to attend events where people have different perspectives and views to your own to expand your knowledge.  If your passion is an subject related to your work then it could be a good idea to do a show and tell about the subject in the workplace, this may encourage others with an interest in the subject to approach you.  There are many opportunities as a tester to talk about your passion, if your passion is testing.  I get regular opportunities to talk about my own passions at conferences, some of these are because I have submitted to talk and others where I have been invited to talk.  To begin with I started small and presented at local meet up events in front of a few people and once I got experience of this I moved on to giving quick talks at conferences before feeling confident enough to talk in front of big audiences. Talking in front of an audience is not for everyone but it is a good way to share your passion with others and I do recommend you try it, even if you only do so with a few people. You never know you may enjoy it and this will motivate you to learn more about hat interests you.

In my workshop I will be discussing this in more detail and investigating intrinsic and extrinsic motivation.  We will be playing games, having fun and hopefully learning.  Look forward to see those lucky participates in Sweden.

If you cannot make it I encourage you to reply in the comments about what you use to inspire and motive you and others.

This article was first publish in Jan 2015 on the Ministry of Testing Website - http://www.ministryoftesting.com/2015/01/qualitative-research-coding-software-testing/

Qualitative Research Coding and Software Testing

Coding is used extensively in the field of social science for qualitative analysis.  Coding is not defined in the same ways as it is for software development.  The following quote by Saldaña provides a useful definition of coding within the social science world:

A code in qualitative inquiry is most often a word or short phrase that symbolically assigns a summative, salient, essence capturing, and/or evocative attribute for a portion of language-based or visual data  [1]

“By “language-based or visual data”, Saldaña means video, audio recordings or written notes.   Once the information has been gathered the researcher then assigns ‘a word or a short phrase’ to represent sections of the data. Then the researcher examines and categorises the codes to see what patterns and theories emerge.  These activities together are known as qualitative research coding.   For example if a researcher were studying the morals of teenagers, the researcher would record conversations and then use coding activities to see what patterns emerge.  Based upon these patterns the researcher would form theories about the morals of teenagers.

Testers using this type of coding may find it helps to label; organise and classify their testing observations.  It provides structure not only to the testers’ observations, but also to the process of observing.  Observation and gathering information are important aspects of testing.

The changing minds website describes qualitative coding as:

Coding is an important technique in qualitative research such as anthropology [2), ethnography [3] and other observer and participant-observer methods. [4]

Strauss stressed the importance of coding when carrying out qualitative analysis:

Any researcher who wishes to become proficient at doing qualitative analysis must learn to code well and easily. The excellence of the research rests in large part on the excellence of the coding. [5]

The same can be applied by testers when analysing their testing effort, there is a need as a tester to be able to ‘code well and easily’. Testers should examine the testing evidence that has been gathered and use coding to formulate flexible theories and ideas of the behaviour of the software.

There appears to be some comparisons between qualitative research coding and exploratory testing [6]; the table below describes some of these comparisons.

Social Science	Exploratory Testing
Evidence based upon immersion in the culture under investigation	Evidence based upon exploring the software under test
Theories formed based upon the evidence gathered during immersion	Theories formed based upon the evidence gathered during exploration
Theories adjusted and altered as new evidence is gathered	Theories adjusted and altered as new evidence is gathered
Further investigations take place to uncover evidence that may support or disprove current theories	Further exploration takes place to uncover evidence that supports or disproves current thinking about the behavior of the software under test
Coding and classifications of evidence to identify patterns	Coding of evidence from exploratory sessions to identify patterns and risks

Table of comparisons between social science research methods and exploratory testing

Testers can utilise social science coding to analyse the testing evidence gathered to help form theories about the behaviour of the software. These theories can be presented to stakeholders to provide valuable information and support decision-making.

When people first start to use social science coding they can find it complex and daunting.  To help simplify coding Chris Hann created the following diagram:

Coding pyramid [7]

The diagram shows the different levels of coding that social scientists go through to form and reform theories.

Testers can use social science coding not only for test execution, but also for other testing activities such as test planning, testing discussion and test reporting.

Coding in action

The following is an example of using level 1 coding for user stories.

User Story

As a user

I want to login in securely

So that my private information is kept private

A tester using coding uses the following codes for this user story. (Each code is separated by the | symbol)

Codes

Security | Login | Operations |Function

The tester then looks at more user stories and codes:

User Story

As a user

I want to record a currently playing live show

So that I can watch the show at a later time

Codes

Recording | Live TV | Device Remote | Operations | Data |Time | Platform | Functions

User Story

As a user

I want to playback a currently recorded show

So that I can watch the recorded show now

Codes

Recording | Playback | Device Remote | Operations |Time | Data | Structure | Interface

The examples above make use of the SFDPOT [8] heuristic.  This heuristic is a useful way of utilising social science coding to identify testing coverage gaps.   When coding it is acceptable to assign multiple codes to each piece of information or evidence gathered.

1. The process of coding can be defined in the following steps: (taken from [9])
2. Decide which types of coding are most relevant
3. Start coding
4. Create a start list of codes
5. Generate categories (pattern codes)
6. Test these categories against new data (start with contrasting data early on!)
7. Write about categories/pattern codes in a memo to explain their significance

The following is an example of a tester using these steps:

The testers’ initial testing effort showed that a particular API appeared to work correctly with the customer data set used.  The tester coded this as ‘API interface: customer data set ingests’.  The tester formed a theory that the API had been implemented and appeared to work for that data set. The tester then tested their theory using a different customer data set and the behaviour was inconsistent, this led to a change in the testers’ theory based upon the evidence gathered. This is level 2 and 3 on the coding pyramid diagram.

Memos and questioning

Testers when carrying out coding can use ethnographic research methods [10] by asking themselves the following questions.

• What is the software doing?
• What is the software trying to accomplish?
• How does the software accomplish this?
• Does the user understand what is being accomplished?
• What assumptions does the software make about the user?
• What surprises you about how the software is behaving?
• What do I see going on here? (To track your assumptions)
• What did I learn from the notes I have taken?
• What intrigued me? (To track your positionality [11] – where do you stand, what biases could be at play?)
• What disturbed me? (To track your tensions, beliefs, attitude)

These questions have been adapted based upon work by (Sunstein and Chiseri-Strater, 2007: 106) Field Working: Reading and Writing Research. [12]

Categorisation, level 4 on the coding pyramid, is the concept of tying together a number of observations and seeing if they have any common characteristics. These characteristics or connections in social science coding terms are called memoing.

Glaser defines memoing as:

(A memo is) the theorising write-up of ideas about codes and their relationships as they strike the analyst while coding… it can be a sentence, a paragraph or a few pages… it exhausts the analyst’s momentary ideation based on data with perhaps a little conceptual elaboration (Glaser, 1978: 83) [13]

The following gives a testing example of this:

After testing a User Interface, the tester defines the following codes from the gathered testing evidence:

UI inconsistent | error messages unclear | undermined feature behavior

The tester memos these codes as ‘unwarranted user behaviour.’ Ideally though, the tester should write more detailed memos than the example given.  The purpose of a memo is to write down your thoughts and reflections on the information you have gathered.

The more extensive the testers’ memos are the more they can form concrete theories.  This aspect of coding can prove to be valuable for testers, since the completed memos can be used as evidence of what the system they are testing is really doing.

Coding can be used when analysing the evidence gathered during testing; the tester can group together and code similar behaviours and observations to see if there are any patterns.  These patterns can be used by the tester to guide their testing effort and provide focus for areas that they may find useful to explore.

Ian Day in his book Qualitative Data Analysis – A user friendly guide for social scientists [14] makes the following statement about coding.

The term ‘coding’ has a rather mechanical overtone quite at odds with the conceptual tasks involved in categorising data. This arises from the association of coding with a consistent and complete set of rules governing the assignment of codes to data, thereby eliminating error and of course allowing recovery of the original data simply by reversing the process (i.e. decoding). Qualitative analysis, in contrast, requires the analyst to create or adapt concepts relevant to the data rather than to apply a set of pre-established rules.

Since coding is a conceptual approach, having pre-defined rules or processes before the tester has the evidence is not something that fits easily in to a qualitative research approach.   Codes are relevant to the actual evidence that has been gathered rather than based upon assumptions of what that evidence may be.

Summary

The use of qualitative coding in testing provides many benefits that testers can utilise across all testing activities, some of these benefits include:

• Pattern Recognition
• Theory Forming
• Critical Thinking
• System Thinking
• Testing Gap Analysis
• Meaningful reporting
• Evidence gathering

Learning about social science coding and applying it in practice can be a powerful tool in a testers testing skills portfolio.  Those who promote that testers should learn to code are correct; nevertheless they may want to explore alternative coding approaches.

For those who wish to learn more about the connection between social science and software testing I recommend the following articles and books as useful starting points:

• A complete beginner guide to qualitative analysis – Taylor and Graham R. Gibbs University of Huddersfield (web site)
• Analysis of Qualitative Data for Beginners – Demola Akinyoade – Department of Peace and Conflict Studies Afe Babalola University, 30th June 2005 (Web PDF)
• Software Testing as a Social Science – Cem Kaner – STEP 2008 (Slides)
• What Testers can learn from Social Sciences – Huib Schoots – TestBash 2.0 – 2013 (Video)
• What Testers can Learn from Social Sciences – Part 1 – Huib Schoots – 2013 (Series of Web Articles)
• Looking to Social Science for help with Metrics – Justin Rohrman – CAST 2014 (Video)

With special thanks and appreciation for their help and patience in making this a much more complete article than it was to begin with:

• Michael Bolton: http://www.developsense.com/ Twitter: @michaelbolton
• Huib Schoots: http://www.huibschoots.nl/wordpress/ Twiiter: @huibschoots
• Duncan Nisbit: http://www.duncannisbet.co.uk/ Twitter: @DuncNisbet

References

1. The Coding Manual for Qualitative Researchers – Saldaña, 2013 –http://www.amazon.com/The-Coding-Manual-Qualitative-Researchers/dp/1446247376
2. What is Anthropology – American Anthropological Association (website – Last accessed (Aug 2014) – http://www.aaanet.org/about/whatisanthropology.cfm
3. Brian A. Hoey. “A Simple Introduction to the Practice of Ethnography and Guide to Ethnographic Fieldnotes” Marshall University Digital Scholar (2014): 1-10. Available at:http://works.bepress.com/brian_hoey/12
4. Changing Minds – Ethnographic coding –http://changingminds.org/explanations/research/analysis/ethnographic_coding.htm
5. Qualitative Analysis for Social Scientists, 1987, p. 27 – Anselm L. Strauss –http://www.amazon.com/Qualitative-Analysis-Social-Scientists-Strauss/dp/0521338069
6. What is Exploratory Testing – James Bach –http://www.satisfice.com/articles/what_is_et.shtml
7. Techniques and Tips for Qualitative Researchers – Chris Hann –http://qrtips.com/faq/FAQ–code%20terms.htm
8. How Models Change – Michael Bolton –http://www.developsense.com/blog/2014/07/how-models-change/
9. 8 Qualitative codes and coding (2014) – Heather Fordhttp://www.slideshare.net/hfordsa/qualitative-codes-and-coding
10. Are testers’ ethnographic researchers? Stevenson (2011) –http://steveo1967.blogspot.com/2011/01/are-testers-ethnographic-researchers.html
11. What is positionality in practitioner research? – Dissertation Scholar (website Last accessed August 2014) http://dissertationscholar.blogspot.mx/2013/04/what-is-positionality-in-practitioner.html
12. Field working Reading and writing research – Sunstein and Chiseri-Strater (2007: 106) – http://www.amazon.com/FieldWorking-Reading-Writing-Research-Edition/dp/0312622759
13. Theoretical Sensitivity: Advances in the Methodology of Grounded Theory – Glaser (1978: 83) http://www.amazon.com/Theoretical-Sensitivity-Advances-Methodology-Grounded/dp/1884156010
14. Qualitative Data Analysis: A User Friendly Guide for Social Scientists – Ian Day (1993)http://www.amazon.com/Qualitative-Data-Analysis-Friendly-Scientists/dp/041505852
    X

Tesbash 2015 - Workshop Day

I was fortunate to be invited as a speaker at Testbash this year and had a wonderful yet tiring time.  There were lots of community based activities alongside the usual presentations.  The following is based upon some of my notes taken during the event and from my ever fallible memory.

This year Testbash made a change and decided to have a day of workshops however before that there was a pre-Testbash workshop meetup.  Walking in there were many familiar faces and some new ones, it felt like meeting up with old friends.  At these kinds of events I try to make an effort to sit with people I have never spoken to or met before.  There is nothing worse than coming to a meetup for the first time and not knowing anyone or being too shy to talk to others.  Sometimes it can be intimidating for those who are new and seeing everyone engaged in conversation as if they have known each other for years.  

One great aspect of many testers is their ability to make friends and be quite social.  I sat with one group of people who had traveled from Glasgow which consisted of Paul, Christine and sorry I have forgotten the other ladies name.  The conversation centered on testing in financial organizations and trying to implement what they had learnt on the RST -  course they had attended that day with Michael Bolton.  We had an interesting discussion about starting small and expanding out to others and implementing by showing success.   I talked about my experiences in doing this and we agreed that to implement RST learning in big organizations can be daunting but possible.

After this chat I brought out a few games I by chance had in my bag.  We settled on playing Loonacy which is a simple matching game with no turns.  This is a fun game and Christine was really good, Paul not so good.   Then it was time to get my beauty sleep for the big day of workshops.

Building an Itinerary for Exploratory Testing - Karen Johnson

The first workshop of the day was by Karen Johnson called ‘Building an Itinerary for Exploratory Testing’.  This workshop was packed with content and to really do it justice would require a lot more than two hours.  I have attended previous workshops by Karen and always feel relaxed, her gentle style of teaching is amazing and she makes everyone feel inclusive.  This as a presenter is a difficult task to achieve and Karen is one of the best I have experienced.   In this workshop it was no different.  The first part of the workshop involved learning about three different techniques that can help you build up an itinerary for exploratory testing.   Karen started by explaining that sometimes we get stuck coming up with ideas for exploratory testing. Karen stated there are tools available that can help you generate exploratory testing ideas, such as personas, heuristics and tours. 

  

Personas:

The first tool/technique that Karen introduced was personas.  This is where you place yourself into a certain type of person or user.  Rob Lambert has a great article on using personas in testing.  Personas are a useful way to place yourself in to the mindset of somebody else and can provide you with different perspectives on how to test the product.   One key aspect of personas is that they should not just be a one line description; it should be a story about the person with some background information. 

A couple of tools mentioned during this part of the workshop for creating personas included the following:

• Fake Name Generator - http://www.fakenamegenerator.com/
• Random User Generator - https://randomuser.me/

Heuristics

Next up from Karen was Heuristics, mainly ones using mnemonics to help trigger test ideas. I have in the past found many of these useful when testing.  Karen mentioned she has a set of cards that contain a set of these heuristics and mnemonics.

These cards can be found on her website - http://karennicolejohnson.com/2012/07/testing-mnemonics-as-a-card-deck-v2/.  I created an expanded set based upon the idea form Karen - http://steveo1967.blogspot.com/2013/06/test-ideas-cue-cards.html

Another resource that Karen mentioned was the use of The Quality Tree Cheat Sheet -  by Elisabeth Hendrickson, James Lyndsay , and Dale Emery.

People found this one very interesting and useful

Others mentioned included:

• FEW HICCUPPS – Michael Bolton
• Testing Heuristics – James Bach

During this section Karne mentioned Test charters and recommended the book by Elisabeth Hendrickson called Explore it - A review of the book can be found here: - http://steveo1967.blogspot.com/2013/11/book-review-explore-it-by-elizabeth.html.

In this book Elisabeth has a template for exploratory testing charters:

Explore (target)

With (resources)

To discover (information)·

Where:

Target: Where are you exploring

Resources: What resources will you bring with you

Information: What kind of information are you hoping to find?

Tours

The final tool/technique that Karen introduced was testing tours.  This is where you are given a scenario and then use this to guide your testing.  First up was Michael Kellys FCC CUTS VIDS tour - http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html

• Feature tour: Move through the application and get familiar with all the controls and features you come across.
• Complexity tour: Find the five most complex things about the application.
• Claims tour: Find all the information in the product that tells you what the product does.
• Configuration tour: Attempt to find all the ways you can change settings in the product in a way that the application retains those settings.
• User tour: Imagine five users for the product and the information they would want from the product or the major features they would be interested in.
• Testability tour: Find all the features you can use as testability features and/or identify tools you have available that you can use to help in your testing.
• Scenario tour: Imagine five realistic scenarios for how the users identified in the user tour would use this product.
• Variability tour: Look for things you can change in the application - and then you try to change them.
• Interopeability tour: What does this application interact with?
• Data tour: Identify the major data elements of the application.
• Structure tour: Find everything you can about what comprises the physical product (code, interfaces, hardware, files, etc...).

This was followed by James Whittaker exploratory testing tours

• The Guidebook Tour
• Guidebooks for tourists identify the best hotels, the best bargains, and the top attractions, without going into too much detail or overwhelming a tourist with too many options. The analogous artifact for exploratory testing is the user manual, whether it is printed or implemented as online help (in which case, I often call this the F1 tour to denote the shortcut to most help systems). For this tour, we will follow the user manual’s advice just like the wary traveler, by never deviating from its lead.
• The Money Tour
• Every location that covets tourists must have some good reasons for them to come. For Las Vegas, it’s the casinos and the strip, and for Egypt it’s the pyramids. For exploratory testers finding the money features leads directly to the sales force. Sales folk spend a great deal of time giving demos of applications and are a fantastic source of information for the Money tour. To execute the tour, simply run through the demos yourself and look for problems. As the product code is modified for bug fixes and new features, it may be that the demo breaks and you’ve not only found a great bug, but you’ve saved your sales force from some pretty serious embarrassment.
• The Landmark Tour
• As a boy growing up in the fields, meadows, and woods of Kentucky, I learned to use a compass by watching my older brother. The process was simple. Use the compass to locate a landmark (a tree, rock, cliff face, and so forth) in the direction you want to go, make your way to that landmark, and then locate the next landmark, and so on and so forth. As long as the landmarks were all in the same direction, you could get yourself through a patch of dense Kentucky woods.
• The Landmark tour for exploratory testers is similar in that we will choose landmarks and perform the same landmark hopping through the software that we would through a forest. Choose a set of landmarks, decide on an ordering for them, and then explore the application going from landmark to landmark until you’ve visited all of them in your list. Keep track of which landmarks you’ve used and create a landmark coverage map to track your progress.
• The Intellectual Tour
• I was once on a walking tour of London in which the guide was a gentleman in his fifties who claimed at the outset to have lived in London all his life. A fellow tourist happened to be a scholar who was knowledgeable in English history and was constantly asking hard questions of the guide. He didn’t mean to be a jerk, but he was curious, and that combined with his knowledge ended up being a dangerous combination … at least to the guide. When applied to exploratory testing, this tour takes on the approach of asking the software hard questions. How do we make the software work as hard as possible? Which features will stretch it to its limits? What inputs and data will cause it to perform the most processing? Which inputs might fool its error-checking routines? Which inputs and internal data will stress its capability to produce any specific output?
• The FedEx Tour
• FedEx is an icon in the package-delivery world. They pick up packages, move them around their various distribution centers, and send them to their final destination. For this tour, instead of packages moving around the planet through the FedEx system, think of data moving through the software. During this tour, a tester must concentrate on this data. Try to identify inputs that are stored and “follow” them around the software. For example, when an address is entered into a shopping site, where does it gets displayed? What features consume it? If it is used as a billing address, make sure you exercise that feature. If it is used as a shipping address, make sure you use that feature. If can be updated, update it. Does it ever get printed or purged or processed? Try to find every feature that touches the data so that, just as FedEx handles their packages, you are involved in every stage of the data’s life cycle.
• The Garbage Collector’s Tour
• Those who collect curbside garbage often know neighborhoods better than even residents and police because they go street by street, house by house, and become familiar with every bump in the road. However, because they are in a hurry, they don’t stay in one place very long. For software, this is like a methodical spot check. We can decide to spot check the interface where we go screen by screen, dialog by dialog (favoring, like the garbage collector, the shortest route), and not stopping to test in detail, but checking the obvious things (perhaps like the Supermodel tour). We could also use this tour to go feature by feature, module by module, or any other landmark that makes sense for our specific application.
• The Bad-Neighborhood Tour
• Every city worth visiting has bad neighborhoods and areas that a tourist is well advised to avoid. Software also has bad neighborhoods—those sections of the code populated by bugs. Clearly, we do not know in advance which features are likely to represent bad neighborhoods. But as bugs are found and reported, we can connect certain features with bug counts and can track where bugs are occurring on our product. Because bugs tend to congregate, revisiting buggy sections of the product is a tour worth taking. Indeed, once a buggy section of code is identified, it is recommended to take a Garbage Collector’s tour through nearby features to verify that the fixes didn’t introduce any new bugs.
• The Museum Tour
• Museums that display antiquities are a favorite of tourists. Antiquities within a code base deserve the same kind of attention from testers. In this case, software’s antiquities are legacy code. Older code files that undergo revision or that are put into a new environment tend to be failure prone. With the original developers long gone and documentation often poor, legacy code is hard to modify, hard to review, and evades the unit testing net of developers (who usually write such tests only for new code). During this tour, testers should identify older code and executable artifacts and ensure they receive a fair share of testing attention.
• The Back Alley Tour
• In many peoples’ eye, a good tour is one in which you visit popular places. The opposite of these tours would be one in which you visited places no one else was likely to go. In exploratory testing terms, these are the least likely features to be used and the ones that are the least attractive to users. If your organization tracks feature usage, this tour will direct you to test the ones at the bottom of the list. If your organization tracks code coverage, this tour implores you to find ways to test the code yet to be covered.
• The All-Nighter Tour
• Also known as the Clubbing tour, this one is for those folks who stay out late and hit the nightspots. The key here is all night. Exploratory testers on the All-Nighter tour will keep their application running without closing it. They will open files and not close them. Often, they don’t even bother saving them so as to avoid any potential resetting effect that might occur at save time. They connect to remote resources and never disconnect. And while all these resources are in constant use, they may even run tests using other tours to keep the software working and moving data around. If they do this long enough, they may find bugs that other testers will not find because the software is denied that clean reset that occurs when it is restarted.
• The Supermodel Tour
• For this tour, I want you to think superficially. Whatever you do, don’t go beyond skin deep. This tour is not about function or substance; it’s about looks and first impressions. During the Supermodel tour, the focus is not on functionality or real interaction. It’s only on the interface. Take the tour and watch the interface elements. Do they look good? Do they render properly, and is the performance good? As you make changes, does the GUI refresh properly? Does it do so correctly or are there unsightly artifacts left on the screen? If the software is using color in a way to convey some meaning, is this done consistently? Are the GUI panels internally consistent with buttons and controls where you would expect them to be? Does the interface violate any conventions or standards?
• The Couch Potato Tour
• There’s always one person on a group tour who just doesn’t participate. He stands in the back with his arms folded. He’s bored, unenergetic, and makes one wonder exactly why he bothered paying for the tour in the first place. A Coach Potato tour means doing as little actual work as possible. This means accepting all default values (values prepopulated by the application), leaving input fields blank, filling in as little form data as possible, never clicking on an advertisement, paging through screens without clicking any buttons or entering any data, and so forth. If there is any choice to go one way in the application or another, the coach potato always takes the path of least resistance.
• The Obsessive-Compulsive Tour
• OCD testers will enter the same input over and over. They will perform the same action over and over. They will repeat, redo, copy, paste, borrow, and then do all that some more. Mostly, the name of the game is repetition. Order an item on a shopping site and then order it again to check if a multiple purchase discount applies. Enter some data on a screen, then return immediately to enter it again. These are actions developers often don’t program error cases for. They can wreak significant havoc.

Once this was finished Karen then got everyone into groups and asked them to come up with test ideas using these techniques and tools to test a website or a mobile application.

Observing the groups it was interesting to see that some used mind maps to document their ideas, others used lists, and some created test charters using the template by Elisabeth Hendrickson and one group story to create user stories.  Whilst observing it became clear that many of the groups found the cheat sheet a useful tool for coming up with ideas whilst others found the tours a good way to come up with novel ideas to test the application. 

At the end Karen brought all the groups together to do a debrief.  During this part of the workshop many came to the conclusion that each of the techniques that had been introduced provided them with far more test ideas that they could have thought of without the techniques or tools. Some of the discussions focused on which technique people preferred with many having a presence for either tours or heuristics.  It was asked why people did not like the personas as much and the majority stated it was hard to come up with people for the sessions; however in their own contexts it may be easier.

To conclude this was a fantastic introduction to some great exploratory testing techniques given by an experienced and knowledgeable presenter.  The style of training that Karen used in the workshop was a calm and unassuming one which made everyone warm to her and the session.  If you get an opportunity to attend a workshop by Karen I would highly recommend you do so.

Creative and Critical thinking workshop

The next workshop I attended was the Games and Tools to Encourage Creative and Critical Thinking within Testing one.  When I say attended I meant presented.  I cannot say much about the workshop in terms of attending it; however I do hope people enjoyed it.

For more details of some of the content I wrote a series of article (seven) the first of which can be found here: - http://steveo1967.blogspot.com/2013/03/creative-and-critical-thinking-and.html

Some of the tools and techniques used during the workshop:

• Disruptus - http://steveo1967.blogspot.com/2014/01/using-games-to-aid-tester-creativity.html
• SCAMPER - http://www.mindtools.com/pages/article/newCT_02.htm
• FiLCHeRS - http://www.csicop.org/si/show/field_guide_to_critical_thinking/
• Huh, Really, So – Video - http://www.youtube.com/v/8TX6rzz60xQ
• The 5 Whys - http://en.wikipedia.org/wiki/5_Whys
• Rule of 3 – Weinberg - http://s0ftwaretesting.livejournal.com/1430.html
• Thinkpak - http://www.amazon.com/Thinkpak-A-Brainstorming-Card-Deck/dp/1580087728
• Disney method - http://www.idea-sandbox.com/blog/disney-brainstorming-method-dreamer-realist-and-spoiler/
• Technique for producing ideas - http://www.scribd.com/doc/242701702/A-Technique-For-Producing-Ideas-Poster-pdf#scribd
• Story cubes - https://www.storycubes.com/

After this session I needed to relax a little and gather my thoughts so I did not attended the next workshop

Next there was an open session where we played some games.

I introduced the following games:

• Zendo http://www.looneylabs.com/rules/zendo using Ice Dice - http://www.looneylabs.com/games/icedice 
• Loonacy - http://www.looneylabs.com/games/loonacy
• Dobble - http://www.amazon.co.uk/Asmodee-Dobble-Card-Game/dp/B00NELITFK
• SET - http://www.setgame.com/set

If you want to learn more about these games then if you see me at testing events let me know.

In the evening there was another meetup “Pre Testbash Meetup” where over 60 testers got together and chatted.  The best part of Testbash is not the workshops or presentation but the community feeling of these meetups.  I met old friend and found new ones.  Guna (https://twitter.com/alt_lv) was a shining star and someone who is fun to talk about anything not just testing.  We end up talking about my tee-shirt whichwas based upon the work by the artist Jenny Parks - http://www.jennyparks.com/.  In fact Guna tweeted afterwards about the artist, what a great memory! I bumped into many testers I had not seen in a while including Vernon ‘tutu’ Richards (https://twitter.com/TesterfromLeic), Danny Dainton (https://twitter.com/DannyDainton) Dan Billing (https://twitter.com/thetestdoctor) and many others sorry if I have not mentioned you.

One interesting person I chatting with who I feel is a future testing star was Emma Keaveny (https://twitter.com/EmJayKay80).  Her story into testing is a wonderful and emotional journey.  I do hope one day Emma will start a blog post and tell others her inspiring journey.  I am sure it will be an interesting read and contain colorful language. 

I did not expect this post to become so big so it has become two parts, the workshops and the key presentations.  Watch out for the key presentations post soon.