Training in India

I recently ran an exploratory testing workshop in India and I thought I would blog about this experience.

There are many differing views about Indian software development teams, some which are unfounded and some that are characteristics of the working style of Indian teams.

From my experience of working with many different teams around the world the statement above can really be applied to any team no matter where they are in the world how people interact and their style of working is dependent on their culture and way of working.

The workshop I was running had a lot of interaction and required engagement from those attending otherwise it is hard to gauge if the audience understands what you are trying to deliver. I was worried due to what I had been informed about the culture within India that there would be little if any engagement and everyone would agree with what I was saying, even if what I was saying was wrong. (I like to set little traps in my presentations and say things which anyone in testing will know is stupid and start a debate.)

I remembered an article that Jon Bach had written about his viewpoint on working with Indian testers and how he ended up making an apology. (http://jonbox.wordpress.com/2009/12/17/to-india-an-apology/). This article was KEY in how I ended up presenting the workshop to the teams in India.

So taking on board the lessons Jon had learnt I started to change my presentation a little to become more personal more about who I was rather than what I was trying to deliver.

I changed the start of the presentation and included a lot of personal information about myself including photos which I had of my family. When I started to run the workshop I explained that this was just an approach, a possible way of working I was not going to say to anyone attending that this is how you MUST do things and if you disagree with anything I am saying then please let me know. I then spent the next 20-30 minutes explaining about myself and my family. I think that this part was the key element – the need to reach out to the India team on a personal level, family in India culture is very important (Joint Family). I talked about our daughter and granddaughter coming to live with us when her husband was away for six months with the army and lots more. I then asked people attending to talk about themselves and their families. Suddenly the atmosphere in the room changed it become more relaxed and people appeared more receptive.

Can this one little change make such a difference?

So I begin delivering the workshop and found the engagement and interaction of those attending to be amongst the best I have come across whilst I have been delivering this workshop. There was passion, interaction, thoughtful questions and in some cases surprising answers. In my opinion it was one the best workshops I have ever been involved with.

Was I just lucky?

If I had not changed the start of the presentation would I have still ended up with the same reaction and interaction?

I cannot really say since I have no comparison – this was a one chance to deliver to a team in India, I was on a tight schedule, so it was important to get it right.

I really must say a big thank you to Jon Bach, since without reading about his experience I think I would have blindly gone and presented and not got the response I required nor would anyone have really learnt anything.

So the tip for anyone working or dealing with teams from India is to make sure you can engage with them on a personal level , open up and let people know who you really are.

The story of organizing a charity event.

*********

I think I may have been a little remiss over the past couple of month by not updating my blog as much as I should be. I see posts by other great bloggers appearing every week or two but mine appear about once a month. I thought I would take time away from testing issues and blog the reason why I have not been as actively involved in the testing community as I would like to have been. This is a subject close to my heart and some may read and feel it is a little self indulgent however the cause IMO is more than worthwhile.

On Saturday 5th of June 2010 my wife and I organized in conjunction with the Amateur Poker Players League Europe (APPLE)

http://www.appleholdem.com/poker/

a poker tournament at the Prince of Wales Pub, Bishopstoke, Eastleigh to raise money for the Help for Heroes Charity.

http://www.helpforheroes.org.uk/

Apart from running the main poker tournament we had a pool tournament, a raffle and various other fun events. The support of local business was outstanding and overwhelming they could not do enough to help and given the current economic climate it was extremely humbling It was a different story with the large companies who I shall not name here who were not interested at all so when you think you need to pop out to get a pint of milk or buy something try to think of your local community businesses first rather than the big uninterested corporations.

The reasoning behind this is that our son-in-law (Lance Corporal Matthew Wellington) who is in the Royal Engineers returned from his tour of Afghanistan. His role with the Royal Engineers is with the EOD (Explosive Ordnance Disposal,) which as you can imagine is a highly dangerous and stressful job. He has a daughter who is now 2 years old and unfortunately has only seen her daddy for about 1 year of her life since this is Matthews’s second six month tour of Afghanistan within two years.

He has done his duty whilst the family at home, apart from the natural worry, felt helpless, so organized this day to help provide something back to those who are serving and the unfortunate ones who return injured. During his current tour he had to go through the trauma of losing some colleagues and a few who came back suffering from horrific injuries.

So you can imagine my wife Tracy and I had lots to organize and do, which took our minds away from the worry of our son-in-law whilst he was on tour, dreading watching the news and of hearing another member of the armed forces had been injured or killed. It has been a very stressful time and to be able to do something good has helped a great deal. At the end of the day the final amount we raised for this cause was over £1500.00 not bad for a single day event.

Part of this blog is to raise awareness of Help for Heroes and all that they do. They are not politically motivated and are doing a wonderful job and ensuring members of the UK armed forces are rehabilitated in an environment suitable for such heroes. SO if nothing else after reading this blog please visit the Help for Heroes website and maybe just maybe make a small donation.

http://www.helpforheroes.org.uk

Bangalore Testers meet-up (15-May-2010)

I thought I would write a brief blog about a meet up of testers whilst I was in Bangalore, India.

It started with me sending a tweet out asking if any testers in Bangalore were interested in meeting up at the weekend. After a short while Santhosh Tuppard (@santhoshst ) responded by putting an invite on his blog site (http://tuppad.com/blog/2010/05/13/bangalore-testers-meetup-–-may-15th-2010/) then all of a sudden there was a flurry of activity on twitter as people asked about timing and where and when.

So on the Saturday I made my way down to the forum and met up with seven testers from Bangalore.

Pradeep Soundararajan (@testedtested - http://testertested.blogspot.com/)

Ajay Balamurugadas – (@ajay184f - http://www.enjoytesting.blogspot.com/)

Dhananjay Kumar (@dhantester)

Santhosh Tuppad (@santhosht - http://tuppad.com/blog/)

@nitinpurswani

+ two others – if you can remember who they add please add their details as a comment.

We had an agenda or some points to talk about:

Should we record our emotions when testing?

Exploratory Testing – good and bad points – how can we solve the bad parts?

Measuring Testing quality – How?

If we kept to these themes was a different matter.

After some brief introductions everyone was keen to know about me. After explaining I had been in IT for over 24 years which was older than some attending!!! Pradeep pointed out I did not look that old!!!!

We then started to discuss some testing issues and the following is what I remember of the conversations that took place. I am sure some of the others who attended will correct any details I have got wrong :o)

One testing issue we looked at is when teams are being managed remotely and managers at not in the same location as the testing team. Pradeep says this becomes a problem because managers want a breakdown of the testing activity all the time and the testing team spend a great deal of time answering questions from managers about what they are testing.

My thoughts on this was that anything that is actually stopping a tester from testing should be recorded and reported so at the end of each test phase/cycle when reporting back to management if a lot of time is being spent reporting test activities to managers rather than testing then this can be clearly seen and a ‘good’ manager will try to rectify it. One idea suggested was that the managers should become more hands on and take part in the testing activities. The exploratory testing approach makes this very easy to do.

There was a lot of interest in reporting what is stopping testing rather than any thing else – this lead on to a discussion of measuring testing and testing quality. This is always a difficult question and I am not sure we fully answered it in our discussions. I mentioned that when testing should we record our feelings and emotions about what we are testing. I suggested that we use a happy, sad face system so when we feed sad about testing we have a sad face when we feel angry an angry face and so on, It would then be very easy to see a trend for a area under test. If there were lots of angry faces then someone will spot and trend and start asking questions of the testers about why they feel this way. Pradeep pointed out that developers may try to influence the testers to always make happy faces. I stated that testers should be still independent of developers and that we should support and help developers but we would not tell them how to write an algorithm so why should they tell us how to test?

On the subject of measuring testing the discussion revolved around looking for trends rather than looking at numbers. The numbers could still be wrong but that is where talking helps to understand what the numbers and trends are saying.

Another point was made about using Tech support people as a resource for testers. I posted this on twitter as a question and got the following responses: (may not be in time/date order)

testingqa: @steveo1967 Tech support people hold some of the skills, but I wouldn't immediately say it implies they will be a good tester too #testing

testingqa @QualityFrog @michaelbolton My 1st role in IT was Tech Support role,those I worked alongside were slack & passed along cases for me to solve

testingqa @QualityFrog @michaelbolton So my faith in Tech Support staff isn't great ;) But I do agree that a good tech support person can hold an... appropriate background useful for testing, but even then Tech Support doesn't require an eye for detail often.

qualityfrog @michaelbolton @testingqa I've seen good ppl move fr tech supt 2 testing & vice-versa. there r similarities in foundation in what makes good

qualityfrog @michaelbolton @testingqa And, sadly, some in tech support are about as astute at faking support as some testers are in faking testing.

michaelbolton @testingqa Not automatic, but the skills and experience of support greatly overlap those for #testing. Great foundation, I'd argue.

michaelbolton @QualityFrog @testingqa I took *good* tech support person as read. Your pessimistic interpretation is also valid, alas.

So even here there are two sides to the argument but it appears that ‘good’ tech support could be valuable resource to testing. Maybe I am a little biased on this since my background is technical support and I believe that if you have a good grounding of how users think and technical knowledge as well (preferably at a coding level) then you have the right attributes to look at becoming a tester.

It was interesting that whilst talking to the group and I was aware I was doing a lot of the talking how much interest was shown in the subjects we were discussing, more importantly how much passion all the people at the meeting had for testing. I could see that in everyone’s faces a passion for learning and understanding. Key attributes of ‘excellent’ testers. I twittered about meeting the future teachers of software testing and I still believe that these people hold the key to the next 10 – 20 years of software testing.

The next subject that came up was one of certification – I explained that on the way to the office the other day I saw a sign that said “Learn software testing in two weeks”, everyone in the group laughed. We talked about the fact that software testing is not like programming in which once you have learnt the foundations of the language you then have the building blocks to create code. Software testing is not computer science but a mixture of many types of science including psychology and philosophy. It is not an exact science and there is plenty of scope to get it very wrong which we have all seen. The question of certification seems to be a very interesting one considering the latest blog posts coming from James Bach and Stewart Reid. (http://www.satisfice.com/blog/archives/457, http://www.eurostarconferences.com/conferences/session-details.aspx?sessionId=209)

I do not understand the need why fellow professionals need to attack each others ideas, it is getting very political. This is just my own personal viewpoint and nothing to do with the meet up. I believe in the need to debate and discuss ideas and opinions but when it appears that someone is personally attacking someone it then appears like a personality problem and distracts from a meaningful debate on the issues – I really want to attend Eurostar now just to hear the debates.

This nicely leads us to the subject of certification that was brought up during the meet up. We discussed certification and all of us agreed that it appeared to be a money making scheme that gave no benefit to experienced testers. The concern of the group was that agencies would now demand that testers have this certification and people will take the exam worried that if they did not they would not get a job in testing. The group felt this was very wrong, to be pressurised in taking an exam because there is no other measurable way to prove you are an ‘excellent’ testers. As a group we came up with some ideas that may be a way forward and needs everyone in the testing community to push forward.

• All testers should have an online presence
• They should be involved in writing blogs.
• Be actively involved in testing discussion (software testing club, twitter)
• Should try to meet fellow testers a couple of times a year at testing meet up – the internet allows this very easily as this meet up has proved

Once this is done when you apply for a job and the agency asks for your certification – point to your active online presence in testing ask them to talk to peers who have met you and who can vouch for you. If we ALL did this then the need to pay organisations to prove you can test goes away. Let us as a testing community certify each other.

So with this last thought I did a little experiment on the people in the meet up involving the calculator experiment as taught by Michael Bolton – if you have experienced this from Michael then you will know what I mean if not then you need to find someone who knows because I am not going to spoil it by explaining on here. Thank you again Michael for giving me a useful way to demonstrate a key point.

So all too quickly the meet up had to end. Did we all learn something? I hope so. I know I did and I had a wonderful time and left feeling encouraged and motivated. I still think I talked far too much and for that I apologise hopefully if there is a next time I will encourage others to speak even more – you have been warned……

The Online Testing Community

I have worked in the field of IT for a long time and seen many changes but none have been as significant as the one that has happened online. The rise in the number of fellow professionals now blogging and tweeting (or should that be twittering) is amazing. The online testing community is increasing more and more as testers take the plunge and start to write and debate about testing online.

We have online testing clubs: http://www.softwaretestingclub.com/

magazines: (http://blog.softwaretestingclub.com/magazine/)

testing knowledge exchange : (http://testing.stackexchange.com/)

Organizations trying to get more of the testing community together online: (http://blogs.stpcollaborative.com/stpcollab/)

I would count myself as a late starter in this revolution only being brave enough to start twittering and blogging late last year. Why do I say brave enough? I am sure I am not the only one who feels that there are so many peers who we read about online and who we, deep down, admire. We may feel that we can never be good enough to write articles about testing or that what we write will be dismissed by the community. In my case I am well aware that my grammar is not the best in the world but all I am doing is writing down what I am thinking and I hope it comes across in a good way. There can never be enough of us online talking about our own opinions and valuable experiences. I had a fear that people would not be interested in what I had to say or worse still would think what I had to say is silly and I would feel rejected and humiliated. Surprising the testing community has not been like that.

I have had some wonderful debates and discussions on testing issues. I have been coached online by Michael Bolton (@michaelbolton) on using Exploratory Testing, in which he gave up his own time for free and is wonderful patience person who really makes me think differently. I have been introduced to such great testing thinkers as Rob Lambert (@Rob_Lambert) who has been a leading character in organizing the online testing community. I have had some great comments on my blog about subjects that I find interesting. I have found it a great outlet for my thoughts and ideas on testing that I once kept to myself, afraid that what I was thinking would not be of interest to anyone. I have also found it to be a wonderful resource for information and ideas about testing and how much people are eager to help.

I would like to say to anyone who is reading this and does not have a presence online to just go for it, start a blog, start a discussion on twitter or join the software testing club and start a debate. I have found it has given me a new lease of professional life, it has made me more aware that whatever problems I come across other are coming across the same problems. It has encouraged me to start writing about my experiences of testing and that I may have some useful information that others want to hear or read about. It has encouraged me to come out of my shell and talk to people about my passion for testing and that cannot be a bad thing……

Where do all the ‘old’ (experienced) software engineers go?

I was having a conversation with a colleague the other day whilst in the kitchen area making a coffee. When they stated the following:

“You never see many engineers over the age of 40 working in a development environment, actually writing code or testing”

I thought this was a interesting statement and it made me think. I have been in the IT business for a long time and yes I am over 40 and I am still actively involved in cutting edge software development projects. However when really thinking about the statement, how many more people did I know or have known who work in software development as developers and testers are still actively involved?

We had a discussion about it and came up with some reasons why.

• They move up further in the company (VP, CEO, etc) and take a less active role
• They switch careers becoming technical architects etc.
• They give up working in IT

One other point was made

“Software engineering is a young person’s career”

Quite a controversial statement!!!!

However could this be true?

Another colleague mentioned a point that as we get older we lose our mental ability or cognitive processing. However this article seems to debunk this: http://www.healthandage.com/html/min/afar/content/other6_1.htm

The article does state that we do lose our attentional ability and processing speed – key elements for software engineers.

What did interest in the article was the following:


“In general, memory tasks that are complex and require manipulating a lot of new information quickly become more difficult with age. Facts, names, and events that are not often accessed may become more difficult to retrieve from memory. However, knowledge that has been accumulated over a lifetime, which is repeatedly accessed and expanded, is generally retained. Well-practiced skills and abilities remain intact. And vocabulary usually continues to increase throughout life.”

So we may become slightly slower mentally as we get older but we retain our well practiced skills and abilities.

One important point made in the article which is related to my discussions on the telling of stories and is a key skill of an excellent tester: “….vocabulary usually continues to increase throughout life”

So my question and the real point of this article is:

“Do companies make a conscious or unconscious decision to remove older software engineers?”

It would be a shame if this is happening since currently I feel like I am in my prime. I am still discovering new and wonderful things about software development each and every day. I still have the same passion for my chosen career as I did when I first started with the added advantage that I have years of experience to fall back on as well.

I would love to hear from other people who, once they look around their respective companies, notice the same trend. Or from anyone who has any more theories on this.

Measuring with Stories

In my experience one of the most difficult tasks I have is to try and change the way we measure testing. When I first started out in the profession of testing there was very little thought given to how we should report the quality of the testing that has been carried out. The most common way was to record the number of test cases and report how many passed and failed.

Then the managers wanted more information such as:

• The number of defects raised
• The number of defects fixed
  
• The severity of the defects.

Does this sound familiar to anyone?

Sadly even today when testing as a profession has started to mature, managers still measure the quality of testing by using these figures and metrics.

I read the following article the other day: Defect Detection By Developers

The article talks about how developers can discover more defects in the same amount of time frame as a tester.

I always have problems when I see articles like this.

How is the statement:

'The number of defects detected by the developer is of the same order as detected by a test engineer in the same time frame'

Quantifiable or Qualitative?

There is no mention of the type of defect and the measure of risk of the project if the defect was not found. What happens if say 90% of the defects found using this method were purely cosmetic? Would this indicate this method is better than using a skilled tester during the same time frame? The skilled tester may find less defects during the same timescale but they may (and normally do) find the difficult to detect defects. Or having the tester and the developer work together during the development phase using continuous build to check as they go?

The approach discussed is one which many companies should already be using

In my experience one of the most difficult tasks I have is to try and change the way we measure testing. When I first started out in the profession of testing there was very little thought given to how we should report the quality of the testing that has been carried out. The most common way was to record the number of test cases and report how many passed and failed.

Then the managers wanted more information such as:

• Code reviews
• Peer reviews
• Documentation reviews (What happens if the project is a prototype project in which no documentation exists? - Maybe BDD could cover this?)
• Unit tests etc.

The article misses vital approaches such as continuous integration? How many defects are trapped during this method?

I think the article has some valid points however I strongly object to the statement that a developer can detect the same quality of defects as a skilled tester can in the same period of time.

I do not like 'measure by defects' to prove quality - quality is proven by the telling of a story to indicate that the product is of the right quality for its purpose.

This leads me nicely into the title of this blog.

Why does management insist on measuring the quality of testing using numbers?

Are managers measured using numbers?

In my role as a test manager I have never, in my experience, been appraised solely using numbers nor as far as I believe has any senior stake holders been measured solely using numbers. When I talk to my superiors about my management goals I try to tell stories and avoid using numbers. For example if I said to my line manager ‘I have met 70% of my targets’ and left it at that. What information would my line manager get? Would that be good or bad? Compare that to some tester reporting to their manager that 70% of their testers have passed. What information can you get from that?

I understand these are extremes but unfortunately these extremes are common when it comes to measuring the quality of testing.

So if I spoke to my line manager and said that over the last six months I have been involved in addressing x, y and z, which are the most important tasks, I still have problems with a and b and I have not had the time to deal with T and S. I have enjoyed the challenge of x y and z and feel I have achieved the best I can in these areas, a and b are ongoing and I am sure that I can complete them soon as long as I get support for b. My estimates where too ambitious for me to even start T and S but these were not too important so I gave them a low priority for completion.

Does this give you a better understanding of what has been happening? Do you notice there is no use of numbers in the above? I feel it gives far more information.

So if we now do the above again and convert it to a story about testing:

For the last six months I have been involved in testing areas x, y and z, which are the most important areas of the product. I still have problems with test areas a and b which appear not to be working as expected and I have raised defects for these issues which are reported below. I have not had the time to test areas T and S. Test areas x y and z are of sufficient quality for release. Currently test areas a and b are not of sufficient quality but once the defects in b are fixed I am confident that the quality will improve. Due to the time taken to address the issues with a and b my estimates where too ambitious for me to even start test areas T and S but since these were not too high priority the product can work without them as long as it is documented.

How does this sound? If you as a manager read this would you have a better understanding of the quality of testing?

Management is normally measured by the telling of a story, so why not measure the quality of testing by the telling of a good story?

I was on twitter the other day I noticed an article by BJ Rollison talking about a similar talking in how we measure the quality of testing. Meaningful Measures

There was an interesting line he came up with:

'At one time I naively believed that there was a core set of metrics that all teams should be collecting all the time that we could put into a ‘dashboard’ and compare across teams. In retrospect that was really a bone-headed notion. Identifying these measures is not easy, and there is no cookie-cutter approach. Each project team needs to decide on their specific goals that may increase customer value or impact business costs. Testers should ask themselves, “why are we measuring this?” “What actions will be taken as a result of these measures?” And, “if there is no actionable objective associated with this measure, then why am I spending time measuring this?'

Some very valid points, testers should be asking why are we measuring this, is it quantifiable? Will it help lay people understand the quality of the product?

When we measure the quality of testing it should be clear, concise and in a language that anyone can understand. Blinding people with numbers does not aid clarity. Trying to measure different teams on different projects with the same metrics (code coverage, defect counts, test case pass/fail) does not indicate one team is better than the other all it does is help management pretend that they understand the quality of testing that took place.

With this article I am not saying we should abandon all numerical metrics to measure the quality of software testing but we need to look more at the story behind the numbers since these can give you far more information on the quality of testing.

Some other useful articles on Software Metrics:
meaningful-metrics by Michael Bolton
Metrics by Kaner and Bond

Does manual testing really lose its value as Companies encouraging more and more automation?

A colleague the other day asked me for my views on the automation/manual debate and asked the following question:

Does manual testing really lose its value as Companies encouraging more and more automation?

I thought this is a very interesting question and decided to blog my response.

I should start by saying I am not split into either camp on the automation vs. manual testing debate. I can see the benefit of both depending on circumstances you are in. If you have an old legacy project in which you are 100% sure nothing will change then automation could be the answer. If you involved in a system in which changes could be made and you want to CHECK that the functionality or the business rule that is in place is still giving the exact same result with no deviation then automation could work.

My thoughts on the question that I was asked (and the title of this article) are completely the opposite. Automation requires no sapience or thinking to be executed, so once the automated checks have been written (which does require sapience) you can run and forget.

The problem is that the world in which we work in is all about changing, adapting and making things better (normally) especially in an agile environment where change is embraced.

I should state that my definition of manual testing is not of following pre scripted tests but of being a test explorer, searching in all the nooks and crannies, trying to discover new and intriguing things about the software. If it is pre scripted then automate it, do not waste good tester intelligence and skill on running a check list, your testers deserve better.

So I would ask the following questions on any company who want to encourage more automation at the expense of manual testing.

• Is it cost effective to write lots of automated checks compared to carrying out manual exploratory testing?
• Which method would in the same time period give the most test coverage?
• Which would be the most easy to adapt to major changes?
• Which would uncover the most problems or issues?

I feel there is some value in using automation at unit level and build level, continuous integration with acceptance checks is a useful tool for the software tester since it lets them have a early look at changes with some confidence that what they have been given will at least have a chance of working. Sure beats the good old days of rejecting x releases in day because of a typo in an install script or a missing dll from the build.

You hopefully can see that I am not against using automation; however there appears to be a view in the testing world that automation can replace manual testing or make its value less. This view does worry me since if the corporate suits think they can get more value and better quality using automation then the message about the art software testing is not being broadcast well enough.

Manual and automation can co-exist very well together; however:

• manual testing can exist without automation
  
• but automation cannot exist without manual testing.

There is an interesting podcast with Jon Bach and Michael Bolton with their viewpoints on the difference between checking and testing available here:

http://www.quardev.com/blog/2010-02-02-1123487836