Creative and Critical Thinking and Testing Part 3

The previous articles in this series have looked at what is critical and creative thinking and defining the stages of testing.  This article is the start of looking at each stage in detail and advising which may be the best type of thinking to apply at that stage as described in the diagram from the first article of this series.

Documentation review

Even though we are moving towards a more ‘Agile’ style of software development it does not mean here should be no documentation and for complex projects it can be vital.  There are various documents that can be created and the ones most commonly accessed by testers are the requirements, design specifications and high level design.  The old schoolers may remember this being referred to as ‘Static Testing’, 'Inspections and walkthroughs.

One of the early stages of testing is for testers to review requirement documents from a testability perspective.  When reviewing the document the tester should be asking questions about the statements been made and critical thinking about what has been stated.  This stage of testing for the tester should mainly be a critical thinking exercise with some aspects of creative thinking.

The question is how do we apply critical thinking to documentation review?

One way is by the use of check lists and heuristics (rules of thumb) to prompt our thoughts on the testability of the requirements.  One example of this is shown below:

• Do questions need to be asked of the requirement before a test can be derived? If so, it is incomplete.
• Are there two or more requirements that are in conflict with each other? If yes, they are inconsistent.
• Can the requirement be interpreted in more than one way? If it can it is ambiguous.
•  Does the requirement contain the words AND, OR, NOR, IF-THEN-ELSE? If it does, it is likely to be non-compound.
•  Does the requirement fail to comply with any of the five criteria? If yes, then it is not testable.
• Does the requirement deal with the ‘what’ rather than the ‘how’ (e.g. design)?
For example, the requirement ‘Provide a database’ states how a function should be implemented rather than what it is that is required. It should instead read ‘Provide the capability for traceability between requirements’
• Is the requirement written as an operation rather than a requirement?
For example, ‘It will be stored in a x20 rack’ describes the operation rather than the environment. It should read ‘It shall be designed for storage in a x20 rack’.
• Is the requirement written using the correct terms?
For example, the terms shall, will and should: Requirements use shall Statements use will Goals use should
• Does the requirement contain the words is, was or must? These should be removed.
• Are there any requirements that are missing? (e.g. reliability, quality)

Other sources/check lists that can be used to encourage critical thinking of the document under review include:

Software Quality Characteristics this is useful for spotting missing information from requirements by asking questions such as:

• Diagnostics: is it possible to find out details regarding customer situations?
• Compatibility: does the product comply with common interfaces or official standards?
• Standards Conformance: the product conforms to applicable standards, regulations, laws or ethics.

There are some similarities between the heuristics and models you use when following test execution and when you are reviewing documentation since your mind is trying to flow through how the program with function so some of the model you use for test execution can also apply to reviewing test documentation.

For example you can use the consistency heuristics created by Michael Bolton and James Bach.  HICCUPPS

• History
• Image
• Comparable products
• Claims
• User experience
• Product
• Purpose
• Statues

Another example that can be used comes from the rapid software testing course and the lessons learned in software testing book and the use of Reference, Inference and Conference. A good article about this can be found here

• Reference: Use all documents and ensure they agree
• Inference: What are my assumptions about the requirements?  Are my assumptions correct?
• Conference: speak to the rest of the implementation team about the issues around testing. 

I am not suggesting using all of these methods but to mix and match and choose which one works best for you.  You may choose not to use any of them and create your own (this is the creative thinking element) If you do create your own it would be great to share with others.

So let us try to apply this in practice.

• Incomplete example:
The system shall restrict access’
Should be rewritten as: ‘The system shall control access via usernames and passwords’
• Consistency example:
‘The system shall calculate all distances in miles’ ‘
The system shall calculate all speeds in km per hour'
These should be rewritten as: ‘The system shall calculate all distances in miles’ ‘The system shall calculate all speeds in miles per hour’
• Inaccurate example:
Requirement: ‘All transactions shall be improved by 1 second’
Customer actually requires faster logon times
Requirement: ‘Daily average user logon shall be improved from 5 seconds to 4 seconds
• Ambiguous example:
‘An employee is entitled to 4 weeks holiday a year’.
 Should be redefined as: ‘An employee is entitled to 20 working days holiday per calendar year
• Embedded (compounded) example:
‘The target platform for the server system shall be Windows 2000 and Windows NT for the client system’
Should be separated into 2 requirements:
‘The target platform for the server shall be Windows 2000’
‘The target platform for the client shall be Windows NT4’

Another useful check list that could be used is the one from the Ron Patton book on Software Testing which is talking about reviewing design specifications

A good, well-thought-out product specification, with "all its t's crossed and its i's dotted," has eight important attributes:

• Complete. Is anything missing or forgotten? Is it thorough? Does it include everything necessary to make it stand alone?
• Accurate. Is the proposed solution correct? Does it properly define the goal? Are there any errors?
• Precise, Unambiguous, and Clear. Is the description exact and not vague? Is there a single interpretation? Is it easy to read and understand?
• Consistent. Is the description of the feature written so that it doesn't conflict with itself or other items in the specification?
• Relevant. Is the statement necessary to specify the feature? Is it extra information that should be left out? Is the feature traceable to an original customer need?
• Feasible. Can the feature be implemented with the available personnel, tools, and resources within the specified budget and schedule?
• Code-free. Does the specification stick with defining the product and not the underlying software design, architecture, and code?
• Testable. Can the feature be tested? Is enough information provided that a tester could create tests to verify its operation?

There are many other areas within the review phase in which critical thinking can play an important part which we have not touched on within this article and I suggest people reading this article to go and investigate more where testers  can add value to a development project by thinking critical.  We can influence such areas as code reviews, walkthroughs and inspections.

The next article will look in depth at Test Planning and the style of thinking required for that stage.

Creative and Critical Thinking and Testing Part 2

The last blog post looked at what critical and creative thinking was and which may be most useful during different stages of testing.  This post will start by giving an overview of each stage and then look in depth at each of the stages.

Stages of testing.

Before we start to look at how we can use these types of thinking within testing we need to breakdown the testing life cycle into manageable stages. This is not an expansive list of the many stages but the labels given are to give a high level overall view and simplify the classification for which style of thinking is most suited for each phase.  It not meant to state that these are the only things we need to thing about in stages nor does it mean that there is not any cross over between stages.  They are meant as ‘guide’ lines only and not as best practice, process or otherwise.

Documentation Review (Static Testing)

• Requirements
• High Level Design (HLD)
• Feature documents
• Specifications
• Standards
• Regulations
• Code review
• Walkthroughs
• Inspections
• Oracles

Documentation review does not just mean reviewing requirement documents it could mean reviewing code, database models, country laws and standards.  Requirements are only ONE source of information (Oracle)   Instead of going into great detail about what role testing plays in the documentation review stage there are many resources available on-line that can do a far better job than me.  I highly recommend that people look at these links if they wish to know more about static testing and documentation review.

Static Testing vs Dynamic Testing

Static testing (Wikipedia)

Static Testing and Static Analysis

Cem Kaner - Testing Computer Software book - has an excellent section on document review 

Ron Patton - Software Testing - Chapter 4 - examining the specification

* Disclaimer - I may not agree with all the approaches;  processes and methods being suggested in the above links but they provide information about the diversity of what documentation review is in the context of software testing

Test Planning

• Test Ideas
• Automation – feature files (cucumber)
• Missions/Charters
• Mind mapping

The test planning stage is one which is much maligned with people within the testing community.  There are people making all sorts of statements about the purpose of test planning with some saying to do as little as possible (Agile style) with others saying it should be wrapped in best practice and process and standards. In my own world there is a important need for planning but there is a case that says do not do too much.  I feel that with test planning you should do enough to enable you to start doing some actual testing. Once you start testing you can then update and maintain your test plan based upon what you experience and discover.  I try to align myself to James Bach's approach to test planning which can be found here page 21.

Test Execution

• Test Models
• Exploratory Testing
• Heuristics
• Note taking
• Session based testing

Now we come to the key element of testing, the part where the tester should engage their brain and start to ask questions of the software.  I am talking about manual testing and specifically exploratory testing.  There are many articles on what is exploratory testing and how to do it and rather than repeat information I will provide just one link to the excellent resource by Michael Bolton here.  This is one page I do have bookmarked and come back to time and time again.

Test Analysis

• Bug investigation
• Defect reporting
• Repeatability
• Questioning
• Future Automation
• Debrief
• Future missions/Charters

This is the often forgotten about stage of testing in which the tester takes a little time to reflect on what they tested, did they do the best they could within the constraints they had? Could they have done things differently?  This time could be spent reviewing the session notes and investigating issues they uncovered during the execution phase, that they thought could be bugs.  It could be used to check if a bug was repeatable and then raised in a defect tracking system with full evidence of how to recreate.  It could be looking at what may be useful to automate from what information was discovered.  It could be adding new ideas from the notes for future testing.  It is also the time to debrief and talk to others about what you did.

Test Reporting

• Dashboard
• Wiki – sessions
• Updating plan
• Qualitative
• Quantitative

Test reporting is a highly controversial subject within the testing community with lots of discussion about the use of metrics.  There are some great articles out there on this subject and I am not going to attempt to summaries it here.  If you want to understand more about the metrics debate then can I suggest you start with this blog post of mine.  Then have a look at some of the following articles from Michael Bolton's resources.

• What Counts
• Issues About Metrics About Bugs
• Three Kinds of Measurement and Two Ways to Use Them

To me the best way to do test reporting is the telling of a story using numbers to help support the story. A good way to do this is by the use of dashboards, a great example of this can be found here.

How does this apply to testing?

So now we have our testing stages we can start to see which style of thinking would be most suited for this stage, not forgetting that the other style will and should still be used, but the focus should be on the primary style of thinking for that stage.

The next article in this series will look in depth at documentation review and the style of thinking that may be most suited for this phase

Creative and Critical Thinking and Testing Part 1

I was asked a question recently regarding which style of thinking to use when at different phases within testing, creative or critical and I started to think deeply about this.  I quickly sketched a diagram to explain my thoughts to the person who posed the question.  Then I thought I need to explain my thinking behind the diagram and started to write an explanation for each section of the diagram, this has turned into a rather long research paper and as such I thought I would share my finding on my blog.  This will be a series of blogs since I think there could be too much information to digest in one reading.

I do not know if my thoughts and my concepts are correct but the best way to find out is to publish and await peer review. Thanks to @simtesting for spurring me to get this blog post started. So your comments, opinions and thoughts would be most welcome.

Where to start?  The first thing would be to show the diagram that started it all off.
The diagram can be download here

Some of the terms I use for stages may be different to what you use but as this series of blogs goes along I will try to explain what I mean by them and also what the diagram represents.


This diagram shows the emphasis of thinking for each testing stage.  The larger the area the more of that style of thinking should be used. To summarize when reviewing documentation you should be more focused towards critical thinking with a small amount of creative thinking. During the test planning stage this is reversed with more effort placed upon creative thinking rather than critical. When execution your testing there is a balance between being creative to find more stuff to test and thinking critical about what you experiences and the theories you are forming.  After you execute your testing you need to be critical of what you have observed and see if there are any problems with the testing you carried out and see if the information you have captured holds up to critical analysis.   Finally you need to report on your testing and you need to find creative ways to present your information without overloading the recipient and confusing them with meaningless data.  To do this you should have a strong slant towards thinking creatively but also keeping in mind a critical viewpoint of what does that information really represent and how it may or may not be misinterpreted.   Then repeat the cycle again and again!!

This series of blogs will look at each of these stages and give examples of how we might be able to improve the style of thinking required for that stage.

To start with it may be a good idea to define what critical and creative thinking are so that we can give context to the subject.

Defining Types of thinking:

The first thing that needs to be done is to look at what we mean by critical and creative thinking to ensure we all understand the same meanings.

“Critical thinking is the active, persistent, and careful consideration of beliefs or knowledge in light of evidence and creative thinking is the generation of new ideas. Critical and creative thinking are fundamental to human intellectual progress and artefacts thereof. Depending on context and purpose, critical and creative thinking skills can be interdependent or separately applied.”  
Creative and Critical Thinking Definitions 

Critical Thinking

“Critical thinking is skilled and active interpretation and evaluation of observations and communications, information and argumentation.”
Critical thinking: It's definition and assessment. Point Reyes, CA: Edgepress Fisher, A. and Scriven, M. (1997).

"the intellectually disciplined process of actively and skilfully conceptualizing, applying, analysing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.” 
Defining critical thinking

Or in simple terms it is a way of thinking that can help you decide if a piece of information is always true, sometimes true or partly true or false.  It is a way of thinking which allows you to question the statement(s) being made.

There are main forms and practices of critical thinking.

One I personally use is

• Recognize the argument – (ask is there a problem here?)
• Analyse the argument – (If there is a problem, why is there a problem)
• Evaluate the argument – (Does the problem cause significant impact/risk/danger)
• Reasoning – deductive, inductive (What solutions could I offer – deductive) (Who can help me resolve or disprove that there is a problem – inductive)

This way of thinking has its roots within scientific research methods and is good for proving or disproving assumptions and statements.

To learn more about creative thinking and how to apply it, there is a good practical guide available here.

Creative Thinking

“Creative thinking is the generation of new ideas.”
Creative and Critical Thinking Definitions 

 “Creativity is any act, idea, or product that changes an existing domain or that transforms an existing domain into a new one.”
Creativity: Flow and the psychology of discovery and invention. Csikszentmihalyi, M. (1996).

Many people see creative thinking as mainly artistic and that they are not creative thinkers.  However being creative is more than art and it can apply to all areas of your work.  The difficulty is being able to focus on innovation and treading a path that has not been mapped.  A little bit like what we do when exploring.  (Can you spot the connection yet?) .

I highly recommend that people watch the Ted talk by Ken Robinson on Schools Kill Creativity, and read his book called Out of our minds.

Summary of the styles of thinking

Creative is divergent – different direction, off the accepted path
Critical is convergent – bringing ideas/thoughts together.

We need to be mindful that critical thinking is important but we should not forget about creative thinking, there is a balance that needs to be made. Which has more influential benefit depends on the context and the stage you are at in testing.  The rest of this series of blogs will look closely at each stage and offers some thoughts on which type of thinking may offer the most return.

The next blog article will look at defining the stages before looking closely at each stage in turn.

Update

Updated to include diagram as suggested by Michael Bolton in the comments

What do you think?
Which do they prefer?
The original one or this one?

Diagram can be download from here.

Something Old, Something New

"Be careful. People like to be told what they already know. Remember that. They get uncomfortable when you tell them new things. New things . . . well, new things aren’t what they expect. They like to know that, say; a dog will bite a man. That is what dogs do. They don’t want to know that man bites a dog, because the world is not supposed to happen like that. In short, what people think they want is news, but what they really crave is olds . . . Not news but olds, telling people that what they think they already know is true." 

TERRY PRATCHETT The Truth: a Novel of Discworld

This passage from the Terry Pratchett novel "The Truth" resonates very much with me especially when it comes to world of testing.  I see a connection with how different people in our craft react with each other when it comes to new ideas, thoughts and innovations.  There are those who embrace change and want to know more.  There are then those who like time to think critically and use different forms of reasoning before deciding if it is good or not.  Then there are those who dismiss out of hand anything new, anything that makes them feel uncomfortable or goes against their current beliefs.

My concern and the reason for this blog post is how within testing we can become more creative and innovative.  As one of my previous post stated I think to be creative we need to think about finding problems than trying to solve them.  Continuing on the path of our focus being only to solve problems restricts our creative thinking.  At the same time we need to find ways to convince those who dismiss anything new or unexpected.  To do this a set of guidelines should be introduced to encourage creative thinking rather than discourage:

• Those who easily dismiss new ideas should not be quick to be negative, negative comments and views are one of the easiest ways to destroy creativity.

“The creative impulses of most people can be suffocated by negative criticism, cynical put-downs or dismissive remarks.” 

Ken Robinson – Out of our Minds

• Company leaders need to lead from the top and encourage new ideas and innovation, making the task of thinking (creative and divergent) as important as other everyday tasks. They need to give time to allow this to happen.

“it’s not enough to think differently. We also have to act differently” 

Abraham Lincoln (Taken from  Ken Robinson – Out of our Minds)

• There is a need to encourage people to try, and to see failure and mistakes as learning opportunities. We should stop blaming and encourage risk taking to enhance the opportunities for serendipity moments.

“We don’t teach people how to deal with failure and this is a fundamental oversight.” 

Ken Robinson – Out of our Minds

• We need to have more diversity within teams, encourage people who may have different views to yourself to work with you.  Employ someone who thinks differently to you.  This gives more chances that new ideas will be generated from these differences.

“Such people will provide a wider range of knowledge from which to extract information and build upon ideas.” 

Why diversity is the mother of  creativity - Jeffrey Baumgartner

• Spend more on training the people who work with you or for you.  Take an interest in their learning, encourage, mentor, and support their creative needs.

 “Creative teaching requires moving from a focus on imparting knowledge to knowledge acquisition, providing opportunities for the learner to engage in deep thought and productive action.” 

Susan Keller-Mathers, Encyclopedia of Giftedness, Creativity, and Talent

• It is not enough to come up with new ideas; creativity involves doing something and applying your ideas.

"Innovation is the process of putting new ideas into practice. Innovation is applied creativity." 

Ken Robinson – Out of our Minds

Some may find this a strange post and wonder what it has to do with testing.  I see testing as a very creative process especially when it is unburdened from too much process and stifled by procedures.  Exploratory testing lends itself very easily to the creative process and encourages the tester to think and discover new and exciting ideas.  We need to do more of this style of thinking so we become more engaged with our creative side.  A few people over the past few years have been saying that testing is dead, I would say that the non-thinking uncreative tester is going to die out and become extinct  (The checking robots as some may classify them).  We need to encourage and develop working environments in which  people can connect to their creative side and be allowed the freedom to explore new ideas and not be afraid of making affordable small mistake from which they can learn.

I will leave you with one of my favourite quotes from the American poet Jack Kerouac,  It is OK to be different and to challenge the status quo we need to encourage more crazy ones into testing.

“Here’s to the crazy ones. The misfits. The rebels. The trouble-makers. The round heads in the square holes. The ones who see things differently. They’re not fond of rules, and they have no respect for the status-quo. You can quote them, disagree with them, glorify, or vilify them. But the only thing you can’t do is ignore them. Because they change things. They push the human race forward. And while some may see them as the crazy ones, we see genius. Because the people who are crazy enough to think they can change the world, are the ones who do.” 

Jack Kerouac

Time to slow down.

This short blog post has been inspired by some of my reading over the holiday period including the following

• Future Minds: How the Digital Age is Changing Our Minds, Why This Matters and What We Can Do About It by Richard Watson 
• Future Shock - Alvin Toffler

We appear to be living in a constantly connected world where we are being bombarded with Terabytes of information each and every day and we could be approaching information overload and the dangers this bring.  Since we are receiving so much information our brains are taking the easy route and mainly accepting what it is been fed without questioning and you start believing things you would not normally accept.

 In A Mind of Its Own, Cordelia Fine makes the point that the brain’s default setting is to believe, largely because the brain is lazy and this is the easier, or more economical, position. However, when the brain is especially busy, it takes this to extremes and starts to believe things that it would ordinarily question or distrust.

Richard Watson in Future Minds states the following:

One of the consequences of rapid information transmission is that we increasingly fail to think properly about the validity of incoming or outgoing information; we are too busy and there is too much of it.

There is so much pressure on us to be doing stuff and looking up stuff and being always available no matter where we or what we are doing, there are very few places where you can escape to think.  The traditional places such as the pub (bar)  or the local park have been taken over by constant ringing and people with their faces bent over a tiny screen.

Alvin Toffler talks about Information Overload in his book Future Shock and talks about how we freeze when we get overloaded with information.  He talks about being overstimulated in war situations and how people will just shut down.  This appears to becoming more common with people turning into mobile phone zombies craving for their next information fix and ignoring all possibilities of serendipity moments from looking at the world around them.

We are using our memory less and less since we can now “Google” it, so we have less storage in our own heads to be radically creative and generate ideas.  Life is being run in the fast lane and we are in danger of doing less and less serious thinking.  We are being told we need a decision NOW, so we skim, scan or ignore and then make a (what could be a wrong) choice.

I feel very much spilt on this subject since I am a techno geek, I have a passion for gadgets and anything technological but at the same time I am starting to realise that I have less time to myself to do nothing and gather my thoughts and do some SLOW thinking.

Richard Watson suggests going for a walk or just starting out of the window.  How many people do you see today in the office day dreaming or just starting into space and really thinking?  What would happen if you did this in your office?  He suggests that we have a day in which we plan and do nothing and allow ourselves to be immersed in our own thoughts.

My own view is that we need to step back a little sometimes and slow down to allow our minds to think and to think deeply.  I have a concern that in the future there will not be any deep thinking and people will just be looking at what they see on the surface and believe that to be true.  We need to start carrying or using notebooks to capture our ideas especially when in a deep thinking moment.  We may have many ideas during the day, some great, some good and some bad but we need to start capturing this and help to provide situations which are conducive to the generation of ideas and deep thinking.

Richard gives the following helpful hint:

If it helps, create three physical notepads, files, or boxes marked “no,” “yes,” and “maybe” and place a note of your thinking in the appropriate one. 

This is a method I use for my ideas for a blog article I have currently about 400 ideas marked 1-3 with one being likely and 3 not likely.  I review these about once a month and change the rating of the chances that it will appear as an article.  This is important since my views and thoughts over time will change and something I felt was relevant at a certain time is no longer relevant.

We can also apply this to exploratory testing There is a still a strong contingent of people in the testing world who measure by number of tasks (test cases*) being completed is a useful way to measure progress and know when we are done testing.  However I feel we need to take a step back and slow down a little to allow ourselves sometime to think.

Exploratory testing is a very human thinking activity and it is easy to start measuring progress by number of missions/sessions completed.  Instead we should allow time to thinking deeply about what we are testing and what information are we uncovering since this could lead to moments of serendipity and that eureka moment.  So the next time you see a tester staring into nothing it may be that they are deeply thinking about what they are doing.  TE Lawerence is quoted as saying the following about people who daydream:

“Those who dream by night in the dusty recesses of their minds wake in the day to find that it was vanity: but the dreamers of the day are dangerous men, for they may act on their dreams with open eyes, to make it possible.”

(*This blog is not going to get into a metrics debate here others have done a much better job than myself see Michael Bolton links here)

So to conclude

• We need to get off the information highway sometimes and reduce the information we are receiving.  
• We need to slow down and allocate some time to thinking
• We need to do something different so we get more experiences that can help generate more serendipity ideas. 
• Do some gardening, take a bath, go for a walk in the countryside with no destination, lose yourself in your own mind.
• Take vacations(holidays) and remain unconnected from the office/work
• Do something you have a passion for and enjoy
• Do not be afraid of making mistakes with the ideas your generate this is valuable experiences

“I may not have gone where I intended to go, but I think I have ended up where I intended to be.” Douglas Adams

Problems and Creativity

I recently posted a tweet

When #testing should we be problem finders or problem solvers? Problem finders appears to think more creatively is this aligned to ET?

This started a few discussions with people tweeting that surely it is best as a tester to be both and that they see their role as being a problem finder and a problem solver.  Some took it that being a problem solver is better since you are looked on more positively.

Francesco @TesterSerendip
@steveo1967 @mpkhosla@JariLaakso @tonybruce77 hence better at finding bugs. Plus you are perceived as one with a positive mindset 

Other that they see themselves primary as problem solvers

Tony Bruce @tonybruce77
@steveo1967 I'm a problem solver, I approach the problem differently#testing

I then made the following tweet:

John Stevenson ‏@steveo1967
@mpkhosla @JariLaakso @TesterSerendip being in a problem finding mode may be better for a tester when testing since you may be > creative

My tweets came about after reading some books and articles on the following subjects

• Design thinking
• Divergent thinking
• Creating Opportunities
• Little Bets – Peter Sims 
• The Click Moment by Franz Johansson

This led to me researching the concept that our mind-set within the field of testing appears to mainly be focused on problem solving and that our default state when testing is be in a 'problem solving' mode.  Very little focus or attention seems to have been placed on using our creative skills and entering a mind-set of being a problem finder.  The more I researched into this the more I felt that in some testing situations especially when doing testing practically we are missing opportunities to think more creativity and we are limiting ourselves when we approach testing with the mind-set of trying to solve problems rather than find problems.

One piece of information I related to was from *James Bach or Michael Bolton in how we should approach session based test management.  When in a session we should be looking for problems, creating new tests and noting things of interest which could be bugs.  We should focus on the opportunities and come back to find out if it is a bug/issue (problem solving exercise) after the session as finished.  The principle of this is if you stop to try and solve the problem you could lose track and focus and your creative thinking will become disrupted.  Einstein is quoted as saying the following:

“We can't solve problems by using the same kind of thinking we used when we created them.”

Which IMO matches the thoughts I have on the issue of problem finding and solving.  Yes we need both but they require different thinking.

This is the key point I was trying to make with my statement whilst we are testing in the context of actually being in a session our mind set should be on problem finding.  Having a mind-set focused on problem finding according to some articles has been proven to be more creative.

“Anyone who is technically proficient can solve a problem that is already formulated: but it takes true originality to formulate a problem in the first place (Einstein and Infeld, 1938).”

“…persons who are likely to innovate tend to have personality traits that favour breaking rules and early experiences that make them want to do so. Divergent thinking, problem finding, and all the other factors that psychologists have studied are relevant in this context.”

 http://www.sagepub.com/upm-data/11443_01_Henry_Ch01.pdf

“Csikszentmihalyi and Getzels (1971) found that originality was high related to problem finding and discovery oirientation”

“Getzels believed that creativity investigators should turn their attention to and examine problem finding in addition to problem solving”

 http://books.google.co.uk/books?id=KDok5dKEhh0C&pg=PA15&lpg=PA15#v=onepage&q&f=false, 

“Many creative individuals have pointed out in their work that the formulation of a problem is more important than its solution and that real advances in science and in art tend to come when new questions are asked or old problems are viewed from a new angle . . . yet when measuring thinking processes, psychologists usually rely on problem solution, rather than problem formulation, as an index of creativity. . . They thus fail to deal with one of the most interesting characteristics of the creative process' namely, the ability to define the nature of the problem”

“We believe that this neglect of problem finding is a deficiency that is observable in the discourse on problem solving within technology education.”

http://scholar.lib.vt.edu/ejournals/JITE/v36n1/lewis.html

What I am not saying is that we do not need to be problem solvers; this is part of our role.  I think the subject matter of this article is similar back to a previous article I have written on which mood we should be in to test software  and which gives us more benefit.

In some situations it may be better to be in a problem solving mind set and in others it may be better to be in a problem finding mind set.  Both have their merits.  What I feel I am trying to get across that if you need to create ideas and be innovative you need to have a mind-set that is looking at being a problem finder rather than a problem solver.   However if have a solution to a problem you may want to flip this around.  Then with a problem finding mind-set examine the solution for problems.

To conclude I will repeat what was said earlier by Einstein and Infield

“Anyone who is technically proficient can solve a problem that is already formulated: but it takes true originality to formulate a problem in the first place (Einstein and Infeld, 1938).”

I feel if there is one thing that people take away from this article it is that statement.

Further reading

1. Out of our minds – Ken Robinson
2. Perspectives in Creativity -   By Irving A. Taylor, Jacob W.. Getzels
3. Wisdom: Its Nature, Origins, and Development – Robert Sternberg
4. Perspective: Problem Finding and the Multidisciplinary Mind -  Linda Austin
5. The Creative Vision: A Longitudinal Study of Problem Finding in -  J. W. Getzels, M. Csikszentmihalyi
6. What's Holding You Back?: 8 Critical Choices for Women's Success -  By Linda Austin, M.D

* If anyone can direct me to a reference where this came from and it was not just a conversation I have had then please let me know.

Ethnographic research feedback

Sometime ago I wrote an article about the relationship between ethnographic researchers and testers and how similar they are.  Recently Peter H-L (@Unlicensed2test) on twitter reminded me that I had also presented at the UNICOM  conference on using some aspects of ethnographic research to aid feedback when we are testing and from this I came up with a new mnemonic and a set of testing related social science questions.   I had thought that I had already posted this but it seems I had not.

What follows is taken from the talk I did.

*************

Within the article there was a section that dealt with questions that the researcher should be asking when studying the subject.  I changed this to make it relate to software testing and came up with the following:

• Substantive Contribution: "Does the testing carried out contribute to our understanding of the software?"
• Aesthetic Merit: "Does the software succeed aesthetically?" Is it suitable for the end user?
• Reflexivity: "How did the author come to write this test…Is there adequate self-awareness and self-exposure for the reader to make judgements about the point of view?"
• Impact: "Does this affect me? Emotionally? Intellectually?" Does it move me?
• Expresses a Reality: "Does it seem 'true'—a credible account of a requirement'?"

Lynne Mckee has been updating a list of testing mnemonics on her blog site  so I thought about this and came up with the following mnemonic:

R.A.I.S.E

From this I created a list of questions under each of these heading that can be used to aid feedback when you have been testing, ideally when you are following session based test management.


Use the following template to do a personal review of the testing that you carried out during the day.
Please try not to answer using yes and no, expand on your reasons for either it being yes or no.
This debrief/review is more about your views, opinions and feelings rather than the product you have been testing.
It should only take you 10 minutes to complete this feedback – try not to write essays.

_______________

Reflect

Personal reflection:

• Could you have done things better if so what? (Both from a personal and testing perspective)
• Have you learnt new things about the product under test (That are not documented)?
• Has your view of the product changed for better or for worse? Why has your view changed?

‘Epistemological reflexivity’ (What limits did we hit?)

• Did your defined tests limit the information you could find about the product?  (Did you need to explore new areas that you had not defined)
• Could your tests have been done differently? If yes how?
• Have you run the right tests?
• If you did things different what do you think you would have found out about the product?
• What assumptions have you uncovered to be true/false?
• Did the assumptions you make impede or help your testing?

Aesthetic:

• In your opinion is the product suitable for the end user?
• In your opinion is the product appealing at first look?
• In your opinion is the product confusing?
• In your opinion does the product flow?
• In your opinion are there any ugly areas?
• In your opinion does the product succeed aesthetically? Does it meet the image the customer is trying to portray?

Impact:

(this section is intended to be used to say how you 'feel' about the product, your first impressions, if you answer yes you should provide more details)

• Does this affect you?
• Emotionally?
• Intellectually?
• Does it move you?
• Does it cause you negative/positive feelings?
• Does it frustrate you?
• Does it annoy you?

Substantial:

• Have we covered a substantial amount of the key product areas?
• Has the testing contributed to your understanding of the product?
• Do you think you have a substantial understanding of the system and sub systems?
• Does your knowledge of the system have any substantial gaps?
• Could you easily explain the system to a first time user?

Expression:

• Does the product seem 'true'—a credible account of a requirement'?
• Does the product express what will happen in ‘real’ world?
• Does the reality of the product match the expectations of the product?
• Does the product express unexpected ways of working?

_______________

To make it easier I have create a MS word document with the questions in which you can download from Google docs here.

*************