Whilst researching for a recent blog article on science v
manufacturing and testing I came across an interesting article about scientific standards
called the RESPECT code of practice and I made a mental note to come back to
this since I thought it could have some relevance to testing. The article can be found here and a PDF version of the code can be located here:
The purpose of this article is to look at each of the
statements made about what socio-economic researchers should endeavour to and my thoughts on how it may apply to testing.
The first paragraph is the one that drew me to the article in
the first instance.
Researchers have a responsibility to take account of all relevant evidence and present it without omission, misrepresentation or deception.
It is so interesting how this is closely related to the responsibility
of the tester when carrying out testing.
We have a duty to ensure that ethnically and morally we provide a
service that meets these responsibilities.
The bit that stood out within the main body
of text was the following statement
does not predetermine an outcome
I still find that within the field of testing there are still
people writing scripted tests in which they try to predict the outcomes before
they actual experience using the application.
This is not testing, testing is exploring the unknown, asking questions and
seeing if there is a problem.
Now if we look at the last line of the paragraph
Data and information must not knowingly be
fabricated, or manipulated in a way that might lead to distortion
Hmmm? Anyone want to start a discussion on testing metrics? Cem Kaner talks about validity of metrics here
Then the article gets into the reporting of findings.
Integrity requires researchers to strive to ensure that research findings …. truthfully, accurately and comprehensively…have a duty to communicate their results in as clear a manner as possible.
I get tired of seeing time and time again shoddy or poorly documented testing findings/bug reports. In my world exploratory testing is not an excuse for poor reporting of what you did and what you found
The most exciting part of the article was the final paragraph
in which they realise that as human beings we are fallible.
…no researcher can approach a subject entirely without preconceptions
It is therefore also the responsibility of researchers to balance the need for rigour and validity with a reflexive awareness of the impact of their own personal values on the research
It is something within this blog that I talk about a lot the need to
understand that we have our own goals and views which could impact and influence
our testing. We owe it to ourselves to
try and be aware of these sometimes irrational and emotional biases.
The following is my attempt to go through each of the
statements made in the article and provide my own personal view (with bias) or
some external links in which others within the testing community have already discussed.
a) ensure factual accuracy and avoid
misrepresentation, fabrication, suppression or misinterpretation of data
See previous link to article by Cem Kaner on metrics, Also by
Michael Bolton here and here by Kaner and Bond
b) take account of the work of
colleagues, including research that challenges their own results, and
acknowledge fully any debts to previous research as a source of knowledge,
data, concepts and methodology
In other words if you use other peoples articles, ideas etc
give them some credit.
c) critically question authorities and assumptions to make sure that the selection and formulation of research questions, and the conceptualisation or design of research undertakings, do not predetermine an outcome, and do not exclude unwanted findings from the outset
STOP accepting that because it has always been done this way then
that means it is right.
d) ensure the use of appropriate methodologies and
the availability of the appropriate skills and qualifications in the research
team
Interesting one, I do not take this as meaning to get certified, other people may. I take it that we have a responsibility
to ensure that everyone we work with has the relevant skills and if they do not
mentor them and support them to obtain these skills. Encourage self-learning and look at all the available
approaches you can use for testing and select the one most suitable for you.
e) demonstrate an awareness of the limitations of
the research, including the ways in which the characteristics or values of the
researchers may have influenced the research process and outcomes, and report
fully on any methodologies used and results obtained (for instance when
reporting survey results, mentioning the date, the sample size, the number of
non-responses and the probability of error
In other words be aware of both your own limits and project
limits such as time, money or risk. Testing
is an infinite task so when reporting make sure it is clear that your sample of
‘tests’ are very small in comparison of all the possible ‘tests’ you could do.
f) declare any conflict of interest that may arise in the research funding or design, or in the scientific evaluation of proposals or peer review of colleagues’ work
Does this apply to testing?
If you are selling a tool or a certification training scheme then this
should be stated clearly on any material you publish regarding testing.
g) report their qualifications and competences accurately and truthfully to contractors and other interested parties, declare the limitations of their own knowledge and experience when invited to review, referee or evaluate the work of colleagues, and avoid taking on work they are not qualified to carry out
To me if you stop learning about testing and act like one of
the testing dead (see article by Ben Kelly – here) then you are not qualified to carry out testing.
h) ensure methodology and findings are open for discussion and full peer review
Do not hide your testing effort inside a closed system in which
only the privileged few have access.
Make sure all your testing effort is visible to all within your company
(use wikis)
i) ensure that research findings are reported by themselves, the contractor or the funding agency truthfully, accurately, comprehensively and without distortion. In order to avoid misinterpretation of findings and misunderstandings, researchers have a duty to seek the greatest possible clarity of language when imparting research results
In other words make sure that what you have done when testing
is what you report and that you report clearly and without ambiguous facts
j) ensure that research results are disseminated responsibly and in language that is appropriate and accessible to the target groups for whom the research results are relevant
Make sure that all relevant parties have access to your
findings, communicate, talk, discuss. As
stated earlier do not hide your findings publish them for all to see warts and
all.
k) avoid professional behaviour likely to bring the socio-economic research community into disrepute
We all have a duty as testers to be professional in our
behaviour and this means even when we disagree we still need to respect each
other’s view and be able to participate in a debate without making others feel inferior.
l) ensure fair and open recruitment and promotion, equality of opportunity and appropriate working conditions for research assistants whom they manage, including interns/stagiaires and research students
Employers and recruitment agencies STOP using multi-choice certification
schemes as a filter for working in testing.
Holding one of these certificates do not mean that you can test.
m) honour their contractual obligations to funders and employers
This is a given no comment needed on this.
n) declare the source of funding in any communications about the research.
If what you are publishing is in your own self-interest or a
vested interest in which you can receive funds then please be honest and up
front about this. As professionals we
can then make an informed decision about the content.
The context driven testing school has a list of principles
here and it is interesting to compare the two there appears to be some overlap but
maybe we could improve the context driven one by using more of the RESPECT code
of practice. What do others think? A good starting point maybe?
