Mentoring a New Tester - First test challenge

The following is the results from setting Matthew the eBay challenge which was created by Jon Bach for a weekend testing session. The reported produced by Matthew including his thoughts and thinking – anything which is in red italics are my comments.

Once the session had finished I arranged a debrief and the results of that can also be found in this blog article.

The Mission:

Use the eBay website and find the most expensive product, the most unusual and to get just one hit

Most expensive:

My initial thoughts on this is to just leave the product description blank and go for the greatest product price given to me in the cost field.

Most Unusual:

I'm thinking on this one to just enter in the product field, a few random letters and numbers. What is unusual…? I suppose that’s up to me to decide.

Just one Hit:

This has stumped me. How do you get just one hit without knowing what is on offer. Maybe if I knew the exact product name I could enter it, but even then a search engine will look at all the words entered and come up with a few other options for me, defeating my challenge. Trial and error? Or just entering one random word that I think will be a rarity on eBay, like an ‘AAD’ – Mentor comment What is AAD? I think I will try the latter.

Most expensive:

Straight away no field for price, so clicked on advanced. Mentor comment – could this have been planned/scripted before using the product?

Mentor comment – notice the use of ebay.co.uk – would ebay.com give difference results? I did not specify which to use (assumption was made) Good example of ambiguous requirements?

Typed in 10,000,000

Error

So now I am going to also enter into the keyword/item number field the number ‘1’ as a I imagine most item numbers will have a 1 in it. – Mentor comment – why number not letter? The use of wildcards?

Chose the field ‘all categories’ as this was the most likely category for the most expensive as it will search the whole site.

As well as entering the min price of 10.000.000 and leaving the max price because I don’t want to limit myself considering what I am trying to find.

Click search

It looks like I might have found the upper limit for eBay with only three items all coming in at the same price. –Mentor comment – interesting when this task was done for weekend testing there was no such limit found on ebay.com – is this just a co.uk limit? Is it a legal issue?

But are these the most the most expensive.

Have now entered ‘2’ into the keywords field. –Mentor comment – good exploring – testing your theories based upon results.

Same 3 as above were found plus another, reinforcing the fact that eBay may have an upper limit of £12,869,224.17

Have now entered ‘3’ into the keywords field.

Nothing else found except what has already been found.

Now going to change the min price to 13,000,000 and fill the keywords field with ‘2’, as this was the one with the most hits.

Nothing found.

There is no individual most expensive item, only that eBay has - mentor comment – has is strong word – are you sure? Appears to have may be better an upper price limit of £12,869,224.17 on a number of items. - Mentor comment – as per previous comment does this apply to all eBay sites?

Most Unusual:

Entered a few random letters and numbers into the keyword field on the homepage as shown below.

Found mainly stamps.

Unusual hobby, but not unusual enough for me. - Mentor comment – you need to expand on your definition of what is unusual to you.

Typed in unusual and found mainly ornaments. Not there yet.

Another word for unusual… ‘Quirky’, entered that, found mainly clothes. - Mentor comment – good trying to find different meaning – a dictionary/thesaurus could be useful here

Still not there.

Maybe ‘weird, entered that and found pens in the style of syringes, more disturbing than unusual.

So I thought that this may take to long with me just entering random words. So I thought what might be unusual to me.

I went back to the home page and I looked through the drop down menu.

And thought that the industrial field is something I know nothing about, and may provide me with some unusual items. So chose that field and left the keywords field blank.

Before I had a chance to look at these results I saw excavator. Unusual to me, can also be quite funny at times, so I decided to choose that from the related searches bar under the search tool.

Scrolled down the results page and mainly saw digging machinery.

Mentor comment – love the thinking process being described on how you move from one way to find a solution to the next

Until I saw this

This to me is unusual. Never seen it nor am I likely to see it anywhere else funny to boot too.

Mentor comment – Great result I would have also pointed out the obvious about the digger being a pick up only and delivery being free

Just One Hit:

I thought I would try just entering one random word that I think will be a rarity on eBay, like an ‘AAD’.

So I put ‘aad’ in the keywords field and searched all categories

No good. The word ‘aad’ can be placed into any word of the thousands of products out there.

So I am now thinking of narrowing that field so I chose the advanced search button on the home page.

Chose the fields below to try and narrow the sites search options.

Not what I was looking for. I narrowed the search options that much that the site could not find anything, so it decides to ‘help’ me and remove some of my search options to give me a few items for me to look at.

I’m thinking now that you cannot randomly try and narrow the search as the search engine will just try and ‘help’. So maybe if I know an item to be there that I type into the keywords field the exact spelling of that known item. I am going to use one the most expensive items (previously found) as an example. As shown below.

Mentor comment – love this way of thinking – very thoughtful analytic mind

Eureka!

Is this cheating though? – Mentor comment – hmmm NO – you achieved the mission the system allowed you to do this - so it is not cheating just manipulating the system to achieve a result.

All I was asked to do was to get just one hit. I have that one hit. So is this testing the system? The search engine did not try and ‘help’ me on this one. It might know that I am not randomly searching and therefore did not want to help me? In this case then, I have tested the system to get just one hit, so long as you know that an item is there.

Mentor comment – useful information to get one hit you need to know about an item first - good piece of knowledge to be made aware of

Debrief:

During the debrief with Matt I loosely used the PROOF (LA) method

We talked about what had happened during the testing session and Matt replied that it a challenge since he had not done anything like this before in the sense of calling it ‘testing’

He enjoyed the session and felt a sense of achievement in meeting the missions but felt his lack of experience and knowledge of testing techniques hindered him and it was something he felt he needs to do some more learning on.

Matt compared the way he went through the list of missions as similar to army briefs in the sense of some upfront planning of what needs to be done to achieve the goal, he felt his experience of this type of context helped a great deal. During the recording of the session and making his notes he stated that there appeared to be a comparison to recording physic experiments. Where you have a theory to prove/disapprove, you then describe your results and form a conclusion based upon the analysis of the results.

The things that hindered Matt were a lack of clear objectives – something the army instils and some testing knowledge. As part of his homework I asked Matt to research wildcard searches and boundary analysis.

Overall Matt enjoyed the session and is looking forward to the next one.

From my experience it is so easy for those of us who have been in the testing field for awhile to forget how much of the techniques and practises we do by instinct and from our inbuilt repository. Something we should remember when starting with someone new to testing.

Until the next challenge…..