The Think Aloud (TA) method of discount usability assessment has yielded for me some interesting observations.
None of the subjects of my TA ever participated in a process like this before and only one has ever been involved in any kind of systems development process. I was fully expecting to have to constantly remind them to “keep talking”, have to re-explain the process, or go “Stanley Milgram” on them…reminding them that this is “all part of the experiment, please continue”. To my surprise they all performed like the process can naturally to them. Only once in the process, did one of my subjects asked something like, “is that right?” and only once did I have to remind someone to “keep talking”. TA is apparently, at least for the audience that I worked with, a very natural process.
I will probably make a poor experimenter! At least if I cannot correct for a continuing and persistent issue that I observed in myself. I found it very difficult to stay “outside the process”. I was fine while the actual TA was being done, but I found that after some of the trials I either said something like, “That is a problem we discovered to.” or “Yeah…I hate that as well”. Luckily, it did not happen that much, but more often than not I did find that I had to catch myself before I spewed out my tainted opinions. This is a weakness of mine that this process exposed which now affords me the opportunity to work on.
This process was made much easier thanks to the Dai’s suggestion that we use Jing, a free screen capture software that can record, as a video, the screen movements of the participant as well as their voice. It really allowed for a much more precise and accurate analysis of the process and I wonder if it put the participants more at ease as I did not have to write as much on my pad and stand over their shoulder in a “Milgramic-like” pose.
There are a few caveats I must add to this observation. All of my participants were under the age of 60, so I wonder if the reverse would have been true for older participants who might be a little less tech savvy and might get intimidated by the “tech” and all the “techy” steps I had to go through to set up each “shot”. No one seemed concerned that they were being recorded. (This is true ever with one of my participants who is a friend of mine that is somewhat of a “the government is listening to us everywhere” type of conspiracy enthusiast!”) However, their lack of concern in being recorded might stem from the fact that they were all either friends or work colleagues. They must trust me!