We can’t read minds, or can we?
When doing a task analysis, we generally analyze what we can see and hear our experts doing. We closely watch what they click, or turn, or touch. We pay attention to information they may enter into a computer or when they use a screwdriver to loosen the screws on a panel. But, we should also be asking, “Why are they doing this?”
That’s a harder question to answer. We may follow a process and the way an expert completes the steps make it appear as if he or she would always do those steps in that order. It appears that you start with step 1 and continue until you reach step X and you are done. But the next time we observe the expert, he follows a slightly different step. Instead of doing step 4 like before, he skips it. Then later he adds a step between step 8 and 9. What’s going on?
The reality is that the expert is making judgement calls during a process to achieve a specific outcome. There are variables we aren’t paying attention to. Those variables tell the expert to do or not do certain things. In order to fully document and analyze a task, we need to understand those variables. We need to see what the expert sees. When she looks at a gauge, what does she see? Does she just see the level it indicates or is she paying attention to how quickly it moves, how steady the needle is, or whether it shows things are rising or falling? What does she do if it increases quickly? What if it decreases slowly? What if it is stable? What if the reading moves up and down? Our task analysis needs to consider all these potential outcomes and document the steps an expert would do for each of them.
Most process documentation tools allow for decision points and to capture the details for each potential outcome. The question here is, “how do we know when there are other options?” We need to be able to dive into our expert’s head and see things through their eyes. The answer here is around what’s called a Cognitive Analysis.
Cognitive Task Analysis (CTA) is a deep dive past the obvious steps we see and explores the thought process an expert uses. There are many ways to accomplish this. Let’s talk about two here, Critical Incident Review and Vocal Thinking.
Critical Incident Review
In the Critical Incident Review, we work with an expert to identify a situation where the task was done either exceptionally well or when something goes wrong. There is often other data points, reports, video, or other accounts we include as a part of the discussion. It’s important to focus on a situation that actually happened and not a hypothetical account of what should happen. So far this doesn’t sound too different but the difference is that we ask the expert to go slowly and methodically and be more explicit on how the situation unfolded. It can help to start with what may seem like mundane details but then as we probe and ask about their thought process as the situation unfolds. We ask questions like,
So what made you decide to do that?
How did you know this was the right action to take?
Was there something you noticed to tell you what was happening?
What did you look at to know that?
Chances are as we ask these questions, the expert is going to pause and have to think. That’s the point! We want the expert to take the time to think through WHY she did something. But, the expert may not consciously know the answer. That’s the interesting part! The expert is going to dive into her memory and try to answer it but it won’t be easy. The reason it’s hard is because of a level of proficiency we achieve called Unconscious Competence. This is a level where we perform tasks without direct conscious thought and it’s usually because of extreme repetition. The expert has done this task so many different times, she figured out what worked and what didn’t and she eventually polished her performance to a point where she succeeds most if not all the time and doesn’t bother with the things that don’t work (for obvious reasons). But now, we are asking her to unpack all that experience. We are asking her why she knows something and the answer is probably some small lesson she learned years ago about how something didn’t work and she tweaked her actions to avoid it. But because she learned it so long ago, she forgot the original lesson. In some ways, asking the experts to think this deeply can actually get frustrating so take your time and be patient. Go slow, ask questions, and let the expert think through the answer.
Vocal Thinking
With Vocal Thinking, you ask the expert to do the task but to talk out loud what he is thinking as he does the task. Depending on the task this may not always be reasonable to do but often it helps uncover what leads to actions. For example, if you are driving a car as an expert, you do the normal things such as insert the key, start the car, strap your seat belt, put the car in gear, look out the window and in the mirrors, and then pull into traffic. But if you vocalize your thinking, you may say,
I have to look at my keys to find the right one for the car. It can go either way but I like it this way so the keychain hangs down correctly.
I turn the key until I feel a little resistance and I’ll hear the engine try to start. I listen to the sounds and when I hear the engine start to fire, I release the key and leave it in the on position. I’ll notice the tachometer increase showing the RPMs of the engine and it will settle down into an idling level.
I used to reach over my shoulder with my right hand to get the seatbelt but now I feel its smoother if I grab it with my left hand, pull it up to my waist, and then reach across my lap and pull it into the buckle. The belt only goes in one way and I’ll feel it slide into the slot and hear the click knowing it is attached securely. I’ll adjust the belt across my chest and my lap so it’s comfortable.
Being in my driveway, I know I need to go backwards, so I’ll adjust the gear shift into the “R” position for Reverse. Sometimes I go too far and I end up in neutral so I go slowly feeling the gear shift from Park to Reverse.
I could go on but you get the idea at this point. Even though these are rather mundane parts of the task of driving a car, they demonstrate small things I’ve learned over time of driving cars. I’ve driven big cars and small cars. Some with automatic transmissions and some with standard transmissions. Some with the shifter in the steering column and some with it between the driver and passenger seats. All of these experiences add up to how I drive today. If someone asked me to vocalize what is going through my head I may say things I haven’t consciously thought of for years and may even surprise me.
I may recommend you record what the expert says and does. It can be difficult to capture everything the expert says as he or she performs the task and sometimes it’s simply HOW she says it and not just WHAT she says. There is a ton of gold here and as you dig to bedrock, you have to tap into these veins of gold to maximize your return on your work.