I had 14 students in a user research class last week. Eleven of them did not pilot their studies before they went full-speed ahead. WHOA?! That made me realize plenty of others might miss the necessity of doing this.
You should road test your discussion guide, tasks, technology, stimuli and everything else you can simulate ahead of time. Get feedback on whether or not the questions are provoking the right kind of responses, fit within the timeframe allotted, are understandable, in the right sequence, are not redundant, and more. Pilot to get more comfortable with the content too.
If it’s an interview, literally interview someone. Think of it like a beta test.
If it’s a survey, share it with a few folks first. Solicit feedback.
Pilot your study with people who closely resemble your actual recruit. If that’s not possible, get as close as possible.
After you pilot, iterate on what could be improved before spending time and money rolling it out.
I was shocked people weren't doing this. I pilot 85% of my studies even though I'm pretty senior at this point and confident in my skills and judgment.
BTW, I don't just pilot live interviews. I pilot unmoderated sessions, surveys, card sorts and almost everything else.
And now you know.
