Further than you can throw a survey

19 Sep

by clarence-lee 19.09.12

I was elated when our clients started asking for user research, but upon having the cake, the challenge turned out to be eating it from a distance.

Remote research has always felt lacking to me. There’s a richness of insight that can only be achieved with face-to-face interaction. Unfortunately, users cannot always afford us such luxuries, and that’s certainly the problem in one of our current projects. The users are busy people, peppered across multiple time zones. Here are some notes from my journey towards embracing remote research.

The briefest brief

Interviews are unpredictable by nature and even more so when factors include differences in culture and language. A couple of times, I found myself drowning in awkward silence after asking what I thought was an innocent question. The first time it happened, I just about managed to make an apology and say thank you before the call ended. The second time, I managed to recover and continue the session. There’s not been a third time, and it’s all down to managing expectations.

We always brief the interview subject in the emails leading up to the call. Then, we brief them again with a short intro at the beginning of the call before the questions begin. But we’ve learned that some people neither read nor listen. To be fair, I don’t like reading either. And when my bank calls me up about why I need x, y, and z, I also don’t listen to that. So now I begin a call by introducing myself and asking them what they’ve been told about the interview. This engages sooner in the conversation and gives me a chance to fill in any gaps and gauge their comfort level with the topics that are coming up. It’s worked a treat so far.

Have a backup and an exit strategy

We use Webex to take the subject through visuals on their web browser, but we use it in conjunction with a phone call instead of Webex’s built in VoIP. We also email a PDF version of the visuals to the subject ahead of time. This sounds like a lot to deal with, but we’ve had no complaints, and it makes the session near bulletproof. If the Internet drops out during the session, the user is still on the line and we can ask them to refer to the PDF.

If they haven’t downloaded the PDF, then asking the subject to describe how they currently deal with the problem at hand is a good way of revealing areas to explore, “Could you walk me through how you do …?” “At this point what are you looking for to get to the next step?” This doesn’t always work, but it’s worth a go. People are happy to get time back, but a fruitless interview will still be perceived as a waste of time on your part and theirs. Give them a chance to contribute, you never know what surprising insight you might uncover.

Leave it to the machines

Quantity is where remote research really excels. Card sorting, for example, can be done without a human facilitator. Yes, it’s much more insightful to watch someone do card sorting, but doing it remotely means you can test many more subjects with much less effort. We’ve used Optimal Workshop’s suite of tools in the past and I would recommend having a look for the following reasons:

  • Setup is extremely easy
  • Results are live, and processed into dendograms, similarity matrices, and other popular analysis methods
  • The data and the graphs are downloadable in csv/svg format so you can edit them
  • Clients can view the live results on a password protected page

Some caveats, though:

  • Card sorting does not support a card going into multiple categories
  • The surveys require Flash support (participants on iPads will not be able to contribute!)
  • You cannot conduct two types of testing. For example, start a participant on a Card Sort, then move them onto a Tree Jack activity

There’s a work around for the last problem. Each survey ends with a thank you page that you can edit. Here, you can put add a link to the next survey. So after a card sort, the user will see “Thank you! Please click here to continue to part two of the survey.” Not particularly elegant, but participants do make it to part two, so it works.

Conclusion

Remote research has its own set of pitfalls and benefits. Make time to plan for failures in technology and in understanding between you and your subject. Automation and quantity is where remote research really shines. Don’t be afraid of it, and have fun exploring.

Back
blog comments powered by Disqus