Since March 2020, our UX team has been working remotely due to the pandemic. We’re expecting to go back on-site in a few weeks, so I wanted to take the opportunity to reflect on what we’ve learned about remote research. It took awhile, but we learned several strategies that allowed us to keep a research practice going.
Recruit from a pool
We built a participant pool that allows us to send out email invitations through Mailchimp. Now over 320 students, faculty, staff, and community members, we’re able to reliably get responses to our requests for participation. See how we set it up in Remote recruitment for UX studies, an article by our intern Rachel Brown from last July. We also just made a public webpage about our participant pool, including some guidelines for using it.
Keep the ask simple and relevant
We’ve sent over 30 recruitment emails since last March, and one of our student colleagues Yashu Vats did an analysis of email data to determine which had the highest response rates. Those that did best:
- Had active subject lines (e.g. “Help us pick the best thank-you items”)
- Were a short time commitment (5 minutes or less)
- Had clear calls to action (e.g. “Vote on your favorite”)
Those that asked the participant to respond to the email to set up a time, or complete an activity like a Padlet or Lookback, didn’t do so well.
Consider unmoderated methods
We tend to prefer moderated methods, and miss Tiny Café terribly since it allowed us to conduct moderated, lightweight research on a weekly basis. But the logistics surrounding scheduling and technology for virtual sessions proved a big barrier for recruitment. Especially when we had no incentives to offer, we struggled to get people to sign up for time slots.
We learned how we can get useful data through unmoderated methods, including first-click tests, preference tests, impression tests, and other well-written surveys. Not always our first choice, but if we hadn’t used these methods we’d have barely heard from our users this past year. And when we kept the responses simple, we could easily get 50 responses within a day.
Make signing up easy
For moderated sessions, the sign-up process can be a barrier. After trying out and failing with SignUp Genius, Zoom registration, and a “Respond to this email to sign up” option, we now use Calendly for almost all moderated research. It’s easy to use for both researchers and participants. You can customize time slots, sync with Outlook, and add screener questions. Calendly also sends meeting request invites, and we have yet to have a no-show.
Provide an incentive
Some compensation or incentive for participants goes a long way. If you want someone to sign up for a moderated session especially, this can be critical. The incentives don’t have to be big, but they should be something. One of our campus partners offered a bag of swag left over from our IT Summit and had a strong response.
After much delay, we were able to secure gift cards for student participants through an incentive program offered by our campus bookstore. Since being able to offer gift cards (specifically $15 for a 30-minute session), we’ve had a huge increase in response rate. We’re working towards other, non-monetary incentives for the fall semester. In a survey, coffee, tea, and items unique and local to Tucson were popular options. In the guidelines we’ve set up for our participant pool, we now require some sort of incentive if you are asking for more than 30 minutes of a participant’s time.
Don’t stop now
We’ll be going back to the office almost fully in the fall semester, but we’ll continue to take advantage of what we’ve learned about remote research. Having this option strengthens our research program, since it allows us to connect with a diverse and distributed population of participants, including remote students and instructors as well as those who might never visit the library in person. That said, I’d be lying if I told you I wasn’t really excited to be back at Tiny Café.
