By Jessica Zeroual, Manager for Research and Design Operations at Service Canada.

  • The problem: The first attempts to recruit users for usability testing without vendor support.

  • Why it matters: We wanted to build in-house capacity to recruit and compensate users to further build trust between government and citizens.

  • The solution: We leveraged social media, notably LinkedIn, Twitter and Facebook, to recruit participants for our research.

WE DID IT! We used social media to reach out to our users for digital product usability testing sessions. To some, this might not be something big to celebrate, but if you work in public service I am sure you are thinking about the infamous red tape around this approach. Luckily, we had set in place the important infrastructure pieces to allow our staff to plan and conduct research activities leveraging a tool that reaches the masses.

This article is a follow-up to my first of many around the topic of Research and Design Operations in a public service context. I will focus on a business case to provide you with an example of how our theory met our practice in a real-world context.

From the comments about our 32-character email address to being told that we are “everything that is wrong with government”, it was hard to keep the noise out and not be discouraged.

Over the past two years, we have worked alongside external contractors to conduct research and design activities. We had the benefit of outsourcing our recruitment and compensation to these third-party vendors. Over the summer of 2022, we were gearing up to set up for a new vendor to work with, but this time, we approached it differently. Our organisational restructure allowed us to re-allocate where our efforts would be as a staff. After many discussions, leadership and management agreed that our staff would be capable of setting up their own research activities with the support of Research and Design Operations. As with anything that is new in a team, we hit a few bumps on the road but overall hit a milestone. We reached our users and conducted usability testing.

Here are the three big lessons we learned:

With a lot of YAYs came a few woes

Our team had to iron out our process for our product team staff to follow so they could seamlessly launch their social media posts. We decided a checklist and presentation were the best approach. We opted to use a generic inbox to allow for a uniform way of connecting with potential participants. We even had a compensation checklist. Since we had opted into paying users for their time, we had two new processes to undertake with two product teams. Once our social media posts were drafted and translated into both official languages for the government of Canada, we were ready to go live on Facebook, Twitter and LinkedIn.

That’s when the instant feedback started. From the comments about our 32-character email address to being told that we are “everything that is wrong with government”, it was hard to keep the noise out and not be discouraged. Luckily, we had incredibly supportive leadership teams. We took a step back and looked at the big picture, and we still managed to get 300 emails within four days. People were re-sharing our posts across platforms. We had geographical coverage and our user pool needs in a short amount of time. The feedback we got on social media was taken as constructive information for us to use for future research activities. We found a shorter email account to use moving forward and updated our checklist to include better practices for accessibility, official languages, and ease of process. And for those comments that stung a bit more than others, we could pat ourselves on the back because we didn’t know many other teams that approached citizens in this way. We had other government colleagues reaching out to hear our lessons learned and wanting to build off what we did. That is a big win in my eyes. Go, team!

Noting the gaps

Now, I know what you are thinking. Wow, there is no way they were able to reach certain populations. You are right. This approach is one of many we have planned to engage citizens. We started with this approach because our product teams needed to get their digital tools out the door and in front of our users. There are pros and cons to every type of engagement approach. The benefits to this approach were that it was fast, our cost was only for the compensation of users, and social media is used by the masses therefore we would have quite a few reaching out to us to participate in our usability testing. The cons were that we might not reach hard-to-reach populations and staff would be leveraging their personal accounts to promote this type of research, narrowing the reach we might have beyond their own networks. Nevertheless, we were able to access a diverse sample of people participating in the usability testing which in turn helped the product team iterate from their current version to the next in their development.

Creating feedback loops from our colleagues

A few weeks ago, we facilitated a retrospect activity with product teams that leveraged social media for their recruitment. We gained insights on ways all parties could improve the process moving forward. More importantly, we all proved to ourselves that we were capable of recruiting and compensating users, leveraging internal mechanisms. That was inspiring and gave us momentum to explore other internal avenues such as leveraging administrative data to contact our users, conducting intercepts and more. This experience bonded our Research and Design Operations team with product teams. We were able to build more trust between each other, which has been helpful in our discussions and planning.

As we build the tools and resources for our product teams to plan research activities, I think one of our best attributes as a team is how willing we are to be open to feedback, be first to try (and fail) and know that in the end, we are building the momentum we need to go from piloting to scaling these infrastructure pieces for our team.

đź‘‹ You can create a post like this one! Share your thoughts with a community of public servants. Learn more

(Image Credit: Unsplash)

Make sure to share your own thoughts with the author by leaving a comment below