A few weeks ago, the State Department held its first conference dedicated to user experience design, UX Exponential. The conference organizers invited me to speak, and in this two-part series, I'd like to summarize (as best as possible) the presentation I gave, “Foster The People: Building Empathy with Stakeholder Interviews.”
In this post, I'll cover what stakeholder interviews are, why they’re valuable, and how to prepare for them. In the second post, I’ll cover how to actually run the interviews as well as some tips for synthesizing and integrating the results into the team’s shared understanding.
Before I continue: The idea of “explaining” stakeholder interviews is a rather audacious goal. Many authors have written entire books about the skills and perspectives necessary to do this work and apply it within the context of product design. I’ll do my best to summarize this subject, but interested readers are encouraged to peruse the books and articles cited at end of this series.
With that out of the way, let's dive in.
What are stakeholder interviews?
As defined in 18F’s Method Cards, stakeholder and user interviews (stakeholder interviews) are "a wide-spanning set of semi-structured interviews with anyone who has an interest in a project’s success, including users."
Stakeholders come in all shapes and sizes. Indeed, given their interest in the project’s success, even your own teammates count as stakeholders! It’s ultimately up to researchers themselves to determine when and with whom they should chat. A good rule of thumb is to try and talk to the people who will spend the most time using the thing you plan to design — but stakeholder interviews can also be useful for determining what that thing actually is.
How many times have you heard this?
“Oh, if only I had been involved, I would have explained that to you sooner/told you that idea would never work/made everything right in the universe.”
—Kevin Hoffman
Because they can help inform your understanding of the people, behaviors, artifacts, and tools that you might affect by way of your work, stakeholder interviews should happen before a project’s “kickoff meeting.” This ultimately does three things:
- It raises important questions and surfaces any constraints for which you’ll need to account.
- It builds trust by showing that you’re open to receiving feedback.
- It drives alignment (something I’ve also called a “shared understanding”) both inside and outside the team.
Properly executed, stakeholder interviews help you reduce the time it takes to reach a viable solution for the people who will take receipt of your work.
Plan the interviews
Before you can interview stakeholders, though, you’ll need to identify who those people are: who cares or will be impacted by your work? The easiest way to do this is to group (or “segment”) stakeholders and users based on their role in relation to the problem you’re trying to solve.
Practically speaking, this means looking for groups of people who do things in a similar way. As Jon Kolko says in his book Well Designed:
The key is segmentation, but the segment[s] [we’re] looking for [aren’t] based on demographics or those funky marketing psychographics. Instead [we’re] going to sort potential participants based on [our] ability to watch real behavior. In a way [we’re] less interested in the people themselves than in what they do.”
Stakeholder interviews help designers inform the three assumptions guiding their work (the “what,” “how,” and “why”). By talking to real people and observing them as they go about their real lives — using systems, artifacts, etc., that, importantly, may or may not have been designed with them in mind — you’ll learn more about what’s working the way it was intended and what might be improved.
This breaks into the following steps:
- Write down the what. What is the problem you’re trying to solve or the experience you’re trying to improve? (It’s okay if you don’t know this yet. Take a guess.)
- Segment potential stakeholders and users into groups, or roles, based on how you think people currently get things done relative to your “what.”
- Write down why you think people do things the way they currently do.
- Identify a few people who can reasonably demonstrate the workflows and speak to the concerns of each of your previously identified groups. Put some time on people’s calendars. I try and talk to four to eight people, for at least 45 minutes each, per round of stakeholder interviews.
Consider context
Next you’ll want to think critically about the world(s) you’re about to enter.
Stakeholder interviews allow designers to collaboratively explore complex domains with help from the people for whom their designing. The goal in conducting these interviews, however, is to focus more on the people-scale problems than the problems affecting the domain itself. Said differently, you’re looking more for pain points that affect the people you talk to than things that affect the entire domain in which those people live or work — although the difference can be subtle.
In the case of 18F’s work with the Federal Election Commission (FEC), for example, one round of stakeholder interviews focused on the “campaign filer” experience. Our team spent time talking with people who the file with the FEC on behalf of a campaign (“campaign filers”) as well as people who work at the FEC itself. This research helped our team better understand how the FEC’s existing tools — its existing website, forms, flyers, systems, etc. — help those who are filing and taking receipt of information about campaigns.
Our goal for this research wasn’t to learn everything about how campaign finance law works. Though 18F’s team has been able to develop a much stronger working knowledge of campaign finance over time, we didn’t need that knowledge to get started. We just needed to learn enough to ask our questions in a coherent way.
Before coming up with your interview questions, ask stakeholders for any materials that they believe are essential to doing their job: What helps them get things done right now? Consider looking into books, blogs, mailing lists, meetup groups, etc. that are related to what they do. Even a quick glance over these materials helps ensure that you’ll use your time together wisely.
(Aside: If your stakeholder's time is especially limited, consider running a diary study. Ask stakeholders to make brief logs of the tasks they accomplish every day over a short period of time.)
Draft your questions
Once you know enough to speak your users’ language, draft the questions you might ask. I stress “might” here for a reason: many interviews naturally ebb and flow based on the person being interviewed and the tasks you’ve asked to observe (and, according to guidance we've received, that means stakeholder interviews aren't subject to the Paperwork Reduction Act — but be sure to check with your agency's PRA officer). Regardless, drafting questions forces you to roleplay and think strategically about the outcomes you want to achieve.
So fire up your favorite text editor and write down four to six areas or activities that you’d like to know more about. Then try and ask each of them in the form of a question.
Generally speaking, the best questions are both open-ended and grounded in reality. The researcher’s rule of thumb is to ask “how” and “why,” then ask “how” and “why” again. For example, you might ask “can you show me how you manage your campaign’s finances?” and “why do you submit that form on a quarterly basis?”
These questions are better than their closed, speculative counterparts. For example, “Do you think you’d use this if [we designed it differently?]” is a poor question because it will likely elicit a yes/no (closed) response. It also asks users to ponder a potential behavior, which often isn’t strictly up to them.
Finally, don’t be afraid to ask clarifying questions about simple things. People tend to shy away from asking clarifying questions because of how others might perceive them: no one wants to look stupid. Yet the goal here is to document your stakeholders’ understanding — not your own — which means you should absolutely ask clarifying questions, even if you think you know the answer.
Solicit critiques
The last step before sitting down with stakeholders is to share your questions with your team and discuss. This isn’t about copyedits; it’s about making sure that everyone’s on the same page with regard to the scope of the interviews and the scope of the project itself.
There’s also the option of soliciting critiques from people outside of the project team. For example, I’ve received a ton of great feedback from people on 18F’s content team about the language I’m using to ask my questions.
Let’s get this show on the road
And there you have it! In this first part, I’ve explained the rationale behind stakeholder interviews, how to document the assumptions guiding a study, how to consider context, how to draft questions, and when to invite your teammates into the process. In the next part, I’ll cover how to actually run the interviews as well as some tips for synthesizing and socializing what you learn.