UX designer Nick Brethauer has spent the past few months working on Discovery, a website that makes the procurement process a lot easier for GSA’s contracting officers — the people who are responsible for buying products and services for the federal government. Discovery lets these contracting officers find and sort vendors based on their experience levels and other attributes.
Nick recently answered a few questions about what he does at 18F and how user research better informs the products 18F builds.
Tell us a little about yourself and what you like about working at 18F.
My name is Nick and I’m a user researcher and UX designer. At 18F I’m working primarily on procurement projects. One of the coolest things about 18F is that, from a user research perspective, one of our fundamental challenges is getting agencies to work in a user-centered way, which is a different way for government to work. It’s very rewarding to essentially be a key change agent for how the government provides services.
I’m not sure everyone is familiar with the terms user researcher and UX designer. How do you describe what you do to people?
The longer I’m at 18F, the more broadly I describe my role. When I started here I thought that as a user researcher I was going to be going out and doing mostly researchy-type things for the projects I was on, like user recruiting, interviews, contextual inquiries, these kinds of data gathering type activities. However, because 18F is still relatively small but yet doing so many things, research and UX are more typically a combined/holistic role on the 18F team.
I would describe the research side of my role as being someone who is responsible for going out and speaking with users or the customers of our client and learning from them -- their problems, their frustrations, their pain points. I also observe them and see what their days are like. I look at their overall day-in and day-out experience and think about the various touchpoints that we could be involved with or learn from when we’re designing. So that is more of the research side.
In terms of the UX side, I’ve been leading the use of the “lean UX” methodology on my team, so a lot of my role there is facilitating the involvement of the whole team in the design process. That includes research and discovery, persona building (fictional users designed to mimic the types of users who would use the site), collectively mapping out our assumptions about the problem/solution/users, figuring out which of these to tackle to learn more, and breaking all of it down into testable hypotheses. We also do lots of team design studios/sketching, and then together test each iteration with our users.
Are you asking people questions?
Well, I ask questions, but I also observe people doing their work. That happens in different ways depending on what stage of a project you’re in. At the very outset of a project, I do more discovery and exploration around general issues. As the project develops, I ask more detailed questions about the specific problem we’re trying to solve.
For the procurement project you just worked on, do you remember any of the questions that you asked people?
Well, at the very beginning we asked very general questions. We asked the contracting officers to teach us about procurement.
That’s a good starting point.
It was, because it covered a lot of ground, and it allowed us to be naive, because at first we were naive, we had to learn everything about their jobs and the process they went through. So that was initially a lot to learn. By asking them questions, we were then able to jump into a more specific line of questions every time we heard something that would make us think ‘Oh, we need to follow up on how they manage their documents’ or something like that.
So they would say something like "I have trouble managing my documents" and then you would say…
They wouldn’t always be that explicit about it. A lot of times, you have to read into things and probe a little bit and ask specific questions and ask users to show you, rather than tell you, because you can’t always rely on what people are saying.
If they say "This is my problem,"" it may be the case that what they’re saying isn’t their problem: it may be a combination of things, or something else that they’re not consciously aware of. My job is a combination of asking questions, of trying to learn the context of how they work, of observing their use of specific tools, and then asking more pointed questions once we start to develop the vision or know which potential problems to focus on. That’s when you can start getting more specific with the types of questions that you’re asking.
So after you learned all about procurement, how would you describe what contracting officer do to people who don’t work in government?
The term procurement covers a whole lot. The simplest way to describe a procurement officer’s job is to say that they buy things on behalf of the government and, at the end of the day, they are responsible for buying that thing at a fair and reasonable price.
How does Discovery make a contracting officer’s life easier?
Depending on what is being bought, the procurement process can be months and maybe even years long. Contracting officers have to write the contracts, do the market research, and go through a lot of internal steps before they can deliver a service.
Discovery fits into a particular part of that process. When we talked with contracting officers, we learned they now have to go to a number of different sources and data sources to learn about potential vendors or communities that deliver a particular kind of service. And so they had to go out to all of these different systems and piece things together to create their report. It was an extremely time consuming process because there wasn’t one place to get that information.
That’s where Discovery fits in.
How did you determine that Discovery was the tool that you needed to build?
It’s funny, because Discovery wasn’t the tool the client initially wanted us to build.
Oh really?
Our client was a program office at GSA. And they initially wanted a task order authoring system or tool. Initially, we were sent in the direction of exploring problems around task order authoring.
What is task order authoring?
It’s basically a simplified contract that contracting officers have to create. So we started off by saying ‘Okay, we’re going to learn about task orders. We’re going to learn about procurement.’
And we learned about the systems that contracting officers used, and learned that everyone had their own way of completing [task ordering] and it wasn’t actually a problem.
So your client initially thought it was a problem but the people who worked with your client said, "Nope, not a problem?"
The contracting officers themselves had ways of making task orders. Through those conversations, we started to gather evidence that an actual problem for contracting officers was not having a user-friendly way to conduct market research -- specifically, identifying potential vendors and understanding their contract histories and things like that.
So we basically gathered up this evidence and then came back to our client and said "This is what we discovered and learned and we would recommend going in this other direction" and the client was actually, really on board with that pivot and going away from his own idea.
How did user research inform that process?
The initial user research led us to work on a completely different problem than the one we began with, which is important because if we hadn’t done the user research and asked questions at the beginning, then we probably would have started down the path of creating a task order authoring tool for the client.
Maybe we would have changed direction eventually, but maybe not. Probably not. Because once you’re invested in going in a certain direction, you have the snowball effect of everyone’s time and resources also being put into the problem. Even in the best of circumstances and the best of lean/agile world, it’s hard to go back and say "No, this is not the right thing to be solving."
Throughout building the tool, we also used lean UX methodologies. We built a working prototype every two weeks and then tested it at the end of each of those sprints with a few contracting officials, and then were able to change the tool to make it better based on their input.
What kind of feedback have you heard from contracting officials about Discovery?
When we showed a prototype to the audience during the 18F Demo Day, there was a group of contracting officers watching through a Hangout and one said ‘This is the best tool ever. It’s going to save my officers so much time.’
We also got very positive feedback while we were doing the testing. The testing was a hybrid of usability testing — looking at the interface itself and finding ways to improve it— and then validating the data and different information people would get out of the tool.
But the true testing of this tool will come after we release this first iteration of Discovery. If you think of the software lifecycle as being always ongoing and never really done, then what we did was the first version. We just released a first version and now we need to collect analytics on that and do more observations about how contracting officers are using it in the real world. Over the course of a six-month-long procurement process, they might go back to it several times. You can only learn so much when you’re developing a software product and then you have to get it into people’s hands and go back and revisit.
So where’s the project now and where’s it going to go?
It was just released. And we’re collecting data, we’ve got analytics on it, so we’re going to collect data and continue building up a user community of people we can talk to or interact with about it.
What would you consider to be a metric of success? What says to you "Boom, we were successful in this project?"
We define this in our project. We want to see reduced overall time spent in doing the market research and creating the market research report. We want to measure that it’s easier for contracting officers to use this tool instead of what they previously used. The biggest metric of success is time. Have we shortened the process down to some reasonable amount of time? Discovery will also lead to people making better decisions, which means cost savings and ultimately better services.
What do you think the most exciting thing you’ve learned during this project?
I think my biggest takeaway is that if you can improve procurement, you improve so many things. You improve, like, everything. The scale of impact that you can have by working on procurement is great. And that’s something that I didn’t understand or expect going into this project, because procurement is not the most sexy thing. It does glaze people’s eyes over. But the government spends billions and billions of dollars a year on stuff and services, so if you can improve the way the government buys things and improve the quality of things that we buy and affect the outcomes of the reason they’re buying things to begin with, then you’re in turn having all of these trickle down effects. I think that’s the biggest thing.
Thanks Nick. Anything else you’d like to add about procurement or 18F?
18F is getting people and agencies excited about digital services. I think that’s really huge. 18F, by doing that, is supporting the voices around government that have been advocating for better digital practices. I think that’s really cool.
(Related — Making Procurement Easier: Some Questions with 18F’s Kaitlin Devine.)