Agencies often come to us because they are experiencing pain using existing systems. Their system might be hard to maintain, outdated, hard to use or hard to navigate. They’re ready for a change. They may even have some ideas for solutions — maybe a dashboard, a landing page, or another web application. 

Our approach to a technical assessment is designed to ensure that everyone fully understands the problem and has come to the best solution — not necessarily the first one that came to mind. To do that, we start by defining the general problem space surrounding the technical assessment in focus, then assessing the system itself, before clarifying the organizational realities and user needs involved. In this post, we outline briefly what 18F does during a technical assessment.

Defining the problem space

People are problem solvers, so it is natural for us to want to find solutions. But before we jump to a solution, do we have a complete understanding of the problem? We may see bits and pieces of the problem and know some solutions we used before or others are using to solve similar problems, which can lead to partial or ineffective solutions. This is where our Path Analysis offering comes in. Path Analysis is the first phase of most 18F projects. It’s how we assess the best ways an agency can solve the problems they’re grappling with. We like to start them by asking questions like these:

  • What are your goals? What will things be like when you’ve solved this problem?
  • What pains are you experiencing?
  • What pains are your users experiencing? How do you know?
  • What do we need to learn more about before we can determine the best path forward?
  • What assumptions are you making about the project or system? Do you assume users need certain features, come in from a single entry point, or need a web application?
  • What assumptions are we at 18F making? 
  • How do we validate or disprove your assumptions, and ours? 

Agencies, of course, want their new system to actually deliver what it promises to do. Especially if the current system does not. To make the best determination of what solution will best deliver, we have to look beyond the technical considerations alone and take a deeper look together at the organizational environment, the systems themselves and who’s using them. Then, we work with our partners to pair quantitative data and qualitative findings with industry best practices and experience to de-risk government IT projects.

Assessing your technical systems

When assessing your technical systems, we look at your code and beyond, to the technology being used. We research how the system meets user needs and its suitability for performing necessary tasks. We talk to your technical team and stakeholders to understand how your systems were developed, how they are connected together, and what constraints the new system will have to take into account. During this process we will:

  • Review and understand your current architecture and technical capabilities
  • Analyze your codebase
  • Examine technology and systems being used
  • Assess your systems based on the criteria we define together

Some of the initial questions we will explore are:

  • What technologies are you using? Why were those technology choices made?
  • What does your current architecture look like?
  • What are the pain points in development and deployment?
  • What is preventing you from making changes to your current process and systems?

By asking these questions and many more, and looking at your systems in depth, we can explore some of the gaps and needs for your future systems. You may have questions you would like to explore as well.

Understanding your organization

Beyond assessing your technical systems, we need to understand your overall organization to suggest pathways to sustainable change. That’s because technical systems are not built in a vacuum;  just fixing the technology will not address the systemic issues that may have allowed problems to develop. When we understand what contributed to the current state, we can develop durable solutions — otherwise an organization can easily fall into the same problems again.

Some activities we would do during this process include:

  • Map out your organizational and funding structures
  • Understand how decisions are made and who are the decision makers
  • List out your organizational goals and constraints
  • Understand the history of your organization and systems
  • Map out staffs’ roles and responsibilities

By performing these activities, we learn how these constraints affect current systems. We can then provide recommendations that you can use for immediate improvements.

Researching your audience and users

Another key part of the assessment is to understand the users of the current systems. You may know from analytics how users are behaving on your systems, or you may have support tickets indicating certain issues and pain points. But this only tells you part of the story. By pairing these data points with real user research, we can help you understand why users are doing what they are doing and how you can make the system meet their needs better. If we make recommendations based on our own analysis and assumptions without talking to users, the new changes we make may not help.

During this process, we will coach you on how a user-centered approach to development helps solve real user problems. By speaking to users using techniques like cognitive walkthroughs and interviews, we help you make sense of your usage metrics, validate or invalidate assumptions, and focus your limited time and resources where they will do the most good: helping your users use your systems. Users are also often excited to discover that you care about their needs. 

Findings and recommendations

Throughout the path analysis, we share findings and recommendations at various checkpoints. Part of the technical assessment of your current system is helping you understand how well it delivers for your users. We make an assessment based on the criteria and goals we identify together early on, and share how well the systems currently satisfy each of them. We also share what we discover in terms of your overall organization constraints and structures.

We then provide recommendations: both long term goals and strategies and immediate next steps. 

Immediate next steps are the most actionable recommendations we’ve found to improve quickly and position your agency for further improvement. After these steps, you validate with your users and continue to make changes accordingly. This will ensure you are always building what the users want and continually meeting their needs. You plan your next steps and improve incrementally and iteratively.

Long term goals and strategies help you navigate and continue to improve, guiding you to where you want to get to with your system while giving you flexibility to incorporate changes as they arise. We focus on helping you define those goals and find a path forward rather than building a complete plan and schedule to finish. We do this because we have found that it is not possible to learn everything up front. Instead, you need to learn as you build, finding out what works and doesn’t work, and changing the long-range plan accordingly. The earlier we learn, the easier we pivot to another path without costly consequences. This is one way we help you reduce long-term risk and cost.

The idea is to think big and start small. You know where you want to go, so you take a step forward and check if you are still going the right direction. If you don’t keep checking and you wait until the end, you may find out you were going the wrong way when you reach the wrong finish line.

Let’s work together

We understand it is painful to see your current systems not delivering. We would love to work with you to start improving your systems for your users. Contact our Business Development team at inquiries18F@gsa.gov to learn more.