Scalable threat modeling

Security vulnerabilities result from interactions between thousands of well-intentioned decisions. What systems or libraries do we use? Should I click yes or no on this authorization prompt? Should we make this system more simple or more safe? Only in the aftermath of security incidents do the regrettable decisions become clear.

Security experts are predictors. They are trained to understand how the conditions caused by technical decisions create risk, and can steer the ship toward calmer waters. Bad decisions made without security expertise tend to propagate and amplify before they can be identified and corrected. As a former team member put it:

A river of code and we’re standing here with buckets trying to keep things dry

How can security experts influence critical technical decisions as they are being made, rather than deal with the consequences? If an ounce of prevention is worth a pound of cure, then an hour spent threat modeling is worth a week of sprint work. It creates a channel for risk communication between security experts and decision makers that can prevent the cost of correcting bad decisions later down the line.

There are many different ways to threat model — a Carnegie Mellon blog discusses 12 Available Methods. Each organization has unique threat modeling practices and evolve their approach over time.

Threat modeling is a creative process, which makes it resource intensive. Security experts aren’t cheap, and threat modeling slows decision making. If you spend time on a process that’s not optimally-focused on reducing risk, you run into scaling limits — no more security experts, too much burden on decision making, or both.

The following ideas have helped me scale threat modeling processes at growing software organizations.

Document the process

This may seem obvious, but invest in well-reviewed “how threat modeling works” documentation. It helps you remember to follow all the steps. It helps other teams know what you mean when you say “threat modeling”. It’s repeatable when performed by different people on your team and can be improved over time.

An example high-level process:

1. Create a “Threat model” Jira ticket in the SEC project
2. Schedule and attend a threat model meeting
3. Answer any follow-up questions from the security team
4. Review and respond to recommendations from security design review report
5. Notify the security team before you go live

Have, use, and improve templates

Standardized and structured documents are easier to create and read. Document templates are like a code base, a leverage point for making sure you’re focused on the right things to optimize your threat model.

Focus on changes, not existing systems

Start with a high-level threat model of the big picture. Prioritize threat models for old and risky systems. But realize planned changes are the most useful ongoing target for threat models because the best time to clarify and correct decision making is before implementation.

Clarify what changes should be threat modeled

If you focus on threat modeling small changes, you’ll soon have more changes than you can threat model. Here are impact-based criteria I’ve developed to answer “what product features do we threat model”:

All new major features, including:

- OKR-sized engineering projects

- Any change involving the use of new infrastructure or services

- Any change involving exposing API endpoints to new clients

Features that affect security-sensitive functionality like authentication, access control, or cryptography

Change to data access flows for sensitive data

Changes to features that have have a history of vulnerabilities

Identify changes based on existing processes

Identify candidates for threat modeling by looping in to existing change communication channels, like an architecture review board, a change management process, or a daily standup. Ask “should we threat model this?” in a way that’s consistent with how your culture builds technology.

Give people an easy entry point

There are two ways to kick off a threat model — you contact the team, or the team contacts you. As part of their security training, people should understand the call to action — a bat signal — to ask the security team for help with a threat model worthy-change.

Allow for different size projects

There’s no good one-size-fits-all threat modeling process. As part of a triage process when threat models begin, categorize the scope, risk, and impact of the project. When there are too many changes and not enough threat modelers, be a parademic for smaller projects and prioritize the most important conversations. This may mean foregoing some of the more rigorous documentation, meetings, and so on. I’ve used t-shirt sizes to communicate the idea:

Small: 30 minute meeting with recommendations in ticket

Medium: Hour meeting, one-page report created

Large: Two hour long meetings, full threat model report created

Get the right people involved

You want the right level of detail and context without being a burden on team members’ calendars. Invite the people who own the answer to “why are we doing this” (e.g. Product Manager) and the people who own the answer to “how are we doing this” (e.g. Lead Engineer). It can be helpful to have multiple security experts, especially if one is new to the organization or early in their career.

Mix sync and async communication

Sitting in a “meeting that could have been an email” is painful, but so is getting consensus about important decisions over text-based communication. Do everything outside of meetings except things that benefit from synchronous interaction, like getting consensus. One good focus for high-bandwidth synchronous meetings is security experts learning how the system works, especially if documentation isn’t complete. Some questions I ask:

• What are the project goals?
• What people are affected by these changes?
• What new data is being introduced, and where is it going?
• What’s the overall system architecture?

Ask “what’s the last thing you want to happen?”

Anyone can have security insight. Ask the people involved about the last email they want to wake up to about this project. This can be a creative seed for the threat modelers.

Recommendations are the useful output

Threats models have several outputs — threats, countermeasures, system diagrams, and so on. Distill the analysis into a set of recommendations for the decision makers to make it actionable. Frame your advice as an opportunity, rather than a demand. In our process, we map each documented recommendation to a documented threat. This clarifies the “why?” before it gets asked.

Communicate the priority of recommendations

A scalable security team clarifies risks and mitigations but leaves big tradeoff decisions to the decision makers. An simple and flexible system for prioritizing threat model recommendations could look like:

High: Do this first, or get approval

Medium: Do this, or get approval

Low: Do this if you want.

Make sure the team agrees with the prioritizations. You may need to calibrate your perception of risk tolerance based on push back. Exceptions should be rare.

Figure out when to revisit

A threat model is a point-in-time snapshot. To make a threat model a living document:

• Link the recommendations to tickets and track their progress
• Ask to be included on code reviews for proposed sensitive changes
• Ask the team to create a new threat model for any additional significant changes

Threat modeling is one of the most impactful ways to work as a security engineer. I recognize potential practice improvements each time I perform it. If you’ve got tips to share, leave a comment or mention me on Twitter (@alsmola).