Building effective security OKRs

This is an abridged version of my LocoMocoSec 2020 conference talk

  • Align and Connect for Teamwork
  • Track for Accountability
  • Stretch for Amazing

Security objectives

Define the mission with high-level security objectives

First, make sure you have a set of security objectives that make sense outside the security team. They should describe how security supports the organizational mission. What risks are your team focused on preventing? If your organization has cascading OKRs, think about how security can be framed as a company-level objective.

  • Protect the Company
  • Assure the Customer

Create balanced security objectives

I work at LaunchDarkly and our customers have high security standards. To assure them, we undergo ISO 27001 and SOC 2 certifications. Most of the effort spent obtaining these certifications isn’t about reducing actual risk. Instead, it’s meant to meet reduce perceived risk and assure the customer so we can enable revenue goals. Following these standards also reduce actual risk. We can have initiatives that fulfill many objectives.

Think of your security team as a product

If you want to build a successful security team, make sure people want what you’re providing. Think like a product manager.

  • Security tools can balloon CI build times. Can you have an objective around developer experience?
  • Can you send out NPS surveys after your threat modeling exercises? Would other teams recommend your security team to other people?

Long-term objectives drive short-term objectives

After you’ve defined high-level objectives, you can create quarterly objectives that have higher resolution.

Security key results

Key results should be decidable

Vague key results are too open for interpretation. The “process is documented” or “false positive rate is improved”. If two people might disagree on whether you hit the key result, change it.

Key results should be actively tracked

Another problem I’ve seen is that the key results sound good but the metric isn’t defined or tracked.

Challenge and inspire to the team

Define the direction you want to go with an objective and the criteria for what it looks like when you’ll get there with a key result. You can set aggressive targets for key results and give people ownership and autonomy to meet them. It’s a great way to get creative solutions to problems and it gets people engaged in their work.

Drive team learning with accountability

Another big failure mode with key results is to “set and forget”. If you don’t track whether you hit your target, you won’t improve. Discipline around planning should extend into grading and analysis.

Example security OKR: code vulnerabilities

Based on these guidelines for effective objectives and key results, I’ll walk through an example OKR. Let’s start with an objective:

  • Severity (CVSS)
  • Service
  • Team
  • Source
  • Time introduced
  • Time identified
  • Time mitigated
  • Time to remediation of vulnerabilities
  • Average vulnerabilities over time

Consider measuring risk

Another style of OKR you could consider is to use risk itself as a metric.

  • We are always doing risk assessment — formally or informally
  • Expert estimation is effective even with incomplete data
  • Instead of risk matrices, measure risk as dollars

Process for creating Security OKRs

I hope you have a better idea of what good security OKRs look like. Here’s our team’s process for building OKRs.

Assess prior OKRs

First we meet as a team for a post-mortem on the previous quarter OKRs. We discuss what worked and what didn’t. If we didn’t meet a key result, we may want to try again, change it, or take a new approach.

Do a live group risk assessment

There are a couple ways I’ve asked the team to prepare for an OKR brainstorm. The first is an informal risk exercise. We come up with the most likely and impactful risk scenarios. What keeps people up at night? Where do we know we don’t have adequate protections? What stories have we heard about trends that make us concerned? This frames our ideas with our risk reduction objective.

Find inspiration through an article or video

Another preparation strategy is to ask everyone on the team to read an article, book, or paper or watch a video together. I’ve used the video of Clint Gibler’s “An Opinionated Guide to Scaling Your Company’s Security” talk from AppSecCali. It’s a great compilation of talks from leading application security teams. This can help open you up to more ideas, leading to a more creative brainstorm.

Do a bottoms-up brainstorming exercise

During the bottom-up brainstorming exercise I gather ideas from the team about what they want to work on next quarter. I like to encourage people to think about it beforehand. I remind them to be good advocates for an idea, they should come prepared with data. They should show the impact of their proposed projects.

Use post-it notes and voting

I get people to write down their ideas on post-it notes. Each person puts them on the wall, clustering related ideas. After narrowing down our ideas to a set of common themes, we run a voting exercise. Each team member ranks their top five ideas. We can then calculate the priority of each project, according to the team.

Set up time to review with other teams

After our team brainstorm, I reach out to other teams to get a sense of their quarterly OKRs. The DevOps or IT team are often leverage points for security efforts. Where possible, I like to establish shared objectives. I use the “two sides of a bridge” analogy. We agree to build out separate parts of a complementary project that meet in the middle:

  • You build the observability dashboard, we add the authentication and access control layer

Stay tuned in to company wide goals

It’s also a good idea if there are company-wide goals to ensure that you’re aware of them and supporting them. Are there any new initiatives that need security focus? It may reduce our bandwidth.

Prioritize objectives based on impact

At this point, I have the materials for a good first draft of OKRs. Since I’m responsible for prioritization, I need to decide what to cut. I rank the projects based on their impact and effort. There are two other things to consider. One is confidence — how likely is this to result in the impact? Am I comfortable taking on high-risk, high-impact projects? The second is cost of delay. A risk that exposes the organization today is higher priority than something that prevents a future problem.

Document objectives, key results, and initiatives

I’ve seen OKRs that are just a few sentences. I think it’s important to invest time in writing OKRs with more detail. How does this objective tie in to higher-level objectives? Why did we focus on this objective over other objectives? How are we measuring the key result? How did we choose the target for the key result?

Publish OKRs to get feedback and alignment

Next, I send out our OKRs to the organization for feedback and alignment. Once the OKRs are set, I ask people in my 1:1s what they want to work on. I assign ownership for meeting the KR based on their interest and capability. It’s a great way to build autonomy and career growth for individual contributors.

Execute, rinse, and repeat

At this point, the team can start documenting project plans. As the quarter progresses we have a monthly check-in on projects and key results. If it’s clear that we’re not going to hit a key result, we can consider changing it. The goal here is to create more and more predictable outcomes, which let us place bigger and bigger bets.

Security for the people.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store