6 buckets of prodsec
A product security team exists to protect people and their data.
We do this by making our companies products as secure as possible. Software security is hard though. As a species we are not capable of writing vuln-free software. Hell, we cannot even determine how secure or not it is with much certainty.
Given that, our goal should be to maximally reduce risk. I do this by tracking the outcomes of vulns.
There are 6 buckets a security bug can fall into on its journey through life
- Prevented — best outcome. Never turned into code.
- Found automatically — Found via static analysis or other tools. “cheap” time cost.
- Found manually — Good even if it took more time. A large set of bugs can only be found this way.
- Found externally — usually via bug bounty. Put users at real risk, expensive time cost but 100x better than other outcomes.
- Never found —Most bugs probably end up here.
- Exploited — the worst.
Prevented > autofound > humanfound > externally found > unfound > exploited.
Whenever we shift a vuln leftward on this spectrum, we are winning.
- Doing a security review of a sketchy part of the codebase? You just moved n bugs from never found -> found manually. Success!
- Writing a static analysis rule based on a bug bounty report? Found externally -> Found automatically. Success!
- Handled an incident where bugs were exploited, learned the lessons from it and turned it into rules, tools and secure frameworks? Success.
We aim to shift as many bugs as possible, as far to the left as possible. That’s it, that is our whole job.
Methods
Here are lots of methods to nudge future bugs towards a given outcome bucket
- Prevented: safe by default languages and libraries, design reviews, training & documentation
- Found automatically: static analysis, dynamic analysis, lint rules, tests, fuzzing.
- Found manually: Security reviews and consulting, self-service tooling
- Found externally: Bug bounty, external scanning services
Each method has a cost (time, $$$, friction to company), a return (# of bugs shifted and how far). Each company, situation, team has different trade-offs here.
Conclusion
All teams will still mess up sometimes and bugs will go unfound until they are exploited. It happens.
This is painful and bad but security is not an activity where anyone can bat one thousand, its just the nature of the problem. What we can do is put our limited resources behind the methods that shift the most bugs, the furthest to the left.