AI Is Breaking the Economics of False Positives

TL;DR

Security teams tolerated noisy scanners for twenty years because the queue appeared manageable.
Coding agents have multiplied the volume and size of code, but AppSec teams have stayed the same size.
The growing pile of findings (and noise) shows that evaluating tools on false positive rate is a losing battle.
Teams must shift focus to exploitability across architecture, code, and infrastructure.

Security teams tolerated noisy scanners for twenty years. False positive rates that would get a tool thrown out of any other engineering discipline became routine in AppSec, and everyone made peace with it, because a human could always work the queue. Slowly. Painfully. But the queue got worked.

That peace is ending, and I hear it in every conversation with a security leader. Their scanners didn't get worse. The code got faster.

The number of vulnerabilities never mattered

False positive rate is easy to measure, so it became the number vendors compete on. Run a bake-off, count the bad findings, pick a winner.

Except nobody tolerated the rate because it was fine. They tolerated it because a human could still clear the queue. That was the only number that ever mattered: not how many findings a SAST scanner produced, but whether the team could get to the end of them. That worked at 100 PRs a week. Put the same scanner on 300 PRs and the same rate buries the team, while the number on the vendor's slide hasn't changed at all.

To be clear, finding vulnerabilities is good. The problem is what the scanner calls a vulnerability. A SQL injection pattern in a function nobody can reach is not a vulnerability, it's a bug. Software is full of bugs. Any codebase of real size has thousands of them nobody will ever fix, and nobody loses sleep over that. The scanner takes a few hundred of those bugs, labels them critical, and puts them in a queue your team is now accountable for.

Email security went through this already (to be fair, it’s going through this again now). Keyword spam filters flagged "free" and "win," caught plenty of real mail, and nobody cared while inboxes were small. Then volume exploded, the same error rate produced an unrecoverable pile of mistakes, and keyword matching died. Bayesian filtering replaced it. SAST ran the identical arc: untuned tools shipped false positive rates above 90 percent, teams hired humans to triage, and the arrangement held for one reason only. Code volume stayed inside what humans could process.

We have more code now.

Cursor says enterprise customers are pushing 25 percent more PRs and the PRs are twice as big. Microsoft says engineers with coding agents merge 24 percent more per day. I don't need the studies. I sit with product security leaders every week and the number they give me is their own: double, sometimes triple, in under a year, with the same three people on the AppSec team.

So teams are trying to solve it.

The first move is always headcount or another tool. I've watched it a dozen times and I've never seen it close the gap, because you get one more person and the engineering org gets Cursor.

The scanner they bought in 2021 is still running. Accuracy is improving with AI overlays, just not fast enough, so a lot of ProdSec teams default to building something homegrown that "cleans up" or "validates" scanner output. They are now paying twice. Once for the scanner, and again in tokens to make its results usable.

And the results are the same. The scanner still has no idea whether the thing it flagged can be reached. Cleaning the queue faster just gets you to the bottom of the same queue, and the real one you were worried about is still somewhere in it.

So what matters today? Attacks.

Once the queue is the constraint, what's in it matters more than how long it is. The only findings worth an engineer's time are the ones an attacker can actually use.

Knowing that takes context a scanner never has: how the system is designed, where the code runs, what data it touches, which components talk to each other, and whether there's a real path from entry point to payload. A scanner flags a line. It cannot tell you if the line is reachable.

That is what Prime is built to answer. Prime keeps shared context across architecture, code, and infrastructure and reasons to exploitability before anything hits the queue. Findings are attack paths, not flagged lines. Customers see 100 criticals become 3. The 3 get fixed. The other 97 were never going to be, no matter how many people were working the queue.

So what's next for product security leaders?

Stop running bake-offs on false positive rate. The number is real. It is also the wrong number. It tells you how clean the queue is, not whether your team will ever reach the bottom of it, and the bottom is where the problem lives.

Ask a different question. How many of these findings can an attacker actually use, and how does that number move when my engineers ship twice the code next year? If the answer goes up with volume, the tool is counting bugs and calling them vulnerabilities.

Finding them was never the problem. The scanner found every one. It also found the 3,000 bugs sitting on top of them, and your team is going to work that queue top to bottom until the day someone outside the company finds the real one first.

See how Prime uses persistent context to identify real exploit paths and reduce security noise at primesec.ai.

External sources worth linking

Copied!

The Vulnerabilities You Will Never Fix