It’s time we rethink security metrics

Introduction

It’s about time we talked security metrics!

Security metrics have a special power. They help leaders make accurate decisions that reduce security risk. When a security dashboard appears in a meeting, it holds attention and carries weight. It’s a data-driven signal to leadership about the most critical security risks and how well a company is addressing them. At Depi, metrics drive decisions. We want to talk about the metrics people rely on, what we think of them, and how we believe they should actually be used.

I recently joined Depi as a technical writer, and part of my job is to help tell that story. In a previous life, I was a security engineer turned manager, spending most of my days translating technical chaos into actual business risk. Along the way, I’ve had my fair share of unforgettable experiences with security metrics: the good, the bad, and the downright ugly. But the engineer in me never really died, she got skeptical. That has kept me hyper-vigilant against feel-good dashboard telemetry, and I have zero hesitation calling out vanity metrics thrown at you to make your eyes sparkle. You know the ones: shiny, polished numbers designed to win sales pitches, while under the hood, they mean absolutely nothing.

Before we dive into security metrics, let’s align on the general definition of metrics. Put simply, metrics are measurements used to track performance, progress, or trends. Metrics shape decisions, define key performance indicators and key risk indicators, and often justify a bigger budget, additional headcount, or new tooling. Without metrics to prove the need for security or to demonstrate its value, many teams wouldn’t exist.

​In this article, we zoom in on a metric famously used in the industry as an indicator of performance of detection systems. One that tells us how quickly threats are identified and is often cited as a sign of detection maturity: Mean Time to Detect (MTTD). But unlike many people in the industry, we do not put MTTD on a pedestal as a performance metric. We like to argue that MTTD is misleading and notoriously gameable. In fact, when measured in a vacuum, raw MTTD is completely useless.

MTTD Breakdown

Before we tear this metric to pieces, we need to understand how the math works under the hood and why the industry fell in love with it in the first place. Because let’s be honest: when scoped accurately and used as an internal diagnostic, MTTD does capture something remarkably real about operational performance.

In its purest mathematical form, the traditional MTTD formula is simple: you subtract the timestamp of when a malicious activity actually began (t_start) from the timestamp of when the alert was fired or confirmed (t_detect), sum those durations across all incidents, and divide by the total number of incidents to obtain the mean. MTTD gives you the average timespan between when a security threat begins and when security teams detect it. Historically, automated detection tools were developed to reduce MTTD to mere seconds, enabling us to detect threats faster. The logic is simple: if I can detect a threat as soon as it hits me, I can better protect my organization.

From a vendor’s perspective, MTTD is the ultimate proof-of-value metric because it is the quickest way to demonstrate immediate, flashy ROI on a slide deck. If you are a detection engine or a modern security platform, you are selling visibility. You can’t easily prove a negative (i.e., we prevented three breaches today), so instead, you sell the speed of sight. In theory, shortening this window directly shrinks the attacker’s detection-free dwell time, that highly dangerous period during which an adversary can move laterally, harvest credentials, escalate privileges, or wipe logs while you remain completely in the dark.

We know that in security, time is damage. Speed of detection isn’t just a technical vanity metric; it has a measurable impact on the overall financial fallout of a breach. Frontline data continuously reinforces this connection. For instance, empirical research from IBM shows that breaches identified and contained within 200 days cost organizations an average of $3.74M, whereas those that take longer than 200 days cost an average of $4.86M. (Source) That is an average delta of over a million dollars purely attributable to slower detection and response. By keeping a tight leash on MTTD, organizations can systematically blunt the impact of attacks before they escalate into catastrophic, multi-million-dollar board-level disasters.

In short, knowing how fast you can spot a fire is a critical baseline. It is the starting point of any mature security program. But, as any veteran who has sat in the manager’s chair knows, knowing that a fire has started is not the same as putting it out, and that is exactly where the illusion of MTTD begins to crumble.

MTTD Teardown

While a falling MTTD looks spectacular on an executive dashboard, a deeper inspection of the metric reveals that it is fundamentally flawed as a standalone measure of security posture. In practice, over-relying on MTTD creates a dangerous illusion of safety while leaving critical operational blind spots completely unaddressed.

To systematically dismantle the MTTD myth, we can examine the data and surface four core arguments that show why speed of detection, as traditionally measured, is a losing metric.

The Selection Bias

Selection bias is the principle that the sample under analysis isn’t representative of the whole population because of how it was selected. Data is collected in such a way that some entries are more likely to be collected.

This image is a Depi-branded meme of a historical example of survivorship bias, a specific type of selection bias that focuses solely on visible data points and ignores those that failed, were missed, or didn’t make it into the dataset. During World War II, analysts studied bullet holes on returning aircraft to determine where to reinforce them. The data showed that the damage was concentrated on the wings and fuselage, so the initial idea was to armor them.

But statistician Abraham Wald pointed out the flaw: They were only looking at planes that made it back. The areas with no bullet holes, like the engines, were likely the spots where hits led to the plane being lost. Those planes never returned and weren’t part of the dataset. In security, the same thing happens when we only analyze detected incidents.

The most glaring limitation of MTTD is that it only measures detected threats, leaving the incidents you missed entirely out of the equation. (Source) If your SOC has finely-tuned detection rules for suspicious API requests but lacks visibility into lateral movement, cloud misconfigurations, or credential abuse, your MTTD will look pristine. The high-fidelity alerts fire instantly and skew your average detection time beautifully downward. Meanwhile, advanced threats and silent compromises can linger in unmonitored segments of the network for months.

Because these stealthy intrusions never trigger an alert, they are completely absent from the denominator of your MTTD calculation. Your metric improves not because your security is tighter, but because your visibility is narrower, allowing the hardest and most dangerous problems to remain invisible.

The Illusion of Stopped Clocks

Traditional MTTD calculations stop the clock at the millisecond at which an alert is generated. This is a reporting convenience that completely ignores operational reality. Imagine a critical alert firing within seconds of malicious activity, registering a near-zero MTTD on paper, yet sitting unreviewed in an analyst’s queue for six hours. Operationally, the attacker had a six-hour window to move laterally, harvest credentials, or stage data before the organization had any actual awareness. (Source)

In an era where the average eCrime breakout time has collapsed to 29 minutes, and initial access-to-ransomware hand-off windows have shrunk to just 22 seconds, an alert sitting unreviewed in a queue is effectively a breach in progress. (Source, Source) MTTD reports on alert generation (the part of the problem the industry has largely solved) while completely hiding the downstream exposure of the post-alert investigation gap.

MTTD vs. Real Dwell Time

The MTTD clock only starts running when your telemetry triggers an alert, whereas the real battle is fought against Dwell Time, which starts at initial compromise.

Dwell Time is the time an attacker spends in your environment before being expelled. It encompasses both the detection gap and the response gap. Retrospective post-incident forensics reveal that sophisticated adversaries spend weeks living off the land using native, legitimate administrative tools and stolen credentials to blend in with normal network traffic. Because these early reconnaissance stages generate no alerts, they are invisible to MTTD. (Source) By the time the first detectable signal fires and starts your MTTD clock, the compromise has already occurred, and the damage is done.

Today, this visibility gap has grown far more glaring as offensive AI compresses the attack lifecycle. (Source) With adversaries leveraging automation to discover, exploit, and escalate privileges internally, breaches that once unfolded over weeks of manual lateral movement can now be executed in minutes. (Source) Attackers have industrialized their speed. Consequently, even near-instantaneous detection is useless if response capabilities lag behind.

The True Indicator of Performance

Knowing that a fire has started is useless if you lack the means to put it out. A rapid detection that leads to an incorrect, incomplete, or delayed response is not a victory. It is a liability. (Source)

If your MTTD is six minutes but your response takes hours, the attacker still gets hours of unchecked network access. (Source) This is why response metrics like Mean Time to Respond (MTTR) and Mean Time to Contain (MTTC, often included in MTTR) are far superior indicators of actual security performance. While MTTD merely tracks a process output (alert generation), MTTR directly measures the containment and remediation of risk, essentially serving as the true indicator of the performance of your actual security response.

The speed and quality of your response lifecycle have a direct, audited impact on the severity of a breach. Faster containment systematically minimizes lateral spread, prevents data exfiltration, and dramatically reduces recovery costs. True operational resilience does not come from watching a stopwatch fire an alert. It comes from decision quality and response accuracy under pressure. Real security performance must be judged by how quickly a team achieves total situational awareness and executes containment playbooks to evict the adversary before damage can occur.

The Shift the Industry Needs

Over the last couple of years, treating Mean Time to Detect (MTTD) as the ultimate security metric has begun to fall out of favor, and for good reason. While defenders have previously made progress in shrinking the attacker’s window, the latest Mandiant data reveal that the global median dwell time has risen to 14 days (up from 11 days) as sophisticated adversaries get better at evading detection. (Source)

We are witnessing an industry pivot away from raw detection speed and straight toward Mean Time to Acknowledge (MTTA) and MTTC, which are part of MTTR. Security operations leaders have realized that if human analysts are drowning in daily alert volumes and suffer from severe alert fatigue, even instantaneous AI detection becomes useless.

Major global players are proving the definitive financial and operational value of prioritizing rapid, automated response (MTTR) over raw detection (MTTD):

• IBM’s Cost of a Data Breach data reveals that organizations deploying extensive security AI and automation shortened their breach lifecycles by 80 days and saved an average of $1.9 million per incident compared to those relying on manual workflows. (Source)
• Real-world enterprise implementations mirror these gains. For example, Nestlé consolidated its IT assets and automated its vulnerability operations to eliminate manual tasks, successfully slashing its Mean Time to Repair (MTTR) by 80% and achieving same-day patching. (Source)
• Even the widespread enterprise adoption of Zero Trust architecture, now fully or partially implemented by 81% of organizations, is fundamentally an MTTR-related containment strategy. By establishing identity-centric trust boundaries, Zero Trust is specifically designed to automatically limit the blast radius of a compromise before responders can even log on. (Source)

Forward-thinking security leaders are now treating MTTD simply as an indicator of the effectiveness of underlying tools and telemetry, while treating MTTR as the true benchmark of organizational maturity and resilience. The industry’s financial and engineering efforts are shifting from simply buying more alarms to building highly orchestrated, automated sprinkler systems.

Conclusion

The math of modern cyber defense has permanently changed. As adversaries utilize automated pipelines to exploit vulnerabilities before patches even exist, and execute initial-access handoffs in a mere 22 seconds, defenders can no longer afford to operate at the speed of human review. The post-alert investigation gap is the new playground where threat actors quietly dwell and destroy before security teams even finish triaging the initial alert.

Relying on a declining Mean Time to Detect as proof of security is like bragging about how fast your smoke detector chirps while the house actively burns down. Security leaders must have the operational courage to stop optimizing for the velocity of the alert and start optimizing for the completeness of the response.

How Depi Reduces MTTR

When a new software supply chain threat emerges, security teams often lose valuable time assessing the blast radius, digging through dependency trees, and coordinating with development teams. That operational overhead is where MTTR slows down. Depi tackles this bottleneck by replacing reactive guesswork with offensive, threat-informed automation.

First, our platform performs complete dependency tree scanning to understand exactly where every dependency lives across your ecosystem. When a new vulnerability is disclosed, you immediately know which projects, repositories, or directories are affected and do not waste hours searching for where it lives.

Next, Depi prioritizes threats based on actual risk. We score findings by how widely they have spread across your ecosystem and the importance of the affected repositories. An actively used production repository carries far more weight than an abandoned proof of concept, allowing your teams to focus on what truly matters. Depi calculates the precise blast radius inside your source code, showing exactly which projects are impacted and how far the vulnerability reaches. Instead of spending hours on manual research, your team receives a clear remediation roadmap. A ready-to-ship fix is then generated directly in your development pipeline, turning what is often a multi-day response into a same-day patch and considerably improving your MTTR. This enables security teams to generate and propose fixes directly, reducing their reliance on development teams whose priorities often compete with security remediation.

Beyond MTTD: Introducing MTBE

At Depi, we’re shifting away from reactive speed metrics like MTTD. Instead, we focus on the metric that actually keeps security leaders sleeping at night: Mean Time Before Exploitation (MTBE).

By combining threat intelligence with offensive research as previously described, Depi effectively shows you how attackers can break the links in your software supply chain before they can exploit them. When an active exploitation of a reported threat is eventually spotted in the wild, our exploitability-based prioritization proves your ROI in negative time, showing you exactly how long your ecosystem remained fully bulletproof.

The time has come to systematically retire our industry obsession with MTTD. Security maturity is no longer defined by boasting about a six-minute MTTD, but by having the operational maturity to aggressively minimize MTTR and neutralize threats before they become breaches. After all, the best incident isn’t the one you detect the fastest.

It’s the one that never happens because your MTBE was longer than your attacker’s window of opportunity.