A recent study has shown once again why organizations need to look beyond vulnerability remediation when it comes to managing and mitigating cyber risk from software.
Introduction to Malware Binary Triage (IMBT) Course
Looking to level up your skills? Get 10% off using coupon code: MWNEWS10 for any flavor.
Enroll Now and Save 10%: Coupon Code MWNEWS10
Note: Affiliate link – your enrollment helps support this platform at no extra cost to you.
A new study by Purdue University researcher showed how the new Exploit Prediction Scoring System (EPSS), which many organizations are now using to prioritize vulnerability remediation, is not as effective as previously assumed.
The study shows that like other vulnerability risk assessment frameworks such as the Common Vulnerability Scoring System (CVSS) severity scores and CISA's catalog of known exploited vulnerabilities (KEV), the EPSS is useful, but not a completely predictive mechanism for protecting against vulnerability related threats.
Here's what you need to know about the limitations of the EPSS — and why your application security (AppSec) strategy needs to extend to more modern approaches.
[ Download Today: 2025 Software Supply Chain Security Report | Join the SSCS Report Webinar ]
Predictive models don't tell the entire story
Ken Dunham, cyber threat director at Qualys threat research unit, said that predictive models like that of the EPSS are helpful indicators, but never tell the entire story — and that they should they be relied upon for the full context of how to identify risk to an organization.
Risk to an organization is best identified using a variety of indicators and measurements. That means, in addition to using CVE, CVSS, EPSS and KEV, organizations need to consider other factors including attack surface, criticality of assets, and environmental factors when making risk mitigation decisions, Dunham said.
"The bottom line is that all organizations should have a risk-based process in place, personalized to their assets and architecture, to best identify and prioritize risk as the threat scape changes and emerges. It's less about how to improve a predictive model and more about the process of how you manage risk using these predictive models as tools, and as only part of the equation."
—Ken Dunham
EPSS, which is maintained by the Forum of Incident Response and Security Teams (FIRST), is a vulnerability scoring system that uses machine learning (ML) to predict the likelihood that a software vulnerability will be exploited in the wild. The goal of the EPSS is to help security teams prioritize patching efforts based on actual exploitation risk rather than just severity scores.
EPSS scores range from 0 and 1 with higher scores indicating a greater probability that a vulnerability will be exploited. Many expected EPSS would help move the needle forward significant on software risk.
A trailing indicator of risk
In the Purdue study, the researcher, Rianna Parla, focused on high-severity bugs in the US Cybersecurity and Infrastructure Agency's (CISA) KEV, and compared it against each vulnerability's EPSS score history. The goal was to find out if a vulnerability's EPSS score was a good indicator of whether the vulnerability would be exploited in the next 30 days, or not. The assumption was that the EPSS would reasonably predict a CVE's likelihood of exploitation before the vulnerability landed in CISA's KEV.
The analysis showed that while in some cases EPSS did exactly that, in many other cases it did not. The results led the researcher to conclude that EPSS is more of a trailing indicator than a predictive system. Parla wrote in her research:
"The Exploit Prediction Scoring System name may be a bit misleading as it is less of a prediction system and more of a risk scoring system. In other words, what is the risk of exploitation of a given vulnerability versus other vulnerabilities. Most of the analyzed data showed that EPSS was less effective as a measurement than simply using the CISA KEV catalog directly."
The results are significant. Because it is impractical for organizations to remediate every single vulnerability in their environment, many use EPSS to prioritize the risks they need to address first. Unlike CVSS scores which measure the potential severity of a vulnerability's impact, EPSS predicts the likelihood of attackers actively exploiting the vulnerability using a combination of real-world exploit data, threat intelligence and ML.
Context is key
Jeff Williams, co-founder and CTO of Contrast Security, said the key to unlocking any benefits from such mechanisms is context. He said you need technical details on what systems are being targeted, what attack vectors are being used, which kinds of vulnerability are being discovered by attackers. Also vital: Business context — which systems are attractive to attackers;, what kinds of data is present, and what access could an exploit provide.
"Unfortunately, if you only look at base score — a score that hasn’t been enriched with context — then you’re never going to get good predictions."
—Jeff Williams
The researcher interpreted the results of the study as highlighting the need for a defense in depth approach to cyber risk mitigation. Though occasionally the EPSS correctly identified high risk vulnerabilities prior to inclusion in CISA's KEV, organizations cannot rely solely on it as a risk mitigation approach, she noted.
John Bambenek, president at Bambenek Consulting, said organizations use all three risk assessment frameworks — CVSS, EPSS and KEV — to inform software risk mitigation decisions. But the best intelligence is still going to be whatever attacks and malicious activity you have seen on your own networks and what attackers have targeted. CVSS was never designed to tell you what an attacker will do and KEV only has data on attacks and vulnerabilities that are relevant to CISA's core federal government agency audience, Bambenek said.
EPSS is a good attempt to get to something that is more predictive but trying to predict the future is always difficult, Bambenek said.
"Using AI/ML for cybersecurity is an inherently risk thing because attackers already have decades of experience in fooling automated systems and their entire mode of operations is doing unpredictable things. Using retrospective models for future prediction breaks down when you are trying to model behavior of those who don’t have to worry about technical debt or reverse compatibility."
—John Bambenek
Though the Purdue study showed EPSS is less effective than assumed, no one is suggesting that organizations stop using it, or any of the other vulnerability risk assessment mechanisms like CVSS and KEV, for that matter. Security experts perceive each framework as providing at least some help in guiding organizations to vulnerabilities that present the highest risk and therefore need priority attention. But the key is to keep expectations in check and knowing that a focus on vulnerability patching alone is risky.
Inherent limitations with ML
Casey Ellis, founder at the crowdsourced cybersecurity firm Bugcrowd, said ML-driven predictions have inherent limitations, making them effective at ranking known threats but not so much at predicting unknown exploits. "To refine these models, integrate broader datasets, consider advanced analytics such as large language models for deeper context, and regularly validate outcomes against real-world incidents," Ellis said.
SOCs and incident response teams should instead prioritize known exploited vulnerabilities and flaws listed on CISAs KEV catalog as active threats, Ellis said. They should also prioritize vulnerabilities based on CVSS and incorporate EPSS for exploitation risk, use EPSS scores to identify vulnerabilities with high likelihood of exploitation, even if not yet listed by KEV, he said.
Beyond that, also look at asset criticality and maintain your own threat model, Eliis said.
"Prioritize based on the importance and exposure of affected assets. Regularly reassess and adjust your prioritization strategy as new threats and data emerge."
—Casey Ellis
Why your AppSec must shift beyond vulnerability mitigation
Nearly a quarter of those CVEs were exploited on or before their public disclosure, meaning organizations had little or no opportunity to patch the vulnerabilities before attackers began exploiting them. Making matters worse, last year saw a 40% increase in total vulnerability disclosures, to 39,000, up from 28,000 in 2023, a new report from GuidePoint Security disclosed. That's an average of 378 vulnerability disclosures every day.
These reports — and the rise of artificial intelligence-derived coding and software supply chain attacks — demonstrate that the time has come for organizations to look beyond just vulnerability mitigation when it comes to shoring up their AppSec — and take steps to develop a modern approach.
Article Link: EPSS is not foolproof: Shift your AppSec beyond vulnerabilities