It is now clear that, starting on February 12, 2024, the National Vulnerability Database (NVD) staff fell far behind in performing its most important function: creating machine-readable identifiers (referred to as “CPE names”) for vulnerable software described in text form in CVE reports. The NVD currently has a huge backlog of 17,000 “unenriched” CVE reports which continues to grow by about 100 reports per day; the NVD has yet to acknowledge the size of the backlog, let alone articulate a plan to eliminate it.
This poses a serious problem for the entire software security community, including both developers and consumers of software: Without a machine-readable software product identifier included in a CVE report (which means it also appears in the NVD), it is impossible for an automated process to learn about vulnerabilities that apply to a software product. This means an automated search for a software product in the NVD today will take account of only a small fraction of the CVEs that were identified since February, when the volume of CPE names created by NIST/NVD staff members dropped close to zero. The volume is now about 75-80 per day, but even that doesn't keep the backlog from growing, let alone start to reduce the backlog itself.
This wouldn’t be a serious problem if there were a clear alternative
to the NVD, to which current NVD users could switch without a lot of disruption
to their current processes. Unfortunately, not only is there no such
alternative now, but it is hard to identify any other database that could be
quickly re-purposed to provide that alternative. This includes the CVE.org
database, which is the source of the CVE reports passed to the NVD. CVE.org contains
all the data currently in the NVD. However, that means it only contains CPE
names that were added by NVD staff members or by the CVE Naming Authorities
(CNAs) themselves (as well as perhaps a small number of CPE names created by
CISA’s new Vulnrichment program).
At this point, CVE.org has no plans to take over the NVD’s
task of adding CPE names to CVE reports – and the CNAs themselves do not seem
to be inclined to do this, even though they have been encouraged to do so. Given
that eliminating the current backlog would require a big effort, for which CVE.org
doesn’t have funding, it is understandable that they don’t want to undertake this
job. Moreover, given the many problems with CPE
names, it is certainly understandable that professionals in the
vulnerability space aren’t inclined to devote much time to keeping CPE going; I
know I am not inclined to do that.
From a technical point of view, what can be done to fix this
problem? There is one technical step that the software security community can
take, although it will require 2-3 years to implement. It would alleviate many
of the above problems, although certainly not “solve” them. That step is to
start including purl identifiers in CVE reports (this is now permitted by the
CVE 5.1 spec). Purl is a much more reliable software identifier than CPE,
mainly because it doesn’t require lookup to a central database like CPE does.
For further information on why purl is superior to CPE, see pages 7-11 of the
OWASP SBOM Forum’s 2022 white
paper on software vulnerability identification in
the NVD.
However, it will take 2-3 years (and perhaps longer than that) before it will be possible for CVE Numbering Authorities (CNAs) to be comfortable with adding purls to CVE reports, and for vulnerability database users to have the tools and knowledge necessary to make effective use of purls. The work of putting in place what is needed to make purl a complete replacement for CPE (including working out a mechanism by which purls can be created for commercial software products) needs to start as soon as possible. The OWASP Vulnerability Database Working Group is ready to begin that work, when and if funds become available.
However, users and developers of software can’t be required to wait 2-3 months to learn about today’s vulnerabilities, let alone the 2-3 years it will take for purl to be fully implemented in CVE reports. We seem to be stuck with this problem: There’s no automated way to find almost any recent CVE (i.e., any CVE identified after early February) that applies to a software product we’re concerned about, since there is no way for an automated process to link the product name to the CVE. In other words, fully automated vulnerability management is no longer possible using the NVD. Given that the NVD hasn’t even acknowledged the size of the backlog it’s facing, it’s highly unlikely it will dig itself out of this hole for more than a year.
However, there is also good news: There are many other
sources of vulnerability data besides the NVD. Most of them have a much
narrower focus than does the NVD; on the other hand, that narrower focus
enables a much higher degree of specialization – meaning they usually do what they do
much better than the NVD does (or did). Software users and software developers that have been
relying on the NVD for all their vulnerability data now need to combine inputs
from multiple vulnerability data sources, which means some alteration of their
current processes. But they will end up with higher-quality data. Most
importantly, they will have current data, which isn’t the case if they
only use the NVD now.
Comparing data sources
Of course, finding the best – or at least a good –
combination of other vulnerability data sources to replace the NVD isn’t easy[i],
since pure “apples to apples” comparisons between sources are often impossible.
The reasons for this include the following:
1.
As previously noted, almost all vulnerability
databases utilize one or the other of the two primary software identifiers: CPE
and purl. No single vulnerability database supports both identifiers, and it is
almost impossible to create a one-to-one map between a CPE name and a purl or
vice versa. Many software users (including developers) will need to use at
least two databases, one based on CPE (which will capture commercial software
and some intelligent device vulnerabilities) and the other based on purl
(for open source software - or OSS -
vulnerabilities. Purl has always been a much more widely used identifier for OSS
than CPE).
2.
Just like software products, vulnerabilities
also need machine-readable identifiers. Even though CVE is by far the most
widely used identifier for vulnerabilities, there are other types of
identifiers as well, including those used in the CERT/CC, ICS-CERT, GitHub
Security Advisories (GHSA) and OSV vulnerability databases. While these other
vulnerability types are sometimes mapped to a CVE identifier, that is not usually
the case. Since the NVD only includes vulnerabilities identified with a CVE
number, any NVD user who is not searching these other databases will never
learn of those vulnerabilities.
3.
Some vulnerability databases (like the NVD) are
free of charge. Others are behind a paywall. Yet other databases provide some
services for free and other services for charge. Moreover, different vulnerability
data providers bill their services in different ways. For example, one provider
might charge based on a certain number of searches, another might charge a
monthly fee, while yet another might charge just for access to a certain type
of “premium” data.
4.
The amount of other information provided with the
vulnerability information (often called metadata) varies widely, including
information on exploits and exploit proofs of concept, presence on the CISA KEV
(Key Exploitable Vulnerabilities) list, EPSS score, threat intelligence,
indicators of compromise, CVSS component detail, review of vulnerability
mentions in blogs, etc. Different software users will want to see different
types of metadata.
It would be nice if there were already some database that contained
up-to-date information on all the various vulnerability data providers in the
world, based on standardized criteria that make comparisons easy (think about
online car shopping). However, that is simply not the case. There are hundreds
(and perhaps thousands) of vulnerability data sources today, but there isn’t
even a simple listing of them, let alone a means to (fairly) easily perform comparisons
among them.
This is why the Vulnerability Database Working Group of the
OWASP SBOM Forum is proposing a project to create an online “catalog” of
vulnerability data providers, which truthfully describes what each provider can
offer to users and discusses how users can make the best use of their
offerings.
Categories of vulnerability data providers
First, we need to distinguish the primary categories of vulnerability
data providers, as well as the data and related services that each can provide:
1.
Vulnerability databases based on CPE.
These include the NVD and CVE.org, although CVE.org, being the source of
CVE information, is a lot more than just a vulnerability database. While these
databases include vulnerability data on both proprietary and open source
software, they are best known for their coverage of proprietary (or “closed
source”) software vulnerabilities.
2.
Vulnerability databases that are based on the
NVD (and CPE) but enhance the content in various ways – including creating their own CPEs for some
of the CVEs that are now consigned to the NVD’s backlog. These include VulnCheck, Vulners,
and VulDB.
3.
Vulnerability databases based on purl and
focused on open source software products and components. These include OSS Index, OSV, Snyk, Rapid7, and others.
4.
Smaller databases focused on particular
languages or development environments, including GitHub Advisory Database, Python Packaging Advisory
Database, RustSec Advisory
Database, and the Cloud Security Alliance’s Global Security
Database. Vulnerabilities found in these four databases are all found in OSV.dev,
in a normalized vulnerability format that allows single searches across all
four databases. The complete list of vulnerability databases currently
searchable in OSV can be found here.
There are many other providers (mostly smaller) in this category, besides the
ones included in OSV.
5.
Service providers whose purpose is to “enrich”
the CVEs included in the NVD. This includes CISA’s Vulnrichment
program.
6.
Service providers that provide platforms or services
in which vulnerability identification plays a big role, even if it isn’t the
primary purpose of the platform or service – for example, “platforms” for
creating and analyzing software bills of materials (SBOMs). There are many of
these service providers.
7.
There are likely other categories of
vulnerability data providers, beyond the six categories listed above.
Our “proposal”
The Vulnerability Database Working Group of the SBOM Forum
proposes to produce a “Vulnerability Data Provider Catalog” of organizations
like those identified above; the catalog will include a separate report
describing the products and services provided by each included provider. This catalog will allow public and private sector organizations that are
“consumers” of software, as well as software developers, to understand their
options for obtaining current vulnerability data for the software they utilize
and/or develop.
Please note:
·
The data and services provided in aggregate by
the above organizations go far beyond what the NVD provided previously, or what
it will ever provide. In other words, it is likely that the Vulnerability Data
Provider Catalog will awaken many organizations to possibilities for
vulnerability management they were never even aware of when they were exclusively
using the NVD to learn about vulnerabilities in their software.
·
The goal of this project is not to produce the
equivalent of a huge spreadsheet with hundreds of columns, which compares the
offerings of a hundred or more vulnerability data and service providers.
Rather, it is to produce a set of short qualitative descriptions of individual
vulnerability data providers, which are focused on what data and/or services
this provider offers. These include discussions of why particular vulnerability
data offerings may be important for some types of users, but not for
others. We assume the users of the catalog will have diverse needs. No vulnerability
data provider, or even combination of providers, meets the needs of most or all
users – and this includes the NVD, even if it gets back on its feet again.
·
Producing each report will require interviews
with the provider’s staff members, viewing demonstrations and marketing
materials, reviewing documentation, etc. It is likely that most reports will
require multiple days to produce.
·
The reports will initially be published on a page linked on the SBOM Forum’s OWASP website, although they may later be published on a separate
site as the number of reports in the catalog increases.
·
Reports will be published as they are created,
so the site will be growing continually.
·
As we interview more vulnerability data
providers and post more reports, it is likely we will learn lessons that can be
applied to reports already posted. With the permission of the affected provider,
we will amend a posted report when necessary.
We will
·
Base our reports primarily on our discussions
with the provider’s staff members, as well as observations we can make
regarding data or services currently provided.
·
Include in our reports the provider’s
expectations for future offerings, while making it clear these are just
projections.
·
Make additions or changes to a provider's report
whenever the provider requests them, if these meet our criteria for verifiability
(see below).
We will not
· Endorse providers or choose a “Best” provider in a
category (or overall).
·
Repeat provider statements that we suspect are
not accurate, unless the provider can verify their accuracy.
·
Reproduce statements made by third parties about
a vulnerability data provider, unless we have been able to verify those
statements.
Donations
Because of the amount of time required to gather information
on each provider and develop a report (expected to be at least two days for
many providers, with large providers requiring more time than that), this
project will not be possible without donations.
You may donate in your organization’s name or as an
individual. We are not aiming for large donations (although they won’t be
turned down!), but rather to have a large number of small donations, each from
$1,000 to $5,000. All donations will come through OWASP, unless the donor cannot
do that for some reason – in which case we will identify an alternative channel
for receiving the donation. OWASP retains ten percent of the donation amount
for administrative costs. We believe this is reasonable, given the amount of
work they save us. OWASP is a 501(c)(3) nonprofit organization; in many cases,
donations will be tax-deductible, although that should always be verified, if
it is a primary concern.
OWASP does not directly pass donations to the project
designated by the donor – in this case, the SBOM Forum. Instead, donations are
retained and paid to the project upon submission of valid invoices for project
expenses, which follow the Expense
Reimbursement Policy in the OWASP Rules of Procedure. This policy is
intended to give donors confidence that their donations are being well spent.
Donations need to be restricted to the SBOM Forum; this can
be done for any donation over $1,000 (in other words, the SBOM Forum will not
receive anything from a donation to OWASP that is under $1,000, although the
donation will still support OWASP itself). Currently, all donations to the SBOM
Forum will be allocated to the Vulnerability Data Provider Catalog project,
unless the donor requests otherwise in an email to Tom Alrich.
To donate with a credit card or other online payment method,
go to the OWASP SBOM Forum
page and click Donate in the top right. On the next page, click on Other and
enter $1,000 or more. Then enter your email address (twice) and your name
(which doesn’t have to match the name on your credit card). Finally, check the
box to restrict the gift to the SBOM Forum.
If you have any questions, feel free to email Tom Alrich
first at tom@tomalrich.com. If you don’t
want to donate through the online method, OWASP can take donations directly as
well.
Even if you have already donated, please email Tom to let him know about this, so he can make sure you’re properly recognized for your donation. If you approve, your organization’s logo (or your name, if you are donating as an individual) will be listed under the “Supporters” tab on our website.
Are you a vulnerability data provider?
If so, you probably realize that it's hard to communicate exactly what you do to end users, and especially to communicate why they ought to look to you to provide more than what they previously received from the NVD. When the only way for a user to learn about their options is to try to compare a number of wildly different provider websites, all providers are at a disadvantage.
If surfing lots of websites is their only option, a lot of users will not even try to replace the NVD; in fact, this is probably the case for a large percentage of users (and perhaps most of them) today. The OWASP Vulnerability Database Working Group is concerned about the fact that most end user organizations, and even software developers, don’t currently have a good way to learn about their options for replacing the NVD (and in the process finding higher quality vulnerability data).
“Audience”
Who will be the “audience” for the Vulnerability Data Provider
Catalog? There are three broad categories of organizations:
1.
Public and private sector organizations that are
concerned about the security of the software they use to accomplish their
mission;
2.
Software developers concerned about
vulnerabilities in the products they’re developing; and
3. Vulnerability data providers that understand the importance of having a trusted third party like OWASP provide standardized, verifiable information on their products and services, as well as those of other providers, through a single web location.
In other words, any organization concerned about software vulnerabilities should consider joining and/or donating to the OWSAP Vulnerability Database Working Group. Please contact Tom Alrich at tom@tomalrich.com.
[i] Of course, there will be no “best” solution for all users, or even all users of a particular type. Each organization needs to figure out the best solution for itself, although the Vulnerability Database Working Group will be glad to discuss these questions with you (and you are welcome to join the group's bi-monthly meetings – email Tom Alrich to do so).
No comments:
Post a Comment