Saturday, March 22, 2025

The NVD seems to be on its way out. What do we do now?


As discussed in this recent blog post, software vulnerability management is facing a serious problem: The National Vulnerability Database (NVD) seems to be giving up one of its two primary responsibilities (besides its responsibility to keep the database itself operating): adding “CPE names” to new CVE records.

There are two problems that need to be addressed:

First problem

·       Software users need to be able to learn about vulnerabilities that have been reported in the software they use. They do this by searching a software vulnerability database.

·       The National Vulnerability Database (NVD) is by far the most widely used vulnerability database in the world. However, just learning about a new software vulnerability does not help a user, unless they know what product or products are affected by the vulnerability.

·       In the NVD, vulnerabilities are identified in CVE records using a format like “CVE-2024-12345”. Products that are affected by a CVE should be identified in the CVE record, using a machine readable “CPE name” like “cpe:2.3:a:microsoft:internet_explorer:8.0.6001:beta:*:*:*:*:*:* ”.  

·       Currently, NVD contractors are responsible for adding CPE names to new CVE records. Before February 2024, this was almost always done within a few days of when the NVD received the CVE record.

·       If a CVE record does not contain a CPE name for a software product affected by the vulnerability described in the text of the record, a user searching for vulnerabilities that have been identified in the product will not learn it is affected by that CVE.

·       The NVD’s problem is that, starting on February 12, 2024, the number of CPE names it created dropped drastically. As a result, about 55% of new CVE records in 2024 were never given a CPE name. This means the product(s) affected by that CVE are invisible to a search.

·       Brian Martin is a well-known figure in the vulnerability world; Brian wrote on LinkedIn recently that it appeared the NVD had only added CPE names to a small percentage of new CVE records in the last 12 days. More than one week later, this situation has not substantially changed.

·       The same week, Bruce Lowenthal, Senior Director of Product Security at Oracle, reported that so far in 2025, only about 25% of new CVE records added in January and February contained a CPE name.

·       This means that, much more than half the time, a search in the NVD using a product name will yield no CVE records since February 2024, even if there has been at least one vulnerability reported for the product since then.

·       Even worse, the user will probably believe the product is vulnerability-free, since the only message they receive says that no records were found.

·       This further means that searching for vulnerabilities in the NVD is increasingly becoming a waste of time; in fact, it will probably give the user a false sense of security, since no vulnerabilities appear in most product searches.

·       What other vulnerability databases are there, besides the NVD? For open source software products, the answer is “a lot”. These include OSS IndexOSVGitHub Security Advisories and others; in fact, a software user is more likely to learn about vulnerabilities that apply to open source software products (or open source components in an SBOM) in these databases than in the NVD.

·       However, for commercial software products, there is currently only one vulnerability database: the NVD. Because the NVD can no longer be called reliable, this means there is currently no reliable source of vulnerability data for commercial software products. Obviously, this isn’t a good thing, given how dependent business and government organizations are on commercial software.

·       Bruce pointed out that one serious effect of the lack of CPE names – or at least the long delay in creating them – is that Oracle (and surely many other software suppliers) – is often not able to provide patches as quickly as their customers want them (normally within 2-3 weeks).

·       When will this problem be fixed? If this question asks when CPE names will be added to all the over 30,000 “CPE-less” CVE records currently in the backlog, the answer is “probably never”. It is possible that the rate of growth in the backlog will be slowed. However, it is unlikely that the backlog itself will be erased, in the sense that every CVE record will include a CPE name.

·       Since the CPE backlog may never go away, what measures can be taken in the longer term? There isn’t much question: CVE records need optionally to contain a different identifier than CPE, although CPE will always be an option. That identifier needs to be purl.[i]

·       If purl were implemented in the CVE record format, it would immediately improve identification of open source products, since purl has – in about eight years – become the primary software identifier used in open source vulnerability databases.

·       The most important feature of purl is that the user never has to look up the purl for a product before they search for the product in a vulnerability database. This is because they should always be able to create the purl for a product by using information they already have. This includes the package manager from which they downloaded the software, as well as the package name and version string in that package manager.

·       Since each purl is globally unique, a purl for an affected product in a CVE record should always match a purl created by a software user before they search for the product in a vulnerability database, meaning searches using purl will have a high success rate.

How can we address the first problem?[ii]

Introducing purl into the CVE ecosystem requires making it possible for CVE Numbering Authorities (CNAs) to designate software products in CVE records using purls. CNAs are mostly larger software developers and organizations like GitHub; they are responsible for reporting vulnerabilities to CVE.org using the CVE Record Format.

Three tasks are required to address the first problem. In each of these tasks, we will coordinate with the CVE.org Quality Working Group.

A.      Develop a new version of the CVE Record Format to accommodate use of purl and submit it as a pull request to CVE.org. The SBOM Forum will work with the CVE.org Quality Working Group (QWG) and the Python Foundation to accomplish this goal.

B.      Develop plans for an end-to-end proof of concept for use of purl in the CVE ecosystem.

C.     Conduct that proof of concept. The PoC will involve software suppliers, end user organizations, CVE Numbering Authorities (CNAs) and vulnerability database operators.

Second problem

·       Today, purl can only be used to identify open source software in package managers, not commercial software. Since most private and government organizations utilize commercial software to run their businesses, it is important that purl be expanded to identify commercial, as well as open source, software products. In 2022, the OWASP SBOM Forum suggested[iii] a way to fix this problem by having a supplier create a “SWID tag” for each of their products. A new purl “type” called SWID was developed and implemented in purl.

·       A SWID tag is a small document containing 5-10 pieces of metadata about a software product. These pieces of information can be used to create the purl for the product, which will always be globally unique.

·       The only three mandatory fields for a purl using the SWID type are “name”, “version” and “tagId”. Note that “tagId” can be almost anything. For example, it could be the URL from which the product can be downloaded.

·       To illustrate this, the purl for the product named Fedora, version 29, is “pkg:swid/Fedora@29?tag_id=org.fedoraproject.Fedora-29”. Note that every purl starts with “pkg:” followed by the type. For open source software, the type usually indicates the package manager or other repository – for example, “NPM” for Node NPM packages and “maven” for Maven JARs and related artifacts. However, for commercial software, the type will normally be SWID.

·       The supplier will make both the SWID tags and the purls for their products available on their website or by other means. If a user wants to look up a product in a vulnerability database, they can download the purl for it, if that is available; otherwise, they can download the SWID tag and use that to create the purl (of course, various tools will automate this process). Neither the purl nor the SWID tag will need to change until the product is upgraded to a new version.

·       As in the case of purls for open source software products, the purl included in the CVE record for a commercial product should always match the purl a user creates when they want to search for that product; this is because both purls will be based on the contents of the same SWID tag. 

How can we address the second problem?[iv]

We can address the second problem with three tasks. We will work with the purl maintainers in accomplishing all three tasks.

D.     Develop “rules of the road” for production, distribution, and use of SWID tags to allow purl to identify commercial software.

E.       Test those rules in a small-scale proof of concept. In that PoC,

                                        i.               A supplier will create SWID tags (perhaps using this tool) for certain products and make them available to their customers;

                                      ii.               CNAs will create test CVE records containing those purls to report test “vulnerabilities” in their products[v];

                                   iii.               One or more vulnerability databases (that support both CVE and purl) will ingest the test CVE records; and

                                   iv.               End users will utilize purls created from the SWID tags to search the vulnerability databases. If all the CVEs that were recorded for a product are revealed when the user searches using the product’s purl, the PoC is successful.

F.       Develop educational webinars and videos for CNAs and other participants in the CVE ecosystem, on use of purl in the CVE Record Format.

Conclusion

Given that the NVD may have stopped performing their most important function – adding CPE names to CVE records – for at least two weeks, and because of what is going on in the federal government today, it is possible that the NVD may no longer exist at all soon. Fortunately, CVE.org (which is part of DHS, not the Department of Commerce like NIST/NVD) seems to be taking steps to replace the functions that the NVD was performing (many people have been saying for a while that the NVD should be consolidated into the CVE.org database, since the CVE records in the latter form the foundation of the NVD anyway).

No matter what happens in the near term, it is clear there needs to be an alternative software identifier besides CPE available to CNAs and software end users. While there are one or two experimental alternatives (such as OmniBOR), purl is already in heavy use. For example, the open source software composition analysis (SCA) tool Dependency Track alone is used over 20 million times every day to look up a dependency from an SBOM in the OSS Index vulnerability database, which is based on purl.

Purl’s availability in the CVE record format will quickly make identification of open source software much easier and more accurate in the NVD and other vulnerability databases based on CVE. And, when the policies and procedures for use of the SWID purl type have been worked out and tested in a proof of concept, identification of commercial software products in the same databases will be much easier (no CPE lookup required!), as well as much more accurate.

Of course, it will probably be 1-2 years before purl is in widespread use in the CVE ecosystem. But there’s no excuse for waiting any longer; two years in the future will still be two years in the future six months from now. The six tasks listed above are mostly non-technical; they mainly require getting agreement among a group of participants in the CVE ecosystem.

The OWASP SBOM Forum will be pleased to lead this effort; we will start out with an initial project to perform the first two Part 1 tasks: development of high level plans and identification of changes to the CVE record format that are required to accommodate purl. 

No comments:

Post a Comment