The NVD’s problems deepen

This morning, one of the members of the OWASP SBOM Forum sent around an update to the group on how the National Vulnerability Database (NVD) is doing in their quest to reduce their current huge backlog of “unenriched” vulnerability records – namely, new CVE Records that don’t have any CPE identifier attached to them. Not having an attached CPE means that searching the NVD for a particular product will never identify any of those CVEs, even though the product might be vulnerable to one or more of them.

The only way to know for sure whether any of those CVEs affect that product is to manually search through the text of every unenriched CVE report. How many are there? I pointed out at the beginning of October that currently there’s a backlog of over 18,000 unenriched CVE records, which is over 2/3 of the new CVEs identified this year. Moreover, that backlog continues to grow.

Did the SBOM Forum member have progress to report? That depends on what you call “progress”. He had been hoping that on October 1, the first day of the federal government’s fiscal year, the NVD would begin a concerted effort actually to reduce their backlog. Alas, that was not to be. He reported:

Starting October 1^st…CPE assignments have fallen off significantly as a percentage of new CVE assignments. Essentially, the backlog has increased by around 1,000 since the week of September 23rd…I was hoping NVD’s CPE assignment was going to essentially catch up. I was optimistic in September, but that is no longer the case. 

Thus, the NVD now faces a backlog of over 19,000 unenriched CVE records (after promising last May to reduce the backlog to zero by September 30). Will they ever turn this situation around? I have no idea, but I do know that it’s foolish to sustain ourselves in the hope that they’ll be able to do this. They have disappointed us at every step of the way, since their problems (still never officially explained) started on February 12.

Given that no vulnerability search on the NVD will yield accurate results unless you only care about vulnerabilities that were identified before 2024, what alternative vulnerability databases are there? There are several databases that are based on the NVD, which have conducted their own enrichment of some of the unenriched CVEs – i.e., they have created their own CPEs and added them to the CVE records in their database. However, it’s important to keep in mind that the CPE identifier (which is only used by the NVD and its derivatives) has a lot of problems; there’s no such thing as a “definitive” CPE, so a CPE created for one database won’t necessarily match one created for another (of course, this in itself shouldn’t be a big problem if you confine all of your searches to one of those databases).

If your primary concern is vulnerabilities in open source software, you’re in luck, since there are multiple good vulnerability databases to choose from for open source (including OSV, OSS Index, GitHub Security Advisories, and others).

What makes the open source vulnerability databases so good? They are literally all based on the purl identifier. Purl is highly reliable and most importantly doesn’t require lookup to a centralized list of identifiers, as CPE does. In other words, every open source product that is available in a package manager (the majority of those products) has an intrinsic purl that can be created by a user on their own, as long as they know the name of the package manager they downloaded the product from and its name and version number in the package manager. Since vulnerabilities are reported using the same purl, the user should always (barring error, of course) be able to learn of all vulnerabilities that apply to the product in a database search.

Most importantly, purls don’t need to be created by anybody, so the NVD’s current problems with not being able to create as many CPEs as are needed would never have happened if the NVD were based on purl.

However, there is currently a big downside to purl: It can only be used to identify open source software, not proprietary software; CPE can be used to identify both. Moreover, currently there is no alterative identifier for proprietary software, other than CPE. Thus, given that the NVD has fallen on hard times, it can truthfully be said that today there is no trustworthy way to conduct an automated search for vulnerabilities in proprietary software. Of course, for most organizations, automated searches are essential to an effective vulnerability management program.

This raises the question, “Can purl be somehow extended to cover proprietary software, and will it be almost as dependable in that domain as it is for open source software?” The answers to those two questions are yes and yes. Then the question arises, “What needs to be done to make this happen?”

I’m glad you asked. The OWASP SBOM Forum has identified two likely paths by which purl can be expanded to cover proprietary software. We want to flesh out the details of both of those paths and test them in a proof of concept. We have scoped out a project to do that and are looking both for volunteers to contribute to the project and modest financial support to make it happen (support can be in the form of donations to OWASP which are directed to the SBOM Forum. OWASP is a 501(c)(3) nonprofit organization).

You can read about the project here. Please email me if you would like to discuss this more.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.