It’s time to move beyond the NVD

I learned about a week ago that, starting around the middle of February, the National Vulnerability Database (NVD), the most widely used vulnerability database in the world, seems to have stopped adding CPE names to the great majority of CVE records. CPE (common platform enumeration) is the only type of identifier for software and devices used by the NVD, and CVE of course refers to a vulnerability documented by CVE.org (formerly just referred to as “MITRE”).

What does this mean? Here’s how vulnerability data reaches the NVD normally:

1.      CVE.org (part of the Department of Homeland Security. It is funded by CISA and governed by a board that includes people from private industry and government) has approved over 300 CNAs, which stands for CVE Numbering Authority.

2.      When a software or device manufacturer identifies a new vulnerability in one of their products, they work with a CNA to create a CVE report. The report includes a CVE number (assigned by the CNA), a description of the vulnerability and an identification of the affected product (e.g., “Product X Version Y, produced by Developer/Manufacturer Z”).

3.      The contents of this report are added to the CVE.org database.

4.      The report is then passed on to the NVD. The NVD is part of NIST, which is in turn part of the Department of Commerce.

5.      A NIST employee that is part of the NVD team (which only has about 20 people, believe it or not) enhances the CVE report in a few ways, most crucially by assigning a CPE name(s) to the product(s) identified in the report.

6.      The CPE name is required for the new CVE to be linked to a product/version listed in the NVD. Unless there is a CPE name on the report, the new CVE will never appear in an NVD search – except when the user searches on the CVE itself (in which case they’ll just see what is already in the report).

And this is the problem: Even though a new vulnerability has been duly reported to CVE.org and the report includes a description of one or more products that are affected by the vulnerability, an NVD user will normally never learn about that vulnerability, unless a CPE name is assigned to every product/version in the report. This is because the great majority of NVD searches are intended to find vulnerabilities that apply to a product/version. If there is no CPE name for the product/version they are searching for, the user will never learn of any CVE that applies to it, even if one has been reported in a CVE report.

I believe (although I may be wrong about this) that the only way this user could learn about a “CPE-less” CVE report that applies to the product/version they are interested in is to do a string search for “Product XYZ Version 4.7” in the CVE.org database. Any CVE whose report includes that exact string will appear (but, for example, if the string in the report reads “Product XYZ, v4.70”, it will probably never show up in the search). The chance of success in this search is slim, and in any case that search can’t be automated. So, doing this for 1,000 product/versions is out of the question.

Therefore, the results of almost all NVD searches for vulnerabilities, that apply to any software or device product/version, are now suspect and need to be taken with a shakerful of salt. This is because there is no way to know whether there are new (since Feb. 15) vulnerabilities that apply to that product/version, but which aren’t linked to a CPE name for the product/version. Given the importance of the NVD in almost all vulnerability management activities today (including using SBOM and VEX for component vulnerability management purposes), this is as close to a catastrophe for software vulnerability management as anything I’ve seen.

If this situation continues, the software security community can’t count on the NVD to always have recent vulnerability information; any discussion of the results of an NVD search will need to have an asterisk pointing to “Does not include most CVEs identified after February 15, 2024.” The need for this will continue until the NVD starts adding CPE names to CVE reports again, and retroactively adds them to all reports that don’t already contain them. In other words, not only is the NVD in a hole now, but they’re enlarging the hole as fast as they can.

Assuming this continues, a lot of (perhaps most) processes and procedures having to do with vulnerability management can no longer be trusted to be comprehensive. This includes most vulnerability scans, since the scanner will usually look in the NVD for vulnerabilities that have been identified for the product/version being scanned. It also includes monthly patch releases, since the software developer can no longer be certain that the release any significant vulnerabilities that were identified after Feb. 15. It may be that some of these processes will need to be abandoned altogether, since getting an admittedly incomplete list of software vulnerabilities in a product/version might be considered in many use cases to be a waste of time.

When I heard about this last week, I thought it might be just a short-term glitch; so I put the problem out of my mind. However, this week the problem has become widely discussed on LinkedIn, in posts by Dan Lorenc of Chainguard (which has drawn a huge number of comments) Chris Hughes, and the company NetRise (I have commented on all three of these). The tenor of these posts, and of the comments, is something to the effect of “Holy s___. What do we do now?”

You may be wondering what the NVD is saying about this problem. As far as I know, the only notice they have put out on the problem is this one, which has been on the site for at least a week or two:

NIST is currently working to establish a consortium to address challenges in the NVD program and develop improved tools and methods. You will temporarily see delays in analysis efforts during this transition. We apologize for the inconvenience and ask for your patience as we work to improve the NVD program.

At first, this might seem comforting. After all, at least the notice shows that NIST understands the problem and is working on solving it. The first step in the solution seems to be establishing a “consortium” that will develop “improved tools and methods”. It’s unfortunate that there will be a temporary degradation of service – but at least there’s light at the end of the tunnel. Or so the announcement would have you believe.

Not so fast. I have a story to tell you:

Once upon a time (in the fall of 2022, to be exact), the OWASP SBOM Forum - at that time just called the SBOM Forum - published a white paper on how to ameliorate the naming problem in the NVD. While the paper got a lot of recognition, including an initial enthusiastic reception by a key person in CISA (as well as people in MITRE), it became clear to us in early 2023 that CISA had no intent of pursuing these ideas any further.

Since we considered this to be a serious problem, we reached out directly to the NVD (through one of our members who is on the board of CVE.org, which also includes NIST people) and asked Tanya Brewer, who leads the NVD, if she could meet with us. She was not only willing but pleased to do this. She said they had read our paper carefully and would like to discuss it with us.

We had three meetings with her (and several other NVD staff members). These three posts - first, second, and third – described our conversations with Tanya, all in the first half of 2023. In the third post, I relayed the news that NIST would soon come out with a video asking for suggestions on a public-private partnership (which she was calling a “consortium”) to fix the problems (as well as accomplish other goals, to be sure). After receiving and digesting the comments, NIST would probably post a Federal Register notice about their plans by late August 2023. Tanya thought the Consortium could be up and running by early 2024.

We heard from other people in government that this goal was probably overly ambitious, given the significant amount of red tape involved with forming a public-private partnership of any kind with a federal agency. However, we were impressed by her sincere desire to put this in place.

She said this to us in late June 2023. Since then, we have had some communication with her about technical issues with the NVD feeds – which she has been very proactive about addressing, along with appropriate staff members. However, we haven’t heard anything more about the consortium, and I assumed it was dead, until reading this notice.

But there was more than that. Not only did I assume the consortium was dead, but I wrote off the idea of working with the NVD to fix their problems. I was looking at the timeline below, which was implied by our discussions with Tanya. This timeline is almost certainly optimistic, yet even then, it’s appalling:

1.      The NVD announces the consortium in videos (which Tanya said would be her first step), then takes comments on those videos. They then analyze those comments and use the results to write a Federal Register notice about the Consortium (which in itself takes a month or two to be posted). This process is probably six months.

2.      Organizations interested in participating in the consortium have a few months to submit an application. The applications are evaluated; then each organization that is accepted will have to sign a Cooperative Research and Development Agreement (CRADA) with NIST. This will take six months at a minimum.

3.      It’s only at that point that discussions can even begin about a) what problems need to be addressed in the NVD and b) what needs to be done to address them. Let’s say these discussions take one year.

Let’s assume that the Consortium reads the OWASP SBOM Forum’s white paper, falls in love with it, and decides to implement it lock, stock and barrel. How long will it take to do that? I won’t discuss everything we proposed in the white paper, but let’s look at the most important part of our proposal: that purl identifiers be incorporated into the NVD. What would be the timeline for implementing that part alone?

1.      To implement purl in the NVD, purl identifiers have to be included in CVE reports. That means the CVE JSON spec needs to be changed to accommodate them. I believe CVE reports are now being produced in the CVE JSON 5.0 spec, but last year I heard the NVD was struggling to fully support that, so I’m not sure 5.0 is even supported yet.

2.      Two years ago, the SBOM Forum submitted a pull request to the group that maintains the CVE JSON format, to add purl identifiers to the format. The PR was accepted and will be designated v5.1. However, the format isn’t being used yet, since the NVD can’t accommodate it now (and if the NVD hasn’t fully implemented the 5.0 spec yet, they have to finish that before they can move on to 5.1). When will the NVD be able to do that? Given what’s happened in the last month, the answer is probably “never”. But let’s be charitable and say it will be one more year before CVE reports will be produced using the 5.1 format, and before they’ll be ingested by the NVD. That’s also wildly optimistic.

3.      Even when it’s possible to use purl identifiers for NVD searches due to the 5.1 spec being accepted by the NVD, that doesn’t mean the CNAs and the NVD staff will understand purl. Purl is a very different identifier from CPE (see the discussion of intrinsic identifiers like purl vs. extrinsic identifiers like CPE in the first part of the SBOM Forum’s white paper). There will have to be training on this, as well as a big ramp-up period. Let’s assume that full use of purls takes another year to accomplish.

Thus, to get to the point where at least a partial solution to the NVD’s problems is in place, it will take at least four years, and probably a good deal more than that (as evidenced by the fact that the NVD is already nine months behind the schedule Tanya outlined to the SBOM Forum last June). And this assumes the whole process starts today, which of course isn’t realistic.

Even worse, I strongly doubt that, even after going through all the above steps, the result will be worth the wait. Hanging over this are two things that I realized from our discussions with Tanya last year:

First, the NVD has a huge technical debt to pay off. Tanya seemed most interested in having Consortium members provide warm bodies (presumably with intelligent brains attached) to help them with making changes to their infrastructure. She said they want any such warm body for at least a six month commitment, mainly because they are going to have to learn a lot of arcane stuff about the NVD (it’s 2-3 decades old).

This includes learning some old language that seems to be fundamental to the NVD, that I’d never heard of (and I know something about old languages. The first program I ever wrote was in Fortran IV with the WATFOR compiler, for a mainframe at the University of Chicago that took up an entire building and probably had a tiny fraction of the memory on my cell phone. The program had to be typed on punch cards and submitted at a window in the building. Then, you would go back to your dorm and come back later that night to retrieve the printout along with the card deck, only to learn that you missed a comma in one of the lines in the program and it “abended”, meaning it abnormally ended. You then had to retype that card and resubmit the deck, until your program finished successfully. This was “interactive computing” in those days).

Second, Tanya is very concerned about strengthening CPE names and making them more widely used, not less. The SBOM Forum paper (in pages 4-6) shows that CPE is hopeless; it will never be fixable, since CPE names are created by human beings with imperfect guidance and - unlike purls - don’t line up with anything in the real world; moreover, that will never change. The SBOM Forum doesn’t want to end all use of CPEs, because there is so much vulnerability information already tied to them. But we also don’t want to strengthen or increase use of CPEs, any more than we want to promote new COBOL applications.

It seems that whoever wrote the NVD notice above had eaten too many magic mushrooms. Do they actually think it’s acceptable to have the vulnerability database the world relies on be down for over four years, with no clear replacement available? Are they thinking at all?

I’ll answer this question for them: No, this is not acceptable. The NVD needs to continue to operate at least in the near term, but it can no longer be accepted as the most important vulnerability database worldwide. There needs to be a short-term solution, a database in which new CVEs will include purl identifiers for open source software, and other identifiers for proprietary (closed source) software, as well as intelligent devices.

But there needs to be a long-term solution as well. It needs to be internationally supported, but it can't depend on government funding (although governments are welcome to participate in funding it). I’ve already proposed one such solution, that might be up and running in less than a year. I’m not saying we must go with that alternative, but I am saying there’s no longer any reason why we need to put up with the NVD’s continual delays and creaky infrastructure.

Before anything can happen, the vulnerability management community needs to discuss the NVD situation and what we can do about it, both in the short and long terms. I know the OWASP SBOM Forum will be discussing this in at least our next 2-3 meetings. If you aren’t a member and would like to join us (the calls are at 1PM ET on Fridays – which I know isn’t great for people in Europe), please drop me an email.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

My book "Introduction to SBOM and VEX" is now available in paperback and Kindle versions! For background on the book and the link to order it, see this post.