The NVD fades away

I didn’t know whether to laugh or cry when I saw the NVD’s most recent announcement (last week) about…what is this about, anyway? Here is what it said:

NIST maintains the National Vulnerability Database (NVD), a repository of information on software and hardware flaws that can compromise computer security. This is a key piece of the nation’s cybersecurity infrastructure.
There is a growing backlog of vulnerabilities submitted to the NVD and requiring analysis. This is based on a variety of factors, including an increase in software and, therefore, vulnerabilities, as well as a change in interagency support.
Currently, we are prioritizing analysis of the most significant vulnerabilities. In addition, we are working with our agency partners to bring on more support for analyzing vulnerabilities and have reassigned additional NIST staff to this task as well.
We are also looking into longer-term solutions to this challenge, including the establishment of a consortium of industry, government, and other stakeholder organizations that can collaborate on research to improve the NVD.
NIST is committed to its continued support and management of the NVD. Currently, we are focused on our immediate plans to address the CVE backlog, but plan to keep the community posted on potential plans for the consortium as they develop.

I ultimately decided that crying was the appropriate response. Because this announcement makes it clear that

1.      The NVD’s staff has no idea what the cause of their current problems is;

2.      They don’t have a plan for finding and fixing the problem; and

3.      They’re not at all concerned about the effect this is having on their remaining supporters (both of them), despite their proud assertion that the NVD is “a key piece of the nation’s cybersecurity infrastructure.”

You’ll notice the words “sorry” and “apologize” are nowhere to be found in this announcement. Don’t you think that the NVD staff should be concerned that this “key piece” they’ve been furnishing for so long is no longer working, and that a huge number of cybersecurity professionals worldwide, who had thought the NVD would always be around, are now officially SOL (that’s a technical acronym)? The absence of those two words with magical properties (at least according to my mother) tells the whole story.

Apologies are certainly due, especially for insulting our intelligence with these ridiculous assertions:

1. The problem is that there is “a growing backlog of vulnerabilities” – i.e., this problem has been gradually building over time. In fact, as Patrick Garrity’s analysis has shown, the NVD was “analyzing” CVE reports at roughly the same rate that new reports were appearing until February 12 of this year. On that day, the number of CVEs “awaiting analysis”, which had been effectively zero so far this year, started trending rapidly upward (the red line in the graph in this post). At the same time, the number of CVEs analyzed almost flatlined.

In other words, on February 12 the NVD’s analysis of CVE reports (which added the CPE names required to determine what product a CVE applies to. A CVE report without a machine-readable identifier for the affected products like CPE is literally like a car without a steering wheel. Yes, it will move, but you would be a fool to expect good results from doing that).

I checked back with Patrick last week and asked if there had been any significant change in the rate of new CVEs being analyzed (which had dropped to less than a tenth of the previous rate in February). Patrick had good news and bad news. The good news is that indeed, the rate at which new CVEs are being analyzed is not on a flat line anymore. The bad news is that the line is declining, not increasing – meaning that the NVD’s productivity has declined from the already low rate it’s been at since February. In fact, someone told me on Friday that the NVD had analyzed exactly one CVE during all of last week. I guess that alone is good news, since it means the rate can’t decline any further!

2. “This is based on a variety of factors…”. If it were really a variety of factors, the problem would have been building for some time, rather than occurring on one day in February.

3. “…including an increase in software and, therefore, vulnerabilities…” Of course! That must be the reason for this problem! After all, how could anyone have anticipated that the number of software and vulnerabilities would increase - other than the fact that it’s been increasing every year since programmable computers first appeared in the early 1950s?

4. “…as well as a change in interagency support.” Translation: NIST’s budget for the current fiscal year (which started last October) was cut by 12% in March, when the budget was finally approved by Congress. I agree it’s appalling that federal agencies like NIST don’t really know what their budget for the fiscal year is before the year is well underway; in fact, Tanya Brewer of NIST (the leader of the NVD project) told the OWASP SBOM Forum last spring that the NVD usually doesn’t get its share of the NIST fiscal year budget until the summer (i.e., 8 or 9 months into the fiscal year), even when the budget has been approved by Congress by the end of the calendar year (i.e., about 3 months into the fiscal year), as it usually is. However, since this situation happens every year recently, does this really explain the sudden drop-off in February?

5. “In addition, we are working with our agency partners to bring on more support for analyzing vulnerabilities and have reassigned additional NIST staff to this task as well.” This would be good news if the rate of vulnerabilities analyzed were increasing – but it’s not. It seems that the more people NIST throws at this problem, the worse the problem becomes! What they’re really saying is, “You won’t see positive results from the additional staff for some time – but just bear with us. And cross your fingers that one of the CVEs that’s still awaiting analysis isn’t the next log4shell.

6. “We are also looking into longer-term solutions to this challenge, including the establishment of a consortium of industry, government, and other stakeholder organizations that can collaborate on research to improve the NVD.” Ah, yes, this is my favorite part of the announcement – and it’s been in the announcement from when the first one was put up in late February (after the problem had been ongoing for a couple of weeks). “Don’t worry, the cavalry is on the way as we speak. They’ll be able to fix everything (even though we still don’t know what needs to be fixed).”

The idea of a Consortium came out of the SBOM Forum’s two discussions with Tanya last spring. In the first discussion (in April), we asked how we could help the NVD implement the recommendations in our 2022 white paper on the problems with CPE and how they can be solved. Her answer was that the best way would be through some sort of public-private partnership, but she needed to talk to the NIST lawyers to find out how that would work.

She talked to us again a month or two later and announced that the right way to do this would be to form a “Consortium” of organizations (including both private-sector and public-sector entities) interested in helping the NVD. She outlined a specific set of steps that would be required:

1.      She would post some videos in July, explaining her ideas for the Consortium and soliciting comments on those ideas.

2.      She would study these comments and draft her statement about the purpose of the Consortium, which would form the basis for the required posting in the Federal Register.

3.      She expected the notification to appear in the FR in August, at which point organizations could start contacting someone on the NVD team to discuss how to join the Consortium.

4.      There would be a number of steps required to join the Consortium, the first being NIST’s decision that they want your organization to participate (I guess you can’t take that for granted!).

5.      By far the most important of the steps was that the organization would have to negotiate a customized Cooperative Research and Development Agreement (CRADA) with NIST. Doing so would require agreeing on an area of expertise that the organization has that would benefit NIST, and on how NIST would be able to take advantage of that expertise.

Tanya told us (in June 2023) that she thought the Consortium would be up and running by the end of the year. We all thought that was wildly optimistic (the CRADA negotiation itself sounds like a six-month effort at least), but we appreciated that she wanted to do this.

Or at least, we thought she wanted to do this. I didn’t hear (or read) anything from Tanya on the Consortium until the NIST announcement in February that pointed to the Consortium as the solution to the NVD’s problems. A month after that announcement, Tanya stated at VulnCon (at the end of March, six weeks after the NVD’s problems started) that she expected an explanation for the NVD’s problems to be posted within ten days (she said they knew what they wanted to say but needed to get the required approvals from higher-ups). She also said she expected the announcement of the Consortium would appear in the Federal Register in two days.

Needless to say, neither of these promises was kept, any more than the ones Tanya made to the OWASP SBOM Forum last June. And there was no explanation or – heaven forbid! – apology for that fact.

People familiar with the workings of the federal government have told me that Tanya’s idea for a Consortium isn’t a bad one on its face, but they find it hard to believe the Consortium would be up and running before 2-3 years.

Moreover, what will the Consortium do when they meet? The current problem is almost certainly due to the fact that the NVD’s infrastructure is a couple of decades old and seems to be almost completely lacking in redundancy. What are they going to do to fix that, other than rip it out and replace it with something newer? If so, why wait for a Consortium to point out something that’s painfully obvious today?

Fortunately, I’m pleased to announce that the cavalry actually is on the way. They’re not flying the flag of the Consortium, but of CVE.org, the database (formerly known as MITRE, which still operates it) in which most of the data in the NVD originates. CVE.org will put up a blog post up within two weeks detailing what’s going on, but my last post should give you some idea of that (not that I’m privy to all of the details of course).

Even better, you can join the SBOM Forum’s Vulnerability Database working group, which will hold its first bi-weekly meeting on Tuesday April 30 at 11AM Eastern Time. We’ll be discussing (with at least one member of the CVE.org Board) what CVE.org offers today and what needs to be added, both to let it replace the functionality of the late, lamented NVD, but also to go beyond that. There are lots of things you can do if you’re not constrained by two decades of…stuff.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

My book "Introduction to SBOM and VEX" is now available in paperback and Kindle versions! For background on the book and the link to order it, see this post.