For a year and a half, I’ve been talking about what I call the Global Vulnerability Database (GVD) – which isn’t a single database at all, but rather a federation of databases behind a single intelligent front end. At first, I just brought it up as one of those things that will take many years to come to fruition, but which is nice to think about, anyway.
That changed in February 2024, when the National
Vulnerability Database (NVD) greatly slowed down performance of their most
important function: adding machine-readable software identifiers called CPE
names to new CVE records. Like a lot of people, I initially believed the NVD’s
assertions that they were getting back on track and would soon resolve their
backlog of over 30,000 CVE records that were “unenriched” – i.e., that did not include
a CPE name(s) for the vulnerable product(s). This deficiency renders those
records invisible to automated searches (such as from the NVD’s command line).
However, those assertions have all fallen by the wayside. On
March 19, the NVD announced, “…we are working to increase efficiency by
improving our internal processes, and we are exploring the use of machine
learning to automate certain processing tasks.” In other words, they’re no
longer even pretending they’ll eliminate their backlog, or even stop its
growth, anytime soon.
Meanwhile, the percentage of CVE records the NVD is enriching[i]
in a timely fashion has probably fallen to around 25% from 45%, where it was at
the end of 2024. Moreover, this week I started to hear complaints about “503”
(“service unavailable”) errors when people tried to reach the NVD (I experienced
six of those in one day, vs. only one success). It’s no exaggeration to say
that the NVD may be falling apart in real time.
There are two problems here, with separate causes. The
problem with the lack of CPE names is in some way due to cuts in funding by
another agency. NIST (the NVD’s parent organization) checked between the couch
cushions and found them some extra cash last year, yet if anything the problem
has gotten worse since they did that. The solution to the CPE problem (although
it will take a couple of years before it’s fully in place) is to make
it possible for CVE records to include purl identifiers, not just CPEs.
That solution doesn’t depend on the NVD, but rather on CVE.org. I trust that
organization much more than I do the NVD, although I’m worried
about what might happen to them as well, given the current political climate.
However, the 503 errors aren’t primarily due to funding. They’re
due to the NVD having a decrepit infrastructure with at least some
non-redundant systems, as well as programs written in an old language that even
I – old as I am – had never heard of. The only good solution to this
problem is to bring up a huge dumpster to the building and gently deposit all
their current servers, etc. in said dumpster. The NVD doesn’t have to worry
about backing up their data, since the entire NVD can be downloaded in about
ten minutes (I’m not kidding); thousands of complete backups of the NVD are
made every day.
The only solution to the NVD’s problems is to replace it.
But simply modernizing the hardware and software isn’t enough. Instead, we
(meaning the worldwide software security industry) need to get together to
decide what objectives we need to achieve in replacing the NVD. Here are my
ideas on that topic:
1.
Access to the GVD needs to be free, at least for
casual users. Users that regularly download large amounts of data using an API might
need to be charged, although doing that could possibly cause more problems
than it would solve.
2.
The GVD must be truly global and not under the
control of any one government. The budgets and priorities of government
agencies often fluctuate due to political decisions, with the needs of their
users - their real “customers” – taking a distant second place.
3.
The GVD should be run by a global nonprofit set
up something like the Internet Assigned Numbers
Authority. IANA runs DNS and assigns IP addresses worldwide. It performs
these tasks smoothly and efficiently. The fact that most of us use DNS hundreds
(or even thousands) of times per day without even thinking about it shows this
model can work well.
4.
Support for the GVD should come from both
governments and private for-profit and nonprofit organizations. Governments that
want to participate (and are not considered to pose a security threat to the
database or its users) might pay an assessment based on GDP, population, etc.
5.
The GVD should be as comprehensive as possible.
That is, it should include all major types of vulnerabilities (CVE, OSV, GHSA,
etc.), as well as all major software identifiers (mainly CPE and purl). Achieving
this goal will require creation of a federation of individual databases that
already exist.
6.
This federation will be enabled by an
“intelligent front end”. This will analyze all requests and interpret them for
one or more databases. For example, a request for vulnerabilities that affect
an open source product designated with a purl identifier could be routed to open
source databases that support purl, like OSV, OSS Index, GitHub Security
Advisories, etc. The responses might refer to different types of
vulnerabilities (e.g., CVE, OSV, and ICSA), but they would all be based on purl.
7.
The individual databases will continue to be
accessible directly – i.e., not going through the intelligent front end.
8.
There will need to be some mechanism by which
vulnerability databases that are currently “for charge” will be able to charge
for their services, perhaps on a per-transaction basis. Of course, the user
will need to be warned when their search is about to be routed to a for-charge
database.
9.
Responses will usually be routed back to the
user without change, since it is usually impossible to “harmonize” different
types of vulnerabilities. The exception to this rule is cases in which the person
reporting the vulnerability (often a CVE Numbering Authority) has assigned it
two identifiers, e.g. a CVE and an ICSA.
10.
The NVD does not need to go away, since it is
the primary “custodian” of CVE records that include CPE identifiers for vulnerable
products. Even when the NVD supports purl, many (and initially most) CNAs will
continue to include CPE names in their new CVE records.
11.
CVE.org will continue to operate as a separate
organization, since CVE records are widely used worldwide, not just in the NVD.
However, it would be better to move CVE.org to an international organization,
perhaps another IANA division that is separate from the GVD. To give CNAs more
incentive to include items like CVSS scores and purl/CPE identifiers in their
CVE records, it would be better if the CNAs could be paid in some way.
We might start having discussions of this in the bi-weekly SBOM Forum meetings, including today at 1 AM Eastern Time. If you would like to join us, please send me an email.
f you would like to comment on
what you have read here, I would love to hear from you. Please email me
at tom@tomalrich.com.
My book "Introduction to SBOM and VEX" is available in paperback and Kindle versions! For background on the book and the link to order it, see this post.
[i] When
looking at the NVD’s statistics, it’s important to keep in mind that they have
changed their definition of “enrichment” of a CVE record. It used to include
adding a CPE name, as well as other items like CVSS score. However, they seem
to have conveniently dropped the requirement for a CPE name to be added, for it
to be enrichment. While it’s good to have a CVSS score as well, CPE (or
hopefully purl in the not-so-distant future) is by far the most important
element that needs to be added to a CVE record.
