Thursday, April 11, 2024

It’s time to figure out this whole vulnerability database problem

Tom’s note: I sent out the notice below to the members of the OWASP SBOM Forum and it’s generated a lot of interest. It seems people agree with me (there’s a first for everything, I suppose!) that there are just too many threads to be pulled for one person, or just a small group of people, to figure out the best strategy (or strategies, more likely) for moving forward on this issue. If you’re interested in this, please let me know.

OWASP Foundation Vulnerability Database Project

Since mid-February, the amount of usable vulnerability data added to the National Vulnerability Database (NVD) has significantly declined compared to its previous average levels. This occurred without prior warning and has not yet been explained. While the level of production seems to be gradually increasing, NIST (which operates the NVD) has not estimated when it will return to normal levels.

Moreover, NIST has not announced any measures to prevent whatever caused the problem from recurring, other than describing their desire to form a “Consortium” of private sector organizations willing to help NIST fix the NVD’s problems. Since the Consortium will take at a minimum 6-9 months to put in place (and someone with more experience with government than I told me this week that, given what has to be done, three years might be a better estimate), and since it is unclear how the Consortium might be able to assist the NVD when it is in place, the Consortium is unlikely to have much of an impact on the NVD’s problems in the short or intermediate terms.

While everyone in the software security community hopes the NVD will be able to fix its problems, it is evident the community cannot count on the NVD being a dependable source of new vulnerability data going forward - although there is no danger that the data currently in the NVD will ever become unavailable. It is time to explore all options for providing dependable and comprehensive vulnerability data to users in the US and worldwide in the short, intermediate, and long terms.   

Fortunately, today there are multiple good vulnerability database options available in both the private and public sectors. These include CVE.org, the database operated by the Department of Homeland Security. This is the source of all CVE data (used in the NVD and other databases), and is based on a modern, fully redundant infrastructure, unlike the NVD’s – although it currently lacks the user interface of the NVD. At least one large software developer has switched to using CVE.org for most of the data it previously retrieved from the NVD.

The options also include multiple privately-run databases of open source software vulnerabilities, as well as databases that include all NVD data, along with enhancements not found in the NVD.

However, the wealth of vulnerability database options also poses a challenge, since it is hard to compare the databases. This is because they address different types of software (and devices), use different identifiers for that software, list different identifiers for vulnerabilities, are at different levels of maturity, have different relationships with data sources, etc.

Even more importantly, the characteristics of the different databases are not set in stone, and some are more adaptable than others. For example, both the NVD and CVE.org databases currently identify all software and intelligent devices using “CPE names”. In 2022, the OWASP SBOM Forum described the many problems with CPE names and the superiority of purl (Product URL) identifiers for open source software, in this white paper. CVE.org currently accepts (on a trial basis) the “CVE JSON 5.1 specification”. That spec (thanks to a pull request submitted in early 2022 by Tony Turner of the SBOM Forum) makes it possible for CVE.org to utilize purl identifiers, once those are added to CVE reports by the CVE Numbering Authorities (CNAs). However, it will be at least 2-3 years before the NVD supports the 5.1 spec, and thus will be able to accept purl.  

The OWASP SBOM Forum believes it is important now to examine the near- and immediate-term vulnerability database options that are available, both to end user organizations (anywhere in the world) and to government agencies. There are two main reasons for doing this:

1.      Users of software vulnerability data need to determine what are their best options for obtaining the current vulnerability data they need, and the advantages and disadvantages of each option. Some users may decide to utilize multiple vulnerability databases, not just one, and thus will want to know what each option provides them.

2.      The US government needs to decide their best options for allocating their investments for vulnerability reporting - if there even will be more investment. Investing heavily in a database with an out-of-date infrastructure would not be a good idea, assuming there are more up-to-date options.

Rather than simply having a small number of people write a white paper, the SBOM Forum wishes to establish a working group that is open to all interested parties in any country. It can include end user organizations, software developers, operators of public and private vulnerability databases, individuals who work for government agencies, vendors of vulnerability management tools, and more. The group will probably hold bi-weekly meetings in the morning US time, to allow as much European participation as possible. However, since the document(s) will be developed cooperatively, anyone will be able to participate in drafting them, regardless of their time zone.

Because OWASP SBOM Forum members will need to devote a significant amount of time to this project, they will need to receive some compensation. Since the OWASP Foundation is a 501(c)(3) nonprofit corporation (a type of nonprofit to which donations are often tax deductible), and since donations to the OWASP Foundation that are over $1,000 can be directed to an OWASP project such as the SBOM Forum, we are requesting that organizations or individuals that are concerned about having a robust software vulnerability management ecosystem contribute to this effort.[i]

You are free to donate any amount you would like, or not to donate at all. Any donation of $5,000 or more will be acknowledged with your logo on our website, assuming you would like to do that. Note that participation in this project does not require any donation.

If we receive sufficient donations and there is interest, the SBOM Forum will extend the project to consider the longer term. In this extension, the question will change from “What are the options in the near and intermediate terms?” to “What is the optimal global vulnerability database structure long term?”

It is close to certain that the optimal long term vulnerability database option is an international one, funded by both public and private organizations but not operated by a single government or for-profit organization. One model (or even a final home) for that option might be the Internet Assigned Numbers Authority (IANA), which operates DNS and performs other functions that support the global internet.

IANA (now part of ICANN) was originally operated by the National Technology and Information Administration (NTIA) of the US Department of Commerce, but is now internationally governed. The global vulnerability database would need to be “incubated” initially by a consortium of private- and public-sector organizations, just as DNS was incubated by the NTIA.

Should the long term project move forward, it is likely the group will consider the idea of not having a single database at all. Instead, there could simply be a federation of existing vulnerability databases, linked by an AI-powered “switching hub”. That hub would route user inquiries to the appropriate database or databases and return the results to the user. Using this approach would of course eliminate the substantial costs required to integrate multiple databases into one, and tot maintain that structure. It would also probably eliminate any need to “choose” between different vulnerability identifiers (e.g., CVE vs. OSV vs. GHSA, etc.) or different software identifiers (CPE vs. purl).

We hope your organization will decide to participate in this important project and will also consider donating to it. Please contact Tom Alrich at tom@tomalrich.com with any questions.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

My book "Introduction to SBOM and VEX" is now available in paperback and Kindle versions! For background on the book and the link to order it, see this post.


[i] Donations by credit card can be made online and directed to the OWASP SBOM Forum, by going to our OWASP site. While the process is straightforward, we request that you email Tom Alrich before donating. For non-credit card donations, please email Tom. Note that OWASP retains 10% of the donation for administrative purposes (although any tax deduction will apply to the entire donation). Given the amount of work that SBOM Forum members would have to do if we were running our own nonprofit organization, we consider this to be quite acceptable.

No comments:

Post a Comment