Thursday, August 1, 2024

What are your vulnerability database options?

It is now clear that, starting on February 12, 2024, the National Vulnerability Database (NVD) staff fell far behind in performing its most important function: creating machine-readable identifiers (referred to as “CPE names”) for vulnerable software described in text form in CVE reports. The NVD currently has a huge backlog of 17,000 “unenriched” CVE reports which continues to grow by about 100 reports per day; the NVD has yet to acknowledge the size of the backlog, let alone articulate a plan to eliminate it.

This poses a serious problem for the entire software security community, including both developers and consumers of software: Without a machine-readable software product identifier included in a CVE report (which means it also appears in the NVD), it is impossible for an automated process to learn about vulnerabilities that apply to a software product. This means an automated search for a software product in the NVD today will take account of only a small fraction of the CVEs that were identified since February, when the volume of CPE names created by NIST/NVD staff members dropped close to zero. The volume is now about 75-80 per day, but even that doesn't keep the backlog from growing, let alone start to reduce the backlog itself.

This wouldn’t be a serious problem if there were a clear alternative to the NVD, to which current NVD users could switch without a lot of disruption to their current processes. Unfortunately, not only is there no such alternative now, but it is hard to identify any other database that could be quickly re-purposed to provide that alternative. This includes the CVE.org database, which is the source of the CVE reports passed to the NVD. CVE.org contains all the data currently in the NVD. However, that means it only contains CPE names that were added by NVD staff members or by the CVE Naming Authorities (CNAs) themselves (as well as perhaps a small number of CPE names created by CISA’s new Vulnrichment program).

At this point, CVE.org has no plans to take over the NVD’s task of adding CPE names to CVE reports – and the CNAs themselves do not seem to be inclined to do this, even though they have been encouraged to do so. Given that eliminating the current backlog would require a big effort, for which CVE.org doesn’t have funding, it is understandable that they don’t want to undertake this job. Moreover, given the many problems with CPE names, it is certainly understandable that professionals in the vulnerability space aren’t inclined to devote much time to keeping CPE going; I know I am not inclined to do that.

From a technical point of view, what can be done to fix this problem? There is one technical step that the software security community can take, although it will require 2-3 years to implement. It would alleviate many of the above problems, although certainly not “solve” them. That step is to start including purl identifiers in CVE reports (this is now permitted by the CVE 5.1 spec). Purl is a much more reliable software identifier than CPE, mainly because it doesn’t require lookup to a central database like CPE does. For further information on why purl is superior to CPE, see pages 7-11 of the OWASP SBOM Forum’s 2022 white paper on software vulnerability identification in the NVD.

However, it will take 2-3 years (and perhaps longer than that) before it will be possible for CVE Numbering Authorities (CNAs) to be comfortable with adding purls to CVE reports, and for vulnerability database users to have the tools and knowledge necessary to make effective use of purls. The work of putting in place what is needed to make purl a complete replacement for CPE (including working out a mechanism by which purls can be created for commercial software products) needs to start as soon as possible. The OWASP Vulnerability Database Working Group is ready to begin that work, when and if funds become available.

However, users and developers of software can’t be required to wait 2-3 months to learn about today’s vulnerabilities, let alone the 2-3 years it will take for purl to be fully implemented in CVE reports. We seem to be stuck with this problem: There’s no automated way to find almost any recent CVE (i.e., any CVE identified after early February) that applies to a software product we’re concerned about, since there is no way for an automated process to link the product name to the CVE. In other words, fully automated vulnerability management is no longer possible using the NVD. Given that the NVD hasn’t even acknowledged the size of the backlog it’s facing, it’s highly unlikely it will dig itself out of this hole for more than a year.

However, there is also good news: There are many other sources of vulnerability data besides the NVD. Most of them have a much narrower focus than does the NVD; on the other hand, that narrower focus enables a much higher degree of specialization – meaning they usually do what they do much better than the NVD does (or did). Software users and software developers that have been relying on the NVD for all their vulnerability data now need to combine inputs from multiple vulnerability data sources, which means some alteration of their current processes. But they will end up with higher-quality data. Most importantly, they will have current data, which isn’t the case if they only use the NVD now.

Comparing data sources

Of course, finding the best – or at least a good – combination of other vulnerability data sources to replace the NVD isn’t easy[i], since pure “apples to apples” comparisons between sources are often impossible. The reasons for this include the following:

1.        As previously noted, almost all vulnerability databases utilize one or the other of the two primary software identifiers: CPE and purl. No single vulnerability database supports both identifiers, and it is almost impossible to create a one-to-one map between a CPE name and a purl or vice versa. Many software users (including developers) will need to use at least two databases, one based on CPE (which will capture commercial software and some intelligent device vulnerabilities) and the other based on purl (for open source software - or OSS - vulnerabilities. Purl has always been a much more widely used identifier for OSS than CPE).

2.        Just like software products, vulnerabilities also need machine-readable identifiers. Even though CVE is by far the most widely used identifier for vulnerabilities, there are other types of identifiers as well, including those used in the CERT/CC, ICS-CERT, GitHub Security Advisories (GHSA) and OSV vulnerability databases. While these other vulnerability types are sometimes mapped to a CVE identifier, that is not usually the case. Since the NVD only includes vulnerabilities identified with a CVE number, any NVD user who is not searching these other databases will never learn of those vulnerabilities.

3.        Some vulnerability databases (like the NVD) are free of charge. Others are behind a paywall. Yet other databases provide some services for free and other services for charge. Moreover, different vulnerability data providers bill their services in different ways. For example, one provider might charge based on a certain number of searches, another might charge a monthly fee, while yet another might charge just for access to a certain type of “premium” data.

4.        The amount of other information provided with the vulnerability information (often called metadata) varies widely, including information on exploits and exploit proofs of concept, presence on the CISA KEV (Key Exploitable Vulnerabilities) list, EPSS score, threat intelligence, indicators of compromise, CVSS component detail, review of vulnerability mentions in blogs, etc. Different software users will want to see different types of metadata.  

It would be nice if there were already some database that contained up-to-date information on all the various vulnerability data providers in the world, based on standardized criteria that make comparisons easy (think about online car shopping). However, that is simply not the case. There are hundreds (and perhaps thousands) of vulnerability data sources today, but there isn’t even a simple listing of them, let alone a means to (fairly) easily perform comparisons among them.

This is why the Vulnerability Database Working Group of the OWASP SBOM Forum is proposing a project to create an online “catalog” of vulnerability data providers, which truthfully describes what each provider can offer to users and discusses how users can make the best use of their offerings.

Categories of vulnerability data providers

First, we need to distinguish the primary categories of vulnerability data providers, as well as the data and related services that each can provide:

1.        Vulnerability databases based on CPE. These include the NVD and CVE.org, although CVE.org, being the source of CVE information, is a lot more than just a vulnerability database. While these databases include vulnerability data on both proprietary and open source software, they are best known for their coverage of proprietary (or “closed source”) software vulnerabilities.

2.        Vulnerability databases that are based on the NVD (and CPE) but enhance the content in various ways – including creating their own CPEs for some of the CVEs that are now consigned to the NVD’s backlog. These include VulnCheck, Vulners, and VulDB.

3.        Vulnerability databases based on purl and focused on open source software products and components. These include OSS Index, OSV, Snyk, Rapid7, and others.

4.        Smaller databases focused on particular languages or development environments, including GitHub Advisory Database, Python Packaging Advisory Database, RustSec Advisory Database, and the Cloud Security Alliance’s Global Security Database. Vulnerabilities found in these four databases are all found in OSV.dev, in a normalized vulnerability format that allows single searches across all four databases. The complete list of vulnerability databases currently searchable in OSV can be found here. There are many other providers (mostly smaller) in this category, besides the ones included in OSV.

5.        Service providers whose purpose is to “enrich” the CVEs included in the NVD. This includes CISA’s Vulnrichment program.

6.        Service providers that provide platforms or services in which vulnerability identification plays a big role, even if it isn’t the primary purpose of the platform or service – for example, “platforms” for creating and analyzing software bills of materials (SBOMs). There are many of these service providers.

7.        There are likely other categories of vulnerability data providers, beyond the six categories listed above.

Our “proposal”

The Vulnerability Database Working Group of the SBOM Forum proposes to produce a “Vulnerability Data Provider Catalog” of organizations like those identified above; the catalog will include a separate report describing the products and services provided by each included provider. This catalog will allow public and private sector organizations that are “consumers” of software, as well as software developers, to understand their options for obtaining current vulnerability data for the software they utilize and/or develop.

Please note:

·        The data and services provided in aggregate by the above organizations go far beyond what the NVD provided previously, or what it will ever provide. In other words, it is likely that the Vulnerability Data Provider Catalog will awaken many organizations to possibilities for vulnerability management they were never even aware of when they were exclusively using the NVD to learn about vulnerabilities in their software.

·        The goal of this project is not to produce the equivalent of a huge spreadsheet with hundreds of columns, which compares the offerings of a hundred or more vulnerability data and service providers. Rather, it is to produce a set of short qualitative descriptions of individual vulnerability data providers, which are focused on what data and/or services this provider offers. These include discussions of why particular vulnerability data offerings may be important for some types of users, but not for others. We assume the users of the catalog will have diverse needs. No vulnerability data provider, or even combination of providers, meets the needs of most or all users – and this includes the NVD, even if it gets back on its feet again.

·        Producing each report will require interviews with the provider’s staff members, viewing demonstrations and marketing materials, reviewing documentation, etc. It is likely that most reports will require multiple days to produce.

·        The reports will initially be published on a page linked on the SBOM Forum’s OWASP website, although they may later be published on a separate site as the number of reports in the catalog increases.

·        Reports will be published as they are created, so the site will be growing continually.

·        As we interview more vulnerability data providers and post more reports, it is likely we will learn lessons that can be applied to reports already posted. With the permission of the affected provider, we will amend a posted report when necessary.

We will

·        Base our reports primarily on our discussions with the provider’s staff members, as well as observations we can make regarding data or services currently provided.

·        Include in our reports the provider’s expectations for future offerings, while making it clear these are just projections.

·        Make additions or changes to a provider's report whenever the provider requests them, if these meet our criteria for verifiability (see below).

We will not

·        Endorse providers or choose a “Best” provider in a category (or overall).

·        Repeat provider statements that we suspect are not accurate, unless the provider can verify their accuracy.

·        Reproduce statements made by third parties about a vulnerability data provider, unless we have been able to verify those statements.

Donations

Because of the amount of time required to gather information on each provider and develop a report (expected to be at least two days for many providers, with large providers requiring more time than that), this project will not be possible without donations.

You may donate in your organization’s name or as an individual. We are not aiming for large donations (although they won’t be turned down!), but rather to have a large number of small donations, each from $1,000 to $5,000. All donations will come through OWASP, unless the donor cannot do that for some reason – in which case we will identify an alternative channel for receiving the donation. OWASP retains ten percent of the donation amount for administrative costs. We believe this is reasonable, given the amount of work they save us. OWASP is a 501(c)(3) nonprofit organization; in many cases, donations will be tax-deductible, although that should always be verified, if it is a primary concern.

OWASP does not directly pass donations to the project designated by the donor – in this case, the SBOM Forum. Instead, donations are retained and paid to the project upon submission of valid invoices for project expenses, which follow the Expense Reimbursement Policy in the OWASP Rules of Procedure. This policy is intended to give donors confidence that their donations are being well spent.

Donations need to be restricted to the SBOM Forum; this can be done for any donation over $1,000 (in other words, the SBOM Forum will not receive anything from a donation to OWASP that is under $1,000, although the donation will still support OWASP itself). Currently, all donations to the SBOM Forum will be allocated to the Vulnerability Data Provider Catalog project, unless the donor requests otherwise in an email to Tom Alrich.

To donate with a credit card or other online payment method, go to the OWASP SBOM Forum page and click Donate in the top right. On the next page, click on Other and enter $1,000 or more. Then enter your email address (twice) and your name (which doesn’t have to match the name on your credit card). Finally, check the box to restrict the gift to the SBOM Forum.

If you have any questions, feel free to email Tom Alrich first at tom@tomalrich.com. If you don’t want to donate through the online method, OWASP can take donations directly as well.

Even if you have already donated, please email Tom to let him know about this, so he can make sure you’re properly recognized for your donation. If you approve, your organization’s logo (or your name, if you are donating as an individual) will be listed under the “Supporters” tab on our website.

Are you a vulnerability data provider?

If so, you probably realize that it's hard to communicate exactly what you do to end users, and especially to communicate why they ought to look to you to provide more than what they previously received from the NVD. When the only way for a user to learn about their options is to try to compare a number of wildly different provider websites, all providers are at a disadvantage. 

If surfing lots of websites is their only option, a lot of users will not even try to replace the NVD; in fact, this is probably the case for a large percentage of users (and perhaps most of them) today. The OWASP Vulnerability Database Working Group is concerned about the fact that most end user organizations, and even software developers, don’t currently have a good way to learn about their options for replacing the NVD (and in the process finding higher quality vulnerability data).

“Audience”

Who will be the “audience” for the Vulnerability Data Provider Catalog? There are three broad categories of organizations:

1.        Public and private sector organizations that are concerned about the security of the software they use to accomplish their mission;

2.        Software developers concerned about vulnerabilities in the products they’re developing; and

3.        Vulnerability data providers that understand the importance of having a trusted third party like OWASP provide standardized, verifiable information on their products and services, as well as those of other providers, through a single web location. 

In other words, any organization concerned about software vulnerabilities should consider joining and/or donating to the OWSAP Vulnerability Database Working Group. Please contact Tom Alrich at tom@tomalrich.com.


[i] Of course, there will be no “best” solution for all users, or even all users of a particular type. Each organization needs to figure out the best solution for itself, although the Vulnerability Database Working Group will be glad to discuss these questions with you (and you are welcome to join the group's bi-monthly meetings – email Tom Alrich to do so). 

No comments:

Post a Comment