Sunday, October 30, 2022

Where’s them SBOMs, now that we really need them?

 

Practitioners of the black arts of software security are collectively sighing and saying, “Here we go again”, since the announcement[i] late last week by the OpenSSL Project that they’re going to release a new version of the almost ubiquitous OpenSSL cryptographic library on Tuesday. It patches a “critical” new vulnerability in that library, about which they won’t release any information.

Since there’s no patch available and the Project didn’t describe any temporary mitigation (presumably because even that might have provided enough information to the bad guys that they could have figured out how to exploit the vulnerability – although this is just my speculation), the question naturally arises, “What was the point of announcing it now? Was it so security professionals could quickly submit their resignations and move on to their new career in fast food delivery?”

No, the reason was that the OpenSSL Project knew that organizations that use OpenSSL will experience the same problem they had with earlier vulnerabilities like Heartbleed (the 2014 OpenSSL vulnerability that led to a massive effort to find and patch it – although there are probably still a lot of unpatched instances of the vulnerable versions out there), Apache Struts (which unfortunately Equifax didn’t get the memo on, resulting in the compromise of private information of more than ¾ of adults in the US. My information was also compromised), Ripple20, and of course the beloved log4j.

That problem is that OpenSSL is a component used in other software products, or in components of products, components of components of products, etc. – yea, unto the 20th generation. No organization has an invoice from a software supplier that identifies OpenSSL as something the organization bought and installed. How does the organization figure out how many instances of OpenSSL they’re running and where they’re running?

The answer is simple: They look at the most recent (hopefully quite recent) SBOM, which stands for “software bill of materials”, which they received from each software or intelligent device supplier whose product(s) they use. These will tell them what are the components (but probably only the first-level ones) in each product. So, while you think of it, go find the SBOM for every software product or device that your organization operates. This should only take you five minutes, right?

Of course, this was a joke. I doubt there are many organizations outside of the military and intelligence agencies, which have virtually unlimited budgets, who have an SBOM even for the majority of software products they operate – and if they do, they probably created the SBOMs themselves (or a consulting organization they hired created them). In fact, I’m sure there are many large organizations that don’t have an SBOM for any of the software they operate.

If you were around for the log4j effort at the end of last year, you’ll know that almost nobody had SBOMs then. But your probably heard assurances from a number of people (which could have included me, although I don’t recall giving such assurances) that by now SBOMs would be much more widely available than they were then. There was a specific reason why we said this: We knew that the SBOM provision in Executive Order 14028 was going to come into effect in August. It required federal agencies to start asking for an SBOM from each supplier of “critical software” by this past August 10.

Of course, the EO only applied to federal agencies. But we all supposed (or I did, anyway) that software suppliers weren’t going to start providing SBOMs to one large group of their customers (the agencies), but not to an even larger group (private sector organizations). Surely, we reasoned, this would be the beginning of a “beautiful friendship” (to quote Humphrey Bogart at the end of Casablanca) between software suppliers and end users – that is, an area of close cooperation between them, to address the common problem of software component security.

It turns out that we reasoned wrong. While I’m sure that at least some suppliers did provide an SBOM on request, I know of no supplier of software or intelligent devices anywhere on this planet (I’ll admit I haven’t checked with other planets) that’s regularly providing SBOMs to their users. It’s great that some of them provided one SBOM when asked.  However, SBOMs age like milk, not wine.

The official guidance is that an SBOM should be re-issued whenever the software changes, including a new patch, new build, etc. In practice, a software user should consider themselves quite lucky if one of their suppliers provides a new SBOM with every new major version release. Of course, if the only SBOM you have for a product is an old one, don’t throw it away; it’s still worth looking at. But before you take action (or don’t take action) based on something you find in an old SBOM (like OpenSSL), you’ll need to check with each of your software suppliers to find out whether OpenSSL 3.0 is present in their product and whether the vulnerability (to be announced on Tuesday) is exploitable in their product – since in over 90% of cases a vulnerability in a component isn’t exploitable in the product itself.

Why aren’t SBOMs being released? It’s not because suppliers don’t have them; they’re using them very heavily now to manage component vulnerabilities in the products they develop. See this post, which describes how just one open source tool, Dependency-Track, is being used 270 million times a month to look up around 27 billion open source software components in the OSS Index vulnerability database; almost all of that use is by software developers.

Rather, the main reason SBOMs aren’t being released is that end users aren’t asking for them. Why aren’t they doing that? There are a number of reasons, which I divide between “show stoppers” (problems that need to be addressed before SBOMs are widely distributed and used by non-developers), and others. Here are some of the show stoppers (I think there are around 6 or 7 in all, but I might be neglecting one or two. Note that all of these problems don’t need to be solved before SBOMs can start being used with regularity. However, users and suppliers will need to be confident that a solution is at least in the works):

1.      There are no tools or subscription-based third-party services that import SBOMs and VEX documents and output a list of exploitable component vulnerabilities, which is continually updated. If this is available, it means that, at any time, a user can obtain an up-to-date list of all known exploitable component vulnerabilities in a particular version of a product. Dependency-Track comes the closest to doing that, but even that tool doesn’t provide a continually-updated list.

2.      Even if the user has the above tool or service, they can never be sure that the list of exploitable vulnerabilities for a product/version they use doesn’t contain a number of CVEs that aren’t in fact exploitable in the product, which the supplier has already identified but which they haven’t yet communicated to customers. I have proposed “real-time VEX” as a solution to this problem. I’m happy to say that the idea will most likely be submitted to the IETF as a proposed standard – part of a larger submission – this year.

3.      When an SBOM is created, fewer than 10% (and maybe fewer than 5%) of components listed will be found – without a lot of manual effort – in the National Vulnerability Database (NVD); this is according to a few major software/device suppliers, but it is widely echoed throughout the software security industry. This is the problem that an informal group that I started, the SBOM Forum, addressed in this paper. That document is being considered by appropriate authorities now; we are quite optimistic that at least the main part of our proposal will be implemented in a year or a little more than that.

4.      There are many software products that have had multiple names over their lifetime, due to mergers, acquisitions, rebranding, etc. – or simply due to an open source product being made available in multiple package managers, with a different name in each. A search on one of those names won’t find CVEs that have been reported for any of the other names. The SBOM Forum has just decided to take this problem up next.

5.      There aren’t coherent “playbooks”, describing how a software user can utilize SBOMs and VEX notifications to manage risks due to software component vulnerabilities in the products they utilize. This makes users reluctant to forge ahead with SBOMs and it makes tool providers reluctant to develop SBOM/VEX consumption tools. This is because they can’t currently be sure that what a tool is required to do won’t change (this is more of a problem for VEXes than it is for SBOMs).

6.      None of the major (or minor, as far as I know) vulnerability, asset or configuration management tools can ingest SBOMs or VEX documents, in order to learn about exploitable component vulnerabilities in products installed on an organization’s networks. Once the first problem on this list is solved, it will be easy to solve this problem, since all of these tools presumably can ingest a list of CVEs linked to products.

As I stated above, these are far from the only problems holding up widespread distribution and use of SBOMs (I have a spreadsheet with about 40 problems including the above, but I won’t say that’s complete, either). However, these are the most serious problems, and I believe they will all have to be on their way to solution, before we can say we’re making real progress on the road to ubiquitous SBOMs. As you can see, 2-3 of them are already on the way to being solved (at least for the most part), and none of the others looks insoluble.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.


[i] My thanks to Kevin Perry, who was the first to call this to my attention.

No comments:

Post a Comment