I had a Road to Damascus-type incident recently, except that, unlike in the original incident, I wasn’t blinded and I didn’t fall off my horse.
What led to my incident? I’ve become increasingly concerned of late about the prospects for consumption of software bills of materials (affectionately known as SBOMs). I’m not worried about production: software suppliers are already producing lots of SBOMs for their products and reaping a lot of benefits from doing so. But those benefits are strictly internal; few suppliers are distributing their SBOMs to their customers, and close to none are doing it with any regularity (in general, a new SBOM should be released whenever there has been any change at all in a software product).
Given that they’re producing lots of SBOMs, why aren’t suppliers distributing them to their customers? Is it because they’re all meanies and they don’t want to share the benefits of using SBOMs with their customers? No, it’s much simpler than that: They’re not sharing them because the customers aren’t asking for them.
So why aren’t the customers asking for SBOMs? There are two simple reasons:
First, a lot of software-using organizations don’t feel like making the (admittedly substantial) investment in time required to start using SBOMs intelligently, at least currently. The people inside these organizations who would be called on to find some use for the SBOMs (i.e. currently overworked security people, fresh from losing their holiday break to log4j, thank you very much) would have to invest a lot of time learning about SBOMs and trying to think of ways their organizations could use them (since it seems the end users within their organizations aren’t asking for them either).
Currently, anyone who’s starting with no knowledge of SBOMs needs to read about five or six NTIA documents and synthesize the sometimes conflicting statements in them into a single narrative (which even then will include a number of gaps that haven’t been addressed by the group yet). Until either the learning burden diminishes, or someone can do all of this learning for them (and maybe transfer the knowledge to their brains via a head-to-head USB cable – have humans evolved USB ports yet?), these people – and the organizations they work for – aren’t going to be interested in SBOMs.
Second, customers who do see benefit in having SBOMs have done enough reading to know that the two major SBOM formats – SPDX and CycloneDX – are both proudly machine-readable. Yes, you can get them in non-machine-readable formats like XLS, but given that for example the AWS client has 5600 components (as Dick Brooks of Reliable Energy Analytics has pointed out), do you really want to try to deal with all of those in a spreadsheet?
But what happens when this second group of customers looks around for easy-to-use low-cost or open source vulnerability management tools that can ingest SBOMs (and later VEXes, since the two need to go hand in hand)? They don’t find them. I believe the best SBOM consumption tool for vulnerability management purposes is Dependency Track, an open source tool developed under OWASP. It was originally developed in 2012, about five years before the term SBOM started being widely used in the software community.[i]
Dependency Track does all the basics required for software component vulnerability management and is widely used by developers. It just requires that a CyloneDX SBOM be fed into it (or it will create one from the source code). Then it will (among other tasks) identify all vulnerabilities (CVEs) in the NVD that apply to components and update this list as often as required. It does suffer from the limitation of not being able to ingest VEXes – but VEX is so new (and still undergoing modification) that no other product currently supports this format, either.
But since Dependency Track is an open source tool that requires more user involvement and knowledge than just pushing a Download button and then hitting Yes or Next a few times, there will always be a lot of users who won’t want to get involved with it. This despite the fact that IMO it’s as of now the only show in town (but that being said, I think there could be a lot more non-developers who would start using D-T for component vulnerability management purposes, if they were informed about easy it is to use the tool once it’s installed. Steve says there are a large number of these non-developer users now; in fact, OWASP may sponsor a webinar soon, focused on exactly this use case. If and when that happens, I’ll be sure to let you know).
To sum up what I’ve said so far, I don’t see demand for SBOMs jumping significantly until two things happen. First, there needs to be a single document that walks a technically-oriented reader, who has no previous knowledge of or experience with software development or SBOMs, through the entire process of using SBOMs for vulnerability management purposes.
Second, there needs to be one of these two items (and hopefully both):
· An easy-to-install open source (or low cost) tool that at a minimum a) ingests an SBOM for a product and extracts component names - hopefully in CPE format, b) regularly (preferably daily) searches the NVD for vulnerabilities applicable to components identified in the SBOM, and c) removes from that list any vulnerability that has been identified by the supplier of the product as not exploitable in the product itself (the latter information may someday be communicated in a VEX document, but it might be communicated in other ways as well).
· A third-party service that processes SBOMs and VEX information and provides to the customer the same list of exploitable component vulnerabilities provided by the hypothetical tool described above. Of course, since there are other sources of component risk besides just vulnerabilities listed in the NVD (such as the “Nebraska problem” and three others, mentioned in this post), the service will probably address these risks as well. It might also address risks due to vulnerabilities not listed in the NVD, but identified in other databases or other non-database sources.
Of course, neither of these items is available today – otherwise I wouldn’t have written this post. I’m sure at least the service will be available by the August deadline for federal agencies to start requesting SBOMs from their software suppliers. I’m not sure about the tool, mainly because VEX isn’t finalized yet, and SBOMs without VEX information just aren’t going to be very useful. A third party service provider would hopefully be able to get VEX-type information directly from the suppliers, whether or not VEX is finalized – since the supplier should be quite interested in getting the word out about unexploitable component vulnerabilities, and the supplier and the third party can easily work out their own format for communicating VEX-type information.
So is what I’ve just described my Road to Damascus moment? No, this is something I’ve come to realize over the last four months. My RtD moment occurred when I asked the question – prodded in part by a suggestion from a friend and client who I won’t name, since I haven’t had a chance to ask his permission for this – why software users should have to bear the responsibility for identifying exploitable software component vulnerabilities. I now think the software suppliers should bear that responsibility. I have three reasons for saying this:
1. Currently, you usually learn about vulnerabilities in a software product that you operate from the supplier of the product. You receive an emailed notice directly from the supplier, or at least the supplier reports the vulnerability to the NVD, where you discover the vulnerability (hopefully, both will happen). But a lot of exploitable component vulnerabilities aren’t currently reported by suppliers, using either method. What is it that makes component vulnerabilities different from vulnerabilities identified in the supplier’s own code, other than the fact that you won’t normally be able to find out about component vulnerabilities without…envelope, please…an SBOM? None that I know of.
2. Does it really make sense to say to each customer of a software product, “You’re responsible for finding component vulnerabilities in Product A and maintaining those lists day in and day out”? This even though the supplier is already gathering this information (or at least they should be)? If the suppliers provide this information to their customers, the latter will only need SBOMs and VEXes (and a tool or service to process them) as a way of checking to make sure the supplier hasn’t left any exploitable component vulnerabilities off the list (of course, this assumes that all suppliers immediately accept this responsibility. Nice idea, but ain’t gonna happen). But the users won’t bear the responsibility for learning about the vulnerabilities in the first place.
3. The third reason makes a lot of sense to me from an economic point of view: The party that introduces a risk for their own benefit should bear the burden of remediating that risk. As everyone knows, developers’ use of third-party components (both open source and proprietary, but mostly the former) has ballooned in recent years, but so has component risk. Log4j, Ripple20, Heartbleed, Apache Struts/Equifax and other disasters have been the result. In fact, having the suppliers be responsible for identifying and tracking component vulnerabilities isn’t a big increase in their current responsibilities, since they should already be doing the hard part now – patching those component vulnerabilities after they identify them and determine they’re exploitable (which, fortunately for the suppliers, is usually not the case).
But it’s not like the users won’t have to do anything at all about component vulnerabilities. They’ll still need to track down and identify – using configuration and vulnerability management tools – the vulnerable software on their network and apply the patches for exploitable component vulnerabilities, which will hopefully be quickly forthcoming from the suppliers. But instead of waiting for the vendors of those tools to develop the capability to ingest SBOMs and VEXes in order to obtain information on component vulnerabilities (something it seems not a single major vendor has done so far), the supplier of the software (or again, a third party they’ve engaged for this work) could provide a feed that follows the vendor’s API – meaning the vendor will have to expend exactly zero effort in order to become “SBOM-ready”. This alone is a huge benefit to users, since waiting for the tool vendors to become SBOM-ready seems like Waiting for Godot: He can’t come today, but for sure he’ll come tomorrow…
And there’s another huge advantage to the idea of making the suppliers responsible for component vulnerability management: the need for VEXes largely goes away. This is for a simple reason: VEX was designed primarily for a supplier to communicate to customers – who have SBOMs and are looking up component vulnerabilities in the NVD – that certain component vulnerabilities aren’t exploitable in the product itself. The user needs VEXes, because if they start looking in the product for every component vulnerability listed in the NVD, some huge percentage of the time they spend doing this (as well as the time they spend calling their supplier about component vulnerabilities they find in the NVD) will be wasted. The customer needs the VEX in order to winnow the list of component vulnerabilities down to only the small percentage that are exploitable.
But if the supplier itself is responsible for producing the final list of exploitable vulnerabilities, the communication that would otherwise require a VEX would be completely internal: Whoever determines whether a vulnerability is exploitable or not would send an email to the person who is responsible for the final list of exploitable vulnerabilities for the product (in fact, they may be the same person). The email would say something like, “Remove CVE-2021-12345 from the list of exploitable vulnerabilities in Product X. Even though the NVD shows that vulnerability is found in Component A, and even though Component A is included in X, CVE-2021-12345 isn’t exploitable in X. This is because A is a library, and we never included the vulnerable module in X in the first place.”[ii]
There will still be some need for VEXes, including what may become the main need for SBOMs – to make sure the supplier is properly identifying exploitable component vulnerabilities in their products. But VEXes won’t be a gating factor for the use of SBOMs at all, which is what they are today.
My unofficial slogan is “Often wrong, but never in doubt.” I have to admit that it seems too good to be true, that having the supplier take responsibility for component vulnerability management would be a win-win in so many ways. If you think I’m full of ____, please let me know. But I will interpret your silence to indicate total agreement (unless you fell asleep before you read this).
Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at firstname.lastname@example.org.
[i] The leader of the Dependency Track project is the same person who leads the CycloneDX project, also under OWASP: Steve Springett, an active member of the NTIA-but-now-CISA Software Component Transparency Initiative (and someone who lives about ten miles from me in suburban Chicago. I suggested to Steve that we meet for lunch recently, but he wanted to wait until we can eat outside. I’ve been looking for restaurants in Chicago that have outdoor seating in January, but it’s hard to find any, for some reason).