A VEXing problem for SBoMs

In my last post, I described what I believe to be the number one problem that needs to be addressed before software bills of materials can be widely used in any industry, including power. Now I’ll describe what I think is the number two problem (although the solution to this problem isn’t anywhere near as hard as it will be for the first one). I have learned about these and other problems (as well as solutions!) based on my participation in recent discussions conducted by the Software Transparency Initiative run by the National Telecommunications and Information Administration (NTIA) of the US Department of Commerce, which I am attending on behalf of a client interested in seeing widespread adoption of SBoMs in the electric power industry.

In another recent post, I described the main use case (although there are many others) for SBoMs: identifying vulnerabilities that are found in a software product, that are due to software components that were incorporated into the final product by the developer. The main advantage of knowing about component vulnerabilities isn’t so your organization can patch them – either the software supplier or the component supplier needs to do that.

Instead, when you learn about a serious vulnerability in a component of a software product that your organization is operating, you can contact the supplier and ask when they will develop a patch for it. The reason why this is a worthwhile activity is that some suppliers are less motivated to patch component vulnerabilities than they are to patch vulnerabilities in the code that they wrote themselves (since after all it’s only the latter that carry the name of their product. A vulnerability associated with a component is just listed under the component’s name). In these cases, your gentle prodding of them (and probably that of other customers as well) may speed up development of a patch.

So far, this is all well and good. But here’s the problem: Only a fraction of the vulnerabilities that apply to a software component will be found in the product that incorporates them. In fact, the application security company Veracode estimated recently that only five percent of component vulnerabilities actually are found in the final product – in other words, only five percent are exploitable in the final product. Why is this the case? There are many reasons. The most common reason is that the portion of the component’s code that contains the vulnerability was never included in the final product in the first place. Another might be that some other aspects of the product have the effect of making the component vulnerability unexploitable.

This means that, if your organization receives an SboM for a product you own and you identify (after matching component names with CPE names, as described in my previous post) vulnerabilities that are found in 20 components that happen to be included in that product, on average 19 of these will be false positives - meaning the vulnerability isn’t exploitable in the final product itself. So if you call all 20 suppliers, 19 of them will tell you the vulnerability isn’t exploitable. And If you were to obtain patches for the vulnerabilities from the suppliers of say all 20 components (sometimes the component supplier can develop a user-installable patch themselves, especially if the component is a library that is called by the code in the product. In those cases, the patch consists of nothing more than a replacement for the library. For example, Treck, the company behind the Ripple 20 vulnerabilities, provided patches for their vulnerable libraries after the vulnerabilities were identified recently) and apply it, in 19 of those 20 instances you would have been wasting your time.

And the suppliers might waste even more time. For one thing, they would have to answer a lot of calls whenever a component vulnerability was identified (this is the number one reason I’ve heard from suppliers for why they’re leery of putting out SBoMs in the first place). Another is (and I just heard this this week from a manager with a large software supplier) that some customers require their software suppliers to develop a patch for every vulnerability in a component, even if it isn’t exploitable in the final product. They do this for regulatory compliance reasons.

As an aside, the previous sentence shows that a NERC entity should never identify a software component supplier as a patch source in CIP-007 R2.1. Even if the vulnerability addressed by a component supplier’s patch isn’t exploitable in the product you own, you’ll be obligated to apply (or mitigate) every patch, because you identified the supplier as a patch source. Normally, you should list the software supplier as the patch source, meaning you only have to apply the patch if the supplier says it’s required.

The upshot is that neither suppliers nor end users are likely to be too interested in producing or using SBoMs if they find that 95% of the work they end up doing turns out to be a waste of time! This problem clearly needs to be addressed before SBoMs will be widely produced or widely used.

Fortunately, unlike the naming problem that I wrote about a few days ago, the road to solving this problem is fairly clear: The supplier of the final product needs to notify customers that a particular vulnerability for a component used in their product isn’t exploitable, and therefore doesn’t need to be patched or otherwise mitigated.

But how should they do this? Should they just put an announcement on their web site? Since many software suppliers are reluctant to publicly disclose the components included in their products, they’re not likely to want to do this – plus this is the kind of notification that needs to be pushed out to the appropriate people, not just made available so that it is up to them to find it.

Should the supplier just send a letter to the main office of their customer and let whoever opens it figure out who needs to see it? E.g. “Dear Duke Energy: We wish to inform you that you don’t need to worry about the fact that Component X, which is included in our Product Y Version ZZ, has been listed as vulnerable to CVE ABC. We have determined that vulnerability isn’t exploitable in Product Y, because of the way component X was implemented in Product Y. Have a nice day.” Of course, this letter will go in the circular file.

Ideally, this notification would come with the SboM when it is downloaded. If the end user obtains the SboM from the supplier of the final product, that supplier could include the notice with the SboM (although it wouldn’t be included in the SBoM. Because vulnerability information changes so quickly, the NTIA group has decided that it shouldn’t be included in an SBoM. That is supposed to be stable, at least from version to version).

But what if the user doesn’t obtain their SboM from the supplier? For example, someone with a reasonable understanding of software (which rules me out) can fairly easily generate their own SboM for a product, based on information included in the installation files they receive. They should also be able to receive this information. And they should ideally be able to receive this information from every supplier of a software package that their organization uses, which lists all non-exploitable vulnerabilities in any component of any one of those packages.

Of course, there’s no way these notifications can all be sent by email. Half of the emails will probably never get to the right person, and when they do, that person’s inbox will probably contain so many of the emails that the person will most likely not even look at most of them.

Ideally, there would be some sort of central repository like the DHS/Mitre CVE database, where the user could enter a CPE name (I discussed CPE names at excruciating length in my previous post) and receive, for the product and version referred to by the name, an up-to-date document that lists all of the non-exploitable vulnerabilities in components of that version of that product. Of course, not only is there currently no such database; there is no format for suppliers to provide that data (like the three main SboM formats that can be used to provide data about components themselves).

And by the way, this format will need to be machine-readable, just like SBoMs themselves are machine-readable. Any organization of even medium size will ultimately find it impossible to manage this data by manual means, just as it will ultimately be impossible for them to manage SBoMs themselves by manual means (currently, since there are so few SBoMs available, managing them by manual means is not only possible but required. This is because there are currently no automated tools available for managing SBoMs and correlating them with vulnerability data).

I wish I could tell you there’s a clear solution to this problem, but that’s far from being true. This is one of the more active questions being discussed by the NTIA Software Transparency Initiative. The name for this activity is currently VEX, an abbreviation of “vulnerability exploitability”. However, that name will likely change at some point, since it seems some organization has a trademark on VEX.

But not falling afoul of trademarks is the least of the problems that need to be addressed.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.