Is it time to abandon VEX?

Monday’s meeting of the CISA VEX working group led off with a rather disturbing exchange: the two leaders of the group – in obviously coordinated comments - both wondered whether VEX even belongs in the CISA SBOM effort. They asked whether it might be “better” to host it somewhere else. Where? They didn’t suggest any alternative venue. Fortunately, one of the most active participants in the group strongly objected to this idea and the leaders said they would be “happy” to continue the meetings (as in “I’ll be happy to take this medicine that I hate taking”).

But they also made it clear that CISA management is now asking questions about what they’re getting from their 19-month (so far) commitment to the VEX working group (the group started in 2020 under the NTIA and was the only NTIA workgroup that continued under CISA, although its exact status under CISA has never been clear). To be honest, if I were CISA, I would ask the same questions. Here’s a little history to explain what I mean:

1.      When the NTIA Software Component Transparency Initiative ended at the end of 2021, the VEX meetings continued under CISA without any interruption in January 2022.

2.      Under the NTIA, the group only produced a single one-page Introduction to VEX (which I drafted). It was a decent effort, but it was out of date by 2022 (e.g., it assumed there was just one VEX format: CSAF); plus, there’s only so much one can say in one page.

3.      Under CISA, the group immediately produced two well-written and important documents on VEX in the spring of 2022: VEX Use Cases and VEX Status Justifications.

4.      Last summer, the group entered a period of wandering in the wilderness, which this year produced two documents that add just about nothing to our understanding of VEX: VEX Minimum Requirements and the close-to-being-finalized When to issue VEX information. Both of these are classic examples of a “document for the sake of having a document”. While neither one contains untruths or misrepresentations (except for the title “Minimum Requirements”, since nothing in that document is “required” by anything other than the document itself), there is nothing I can point to in either document that wasn’t already obvious or wasn’t stated better in one of the 2022 documents.

Given the above, I can certainly understand why CISA is concerned that the investment they’ve made in the VEX working group (primarily in the form of employee time) during at least the past 12 months has been unproductive. So, why do I think this group shouldn’t be discontinued?

That’s simple: The VEX effort started because several huge software and intelligent device suppliers had realized, early in the NTIA SBOM effort, that distributing SBOMs to their customers regularly would inevitably lead to their help desks being overwhelmed with calls from customers about “false positive” component vulnerabilities. This is because more than 90% (some say more than 95%) of component vulnerabilities – which a customer can identify by running an SBOM through a tool like Dependency-Track – are not exploitable in the product, even though they are exploitable in the component when it is considered to be a standalone product. Not only would these calls overwhelm their help desks, but they would be a source of endless (and unnecessary) frustration, both to customers and to help desk personnel.

It gets down to this: Software users won’t be interested in receiving SBOMs if they know that a) over 90% of the component vulnerabilities they learn about will be false positives, yet b) they won’t know how to distinguish these from the ten percent that are real; they will just want to learn about the real ones. And suppliers won’t be interested in providing SBOMs to their customers unless there’s some way they can let them know, in an automation-friendly fashion, which component vulnerabilities are exploitable (along with information on downloading a patch for each, or an upgrade path to a patched version) and which aren’t exploitable.

A recent meeting of the Healthcare SBOM Proof of Concept (which started under the NTIA but is now sponsored by the Healthcare ISAC and the Healthcare Sector Coordinating Council) produced these two relevant statements, one from a large healthcare delivery organization (HDO) – aka hospital chain - and one from a very large medical device maker (MDM):

1.      The HDO said they had recently looked up vulnerabilities for the software components identified in the SBOM for a medical device they use and found 2,000 vulnerabilities in that one device! Of course, at most 200 of these are exploitable, and the actual number of exploitable vulnerabilities is probably less than 100. But knowing this is cold comfort if the user organization has no way of knowing which of the 2,000 vulnerabilities are exploitable and which aren’t. They made it clear: “We need VEX!”

2.      The MDM said they won’t issue SBOMs until they can also issue VEXes – due no doubt to the fear of being overwhelmed with false positive help desk calls. More importantly, they need to issue VEXes with the knowledge that they will be used by the HDO in an automated fashion to “winnow down” the list of component vulnerabilities to only those that are exploitable. Obviously, if a user organization identifies 2,000 component vulnerabilities in a single device, they will need a tool that will ingest VEXes (as well as SBOMs) and do the winnowing for them. Currently, no such tools exist, at least with commercial support, which is required by most non-developer organizations.

There you have it: Suppliers won’t be interested in distributing SBOMs to their customers, and software users (meaning organizations whose primary function is not developing software) won’t be interested in using SBOMs to learn about software component vulnerabilities, unless VEXes are made available and can be used effectively, in an automated fashion.

Absent VEX, SBOMs will go nowhere. This is why we need VEX.

Tom's note 7/31: Allan Friedman announced in the VEX working group meeting today that CISA has no intention of abandoning VEX, so that's good news. 

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.