Sunday, February 19, 2023

How do you account for hacker skill level in VEX?

The informal group I formed last spring to discuss problems that are holding back widespread (or even narrowspread) distribution and use of SBOMs has been focusing on VEX recently. This isn’t an accident, since I (and many others) now believe that the biggest factor inhibiting the success of SBOMs is the fact that VEX is such a poorly understood concept – mainly because there’s so little documentation of VEX, and what little there is contradicts itself. For example, the recently approved, but not yet published, VEX “Minimum Requirements” document directly contradicts the previous CISA Use Cases document on several key points, without even mentioning that fact. The user is left to discover this contradiction on their own.

The idea of VEX came about to address the case in which a user has received an SBOM and identified a number of vulnerabilities in the components listed in the SBOM, by looking them up in the NVD or another vulnerability database. Since there’s broad agreement in the software security community that over 90% of vulnerabilities in components of a product aren’t exploitable in the product itself, the supplier needs to have a means to let their customers know about those non-exploitable vulnerabilities, before they inundate the help desk with calls about both types of component vulnerabilities. VEX is a tool they can use to do that (although I and other members of the SBOM Forum now believe that VEX, as currently conceived, will never be able to accomplish this purpose. We’re now discussing how the VEX “delivery vehicle” can be improved to make it an effective one, although we’re not trying to change the basic use case for VEX itself).

One of the big issues that has come up multiple times in the SBOM Forum discussions (which also was discussed multiple times by the NTIA VEX Working Group in 2021. We thought we’d resolved the question then, but we didn’t put it into a document. That turns out to have been a big mistake) is how the supplier accounts for the attacker’s skill level in a VEX document.

The problem is that the supplier’s decision whether a particular component vulnerability is exploitable or not within one or more versions of one of their products will depend heavily on what they assume to be the likely skill level of the attacker. Consider three cases:

1.      The attacker has my skill level, which is somewhere around that of a (poorly) trained monkey. Using this assumption, almost no vulnerability should be considered exploitable.

2.      The attacker has a moderate skill level, meaning that vulnerabilities whose exploitation doesn’t require a “super hacker” should be considered exploitable, while others are not exploitable.

3.      The attacker is a super hacker, who could figure out a way to compromise a brick and make it do back flips.

Ruling out case number 1 as not at all realistic leaves cases 2 and 3. Which one of these should the supplier assume? The decision the VEX workgroup made in 2021 was that it doesn’t make sense to assume anything other than an attacker of average skills (with the understanding that none of these three levels is defined exactly and it would be impossible to do so). The problem with assuming a super hacker is that almost any vulnerability will be exploitable, so instead of saying that 90% of component vulnerabilities aren’t exploitable, we’ll have to say that literally all of them are exploitable. The supplier will have to patch every component vulnerability and their customers will have to apply all of them.

So, assuming case 3, there’s no need for VEX. Plus, this means that all the software security professionals who say that more than 90% of component vulnerabilities aren’t exploitable are grossly mistaken (and Chris Wysopal of Veracode was very mistaken when he said at RSA in 2017 that 97% of component vulnerabilities in Java apps aren’t exploitable in the app itself). Obviously, one should never rule out mass delusion, but I’d like more evidence of that before I make this assumption.

Thus, it seems that suppliers should assume that attackers will be of medium skill level, when they’re deciding whether or not to rule that a component vulnerability is exploitable in one or more versions of one of their products.

But here’s the problem: There are a number of users who assume, with a lot of justification, that they actually face the super-hackers. This can’t be dismissed as paranoia. After all, you’re not paranoid if everybody really is out to get you!

It’s not hard to find examples of this. Tim Walsh of the Mayo Clinic regularly states in the SBOM Forum meetings that he and his team prepare for the most skilled hackers. There’s a lot of justification for this assumption, since in the last couple of years, the ransomware gangs have figured out that they’re most likely to get a serious payout if they compromise a hospital, and the bigger the better (although I haven’t heard about big hospital attacks lately, so maybe their defenses are getting better. Sure hope so). Other obvious targets are electric utilities, oil refineries and military contractors (plus the military itself, of course). All these organizations would be quite justified in wanting suppliers to assume a much higher than average attacker skill level when they designate a vulnerability’s exploitability status in a VEX document.

However, it’s out of the question to require all suppliers to assume that all attackers are of the super-hacker level, unless the supplier only sells into one of the high security markets like hospital organizations (officially called “healthcare delivery organizations”, or HDOs, nowadays). After all, if 90% of component vulnerabilities aren’t exploitable by a hacker of average skill level, requiring the supplier to assume that all attackers are super-hackers means they will have to develop ten times as many patches as they would otherwise; and it will require their customers to invest ten times as much time to apply patches to the supplier’s products. Security always comes with a cost, and we shouldn’t require users and suppliers to invest ten times as much time (and therefore money, of course) in security measures, if we’re not convinced this is necessary.

How can a supplier who sells into all markets accommodate both the Mayo Clinics of the world and my local dry cleaners, which is – I’m just guessing here – perhaps not at the top of Vladimir Putin’s target list?

Fortunately, there is a vehicle to address this issue in VEX. It’s been available in CycloneDX for a couple of years, and in the spring of 2012 the CISA VEX working group published a document describing their idea for it. The document is called “VEX Status Justifications” (the CycloneDX capability is just called “justifications”).

Justifications are short, machine-readable explanations of why the supplier believes a particular vulnerability isn’t exploitable in a product/version they use; a justification is only used when the status of the vulnerability in the product is “not affected”, meaning it’s not exploitable. The justifications were developed based on the realization that some software users will not be satisfied with just learning that a vulnerability, in the supplier’s estimation, is not exploitable. Instead, they will want to know why the supplier believes this to be the case, so they can take a more conservative approach to determining exploitability than did the supplier.

For example, if a supplier is producing a CycloneDX VEX document for a version of one of their products and they list the status of CVE-2023-12345 in the product as “not affected”, they should examine the nine options for “justification”:

  • code_not_present = the code has been removed or tree-shaked.
  • code_not_reachable = the vulnerable code is not invoked at runtime.
  • requires_configuration = exploitability requires a configurable option to be set/unset.
  • requires_dependency = exploitability requires a dependency that is not present.
  • requires_environment = exploitability requires a certain environment which is not present.
  • protected_by_compiler = exploitability requires a compiler flag to be set/unset.
  • protected_at_runtime = exploits are prevented at runtime.
  • protected_at_perimeter = attacks are blocked at physical, logical, or network perimeter.
  • protected_by_mitigating_control = preventative measures have been implemented that reduce the likelihood and/or impact of the vulnerability.

If the reason the supplier chose the “not affected” status for CVE-2023-12345 corresponds to one of these justifications, they should include that in the VEX document. However, if none of these describes their reason well, the supplier needs to include a textual description of their reason in the “details” field.

Of course, for Tim Walsh, just seeing any justification for a “not affected” vulnerability status isn’t enough to make him feel good. For example, I know that Tim assumes that Mayo’s networks are hostile, so the threats they face are never just external ones.

If the supplier lists the “protected_at_perimeter” option, my guess is Tim won't think that does him much good, since it assumes the only attackers will be external. So, Tim will modify the tool that interprets the VEX document to make it change the status of any vulnerability, for which the “protected_at_perimeter” justification is listed, from “not affected” to “affected”.

Tim might do the same with most of the other justifications as well. However, there is probably at least one justification that he would be willing to accept: “requires_dependency”. This means that the component that is the source of this vulnerability isn’t even present in the product (I assume this would happen when the tool that generates the SBOM mis-identifies a component altogether). He might do the same for the “code_not_present” justification, which means the component is included in the product, but the code that’s the source of the vulnerability isn’t included in the component.

Let’s say that Tim decides to accept these two justifications but reject the other seven. He’ll modify the tooling to change the status of a vulnerability from “not_affected” to “affected” if any of the seven unacceptable justifications is used, while leaving the status at “not_affected” when one of the two acceptable justifications is used.[1]

So, boys and girls, our story has a good ending after all: Tim is content, because he knows he can control the status designations of the vulnerabilities in the VEX documents he receives; they will now more closely reflect his understanding of those designations.[2] Meanwhile, the dry cleaners is content, since they don’t have to lay our serious cash to pay some security person to do ten times as much patching work on the software they use, as would otherwise be necessary.

And they all lived happily ever after.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.


[1] CycloneDX VEX uses different status designations than these. I’m using them here in the generic sense of the NTIA and CISA documents, in which they apply to both VEX formats. 

[2] Of course, this assumes that suppliers will always list a justification when it’s appropriate. Unfortunately, this may not be a good assumption for at least the next few years.

No comments:

Post a Comment