It’s safe to say that most organizations today that are
concerned about patching vulnerable software on their networks have many more
patches to apply than they have time and personnel to apply them. Of course,
this isn’t a new problem, but it’s gotten worse. In fact, Chris Hughes
published an (as usual) excellent blog
post recently that mentioned “…a report from
Rezilion and the Ponemon Institute showing that…more than half of organizations
have more than 100,000 open vulnerabilities in their backlog…”
However, what is a new problem for these organizations
is that, within the next 2-3 years, they will be receiving SBOMs regularly from
at least some of their software suppliers. And they know those SBOMs are only
going to make this problem worse, not better.
Why do they say this? To simplify the analysis, let’s say
that each software product or software component will on average be
found to have one vulnerability in an NVD search. If we start with the situation
that almost 100% of organizations are in today – that they are not regularly
receiving SBOMs for the software products used on their network – this means
that, if the organization uses 1,000 products, they will usually have 1,000
vulnerabilities on their “to be patched” list. Of course, that doesn’t mean
1,000 devices to be patched. If each of those vulnerabilities is found on ten
devices, there are actually 10,000 devices to be patched.
Since we’ve already stipulated that the organization is
overwhelmed with their current vulnerability load, this means that 1,000
vulnerabilities is too many for them to handle at one time. Therefore, they’re
probably doing what most organizations would do; performing triage on the vulnerabilities
they have to patch, perhaps by breaking them up into three categories – high,
medium and low priority. They will move heaven and earth to patch
vulnerabilities in the high category. They’ll do their best to patch
vulnerabilities in the medium category. And they’ll get around to the low
priority vulnerabilities when and if they get the chance to do so, with no
guarantee that they will be able to patch the lows at any time in the future.
Now, let’s assume the organization suddenly starts receiving
regular SBOMs from their suppliers for one quarter of the software products
installed on their network – that is, for 250 products. And let’s say the
average SBOM identifies 50 components in a product, which is much less than the
average of around 150. Since we’re assuming that components have on average one
vulnerability, as do the products themselves (i.e., the “first party” code in
the product), this means the organization now knows of 250 X 50 = 12,500 new
vulnerabilities, plus the 1,000 vulnerabilities they already knew about. And,
if our previous assumption that every vulnerability is found on ten devices
still holds, this means there are now (12,500 + 1,000) * 10 = 135,000
devices to be patched (one hopes the organization has an automated solution
for this, but, as any experienced patch management person can tell you, it’s
impossible to automate 100% of patch applications).
Remember, we’ve already stipulated that the organization considers
their current load of 1,000 unpatched vulnerabilities to be overwhelming,
meaning there is no good way the security team will ever be able to patch all of
them (this is because more vulnerabilities are being identified every day. By
the time they patch the high priority vulnerabilities and maybe some of the
medium ones, they’ll have a new batch of highs, which demands their attention
before anything else). What will their security team say (that’s printable in a
family blog post) when their boss lets them know that, because of this new
thing called SBOMs, their number of unpatched vulnerabilities has grown from 1,000
to 13,500? And keep in mind that the above analysis makes assumptions that
could turn out to be too low. The actual number could be much higher.
This shows that the only thing that, once SBOMs start being
distributed to end user organizations (by which I mean all organizations except
those whose primary business is developing software or maufacturing intelligent
devices) at the needed frequency (i.e., with every new major or minor version),
the only thing that will save vulnerability management teams in end user
organizations will be developing a
strict methodology for triaging the vulnerabilities they face. While there are many such methodologies, one that a
lot of organizations are using now is to triage vulnerabilities according to
their EPSS scores or whether they’re in
CISA’s KEV
catalog. Both of these measures are based at least in part on whether actual
exploits based on the vulnerability in question are taking place.
So now, the security team that thought 1,000 unpatched
vulnerabilities was the workload from hell is faced with 13,500 unpatched
vulnerabilities. What will they do? They’ll do the only two things they can do:
a) ask for more head count, and b) re-prioritize. They might do the latter by
first prioritizing the 13,500 vulnerabilities into high, medium and low; then
they’ll take just the highs and re-prioritize them into high/high, medium/high
and low high. Finally, they’ll concentrate on the high/highs and perhaps a few
of the medium/highs. Those are probably all they’ll ever be able to patch.[i]
What happens when we introduce VEX into this picture? You
might think that, since VEX has to do with exploitability, it won’t have much
to add to the EPSS score or the Kevin catalog. However, if you think that,
you’re wrong. This is because there are two types of exploitability. One is the
VEX sense, which applies just to a single product and is binary; it answers the
question, is CVE-2023-12345 exploitable in product A version 2.5? The answer is
yes or no. Note that it doesn’t apply to more than one version of a product, unless
the statement applies to a version range (which is currently only an option in
the CycloneDX VEX format).
The second type is the exploitability discussed above,
measured using (among other things) the EPSS score and KEV catalog. This is a
percentage (0 to 100%) and applies to a vulnerability, not a product; indeed,
this type of exploitability applies across all products.[ii]
It answers the question, given the most up-to-date information on exploitation
and availability of exploit code, how likely is it that CVE-2023-12345 can be
used to compromise any software product containing that vulnerability?
Let’s get back to the original question: If a security team needs
to prioritize the vulnerabilities it faces and just patch the most exploitable
ones (in the second sense of the word), how can VEX help them do that, since it
deals with a different type of exploitability? More succinctly, can VEX provide
much help at all?
My answer to this question might surprise you: VEX can only
play a secondary role in addressing the problem faced by the intrepid security
team we’ve described above. The team needs to use EPSS, KEV, etc. to prioritize
the vulnerabilities it’s addressing; VEX won’t help with that job.
However, VEX can play a secondary role that could in
theory (but not in practice today, unfortunately) make the team’s problem more
manageable than it would be otherwise. Remember that at least 90% (and maybe
95-97%) of vulnerabilities identified in components of a product that are
listed in an SBOM will not be found to be exploitable in the product itself. If
our team could learn, for every version of every software product that their
organization uses, which component vulnerabilities (i.e., vulnerabilities in
components listed in an SBOM, that can be found in the NVD or another
vulnerability database) are not exploitable in every software product used by
the organization, they could then remove those versions from the list of those
to be patched for the non-exploitable vulnerability.
Let’s illustrate this with an example:
1.
Suppose the security team has identified and
prioritized 1,000 vulnerabilities for devices on their network, as well as
12,500 vulnerabilities for components listed in an SBOM for one of the software
product/versions installed on those devices (as in the case we described
above). They have identified 250 “high/high” vulnerabilities, based on EPSS
scores and the KEV catalog. Using our estimate of ten vulnerable devices for
each vulnerability, this means the team needs to patch 2,500 devices, in order
to patch every instance of each high/high vulnerability.
2.
Their initial patching push for the month will
focus just on the high/highs. They will start by identifying every device on
their network(s) on which a software product subject to at least one of the
high/high vulnerabilities is installed. They will plan to patch each of these
until they have patched every one of the high/high vulnerabilities, on every
device on which that vulnerability is found. In other words, when they have
finished the initial phase of the effort, they will have patched every instance
of every high/high vulnerability.
3.
However, what if the organization were
receiving, from the supplier of each of the 1,000 software products installed
on their network, a revised SBOM corresponding to at least every major new
version of the product? And on top of that, what if the supplier were sending a
new VEX document whenever they have determined that a particular component vulnerability
is exploitable (or not) in the product in which the component is included (hey,
a guy can fantasize, can’t he)?
4.
Moreover, suppose each of the organization’s
software suppliers is really thorough and, within say one month of the release
of a new version of a product, they have determined the exploitability status
(in the VEX sense, of course) of each component vulnerability in that version.
5.
Finally, after aggregating all the VEX
information it has received about each product/version installed on its
network, the security team realizes that 95% of all their component
vulnerabilities are not exploitable in the full product; of course, this is
probably close to the average number.
6.
When the security team realizes this, there will
be great rejoicing. After all, they had at first thought they’d have to patch
2,500 machines just in order to patch all the high/high vulnerabilities.
However, it turns out they only need to patch 5% of those machines, or 125. A
nice improvement, n’est-ce pas?
7.
Furthermore, instead of having to patch 135,000
devices in order to eliminate every vulnerability in their backlog from the
network, the team “only” has to patch 135,000 * 5% = 6,750 machines. That’s
still a lot, of course, but what’s most striking is that the security team is
now at the point where they can seriously discuss completely eliminating their
vulnerability backlog, vs. at best patching all of the high/high and medium/high
vulnerabilities.
Of course, I’ve already admitted that this vision is a
fantasy. Neither SBOMs nor VEX documents are being provided to end users
whenever needed (an SBOM is needed at least whenever a new version of the software
is released, while a VEX document is needed whenever the exploitability status
of a component vulnerability has changed).
Moreover, I don’t see that changing until both types of
documents are regularly being released.[iii]
VEX without SBOM is irrelevant, while SBOMs without VEX will never be accepted,
because of the huge false positive vulnerability problem. This isn’t a
chicken-or-egg problem. After the VEX problem is solved and the naming problem is
substantially ameliorated (through changes to the NVD, primarily), SBOMs will start
to be distributed in the required frequency and quantity. Fortunately, progress
is being made on both fronts.
[i] There’s another way to triage their work, and that’s according to the importance of the device on which the software is running. For example, say that CVE-2023-12345 is at the top of the prioritization list, but it’s found on three types of devices. One is the computers used by the team that plans next month’s lunchroom meals (are there still lunchrooms?). The second is computers used in the finance department, although not for trading. The third type is devices that control the manufacturing process which brings in literally every dollar the company makes.
It might seem like a no-brainer that even though the team would start their day by patching CVE-2023-12345, they would first patch the manufacturing machines, then the machines used by finance and lastly the menu people. However, even that isn’t a sure thing. There are all sorts of reasons why individual machines will be patched in a different order than others, which have nothing to do with prioritization by CVE or type of machine.
[ii] Of course, in both types of exploitability, the words “other things being equal” apply. For example, if a device is isolated from the internet and located in a room that requires a retina scan and a smart key to enter, it’s likely it will be close to unexploitable in either sense. This means that in both types of exploitability, protections that need to be separately applied and will not always be in place in all environments should be ignored.
[iii] I
have given up on the idea that VEX documents will ever be released in
the quantities that will ultimately be needed, which might be tens or even
hundreds of VEX documents for a single product every day. This is why I’m
focusing on the idea of an API by which users can query VEX information from a server
maintained by a supplier, freeing the supplier from having to push out any VEX
documents at all.
No comments:
Post a Comment