For many years, I’ve believed that the big problem with the
NERC CIP cybersecurity standards isn’t that they’re not strong enough to be
effective. Au contraire, the problem is that they’re very effective, but
that effectiveness comes at a huge cost. In other words, although CIP
compliance has protected the North American Bulk Electric System (BES) very
effectively, the cost of compliance paid by NERC entities (in both time and
dollars) has been huge.
One of the main reasons for this is that there are a few
very prescriptive CIP requirements that require the NERC entity to document
compliance in a huge number of single instances, usually regarding individual
Cyber Assets. My biggest examples for this statement by far have always been
CIP-007 R2 for patch management and CIP-010 R1 for configuration management.
Both require tracking and documenting a huge number of individual instances of
compliance across many individual devices.
But there’s a difference between the two requirements.
CIP-010 R1 compliance requires that the NERC entity track every change to every
Cyber Asset in scope (mostly BES Cyber Assets that are part of medium or high
impact BES Cyber Systems), as well as whether it was authorised, whether the
baseline configuration information was updated after the change, etc. All these
items are important, but if this requirement (and its requirement parts) were
rewritten as a risk-based one, the change wouldn’t impact security very much,
yet it would greatly ease the burden of compliance.
CIP-007 R2 is a different story. It’s very hard to say that all
the steps it requires are important for security. For example, is it really necessary
for a NERC entity to check every 35 days to see if a new security patch is
available for every software or firmware product (as well as every version
of every software or firmware product) that is installed on any Cyber Asset
within their ESP?
For some very important products that often issue patches,
the answer is yes. However, for other products that seldom if ever issue patches,
the answer is probably no. If this requirement and its parts were rewritten as
risk-based, NERC entities could save a huge amount of time and money every
year, with little impact on security.
The heart of CIP-007 R2 is the requirement to apply every
security patch issued by every supplier of any software or firmware product
installed in the ESP. Why is that important? Of course, it’s important because
software “develops” vulnerabilities over time and they (at least in many cases)
need to be mitigated.
But what about vulnerabilities for which a patch is never
issued? For example, what if your organization uses a custom software product
for which there is no third-party “supplier”? Maybe the person who developed
the software left the scene long ago; no current staff member has the slightest
idea how to make any change to the software, let alone issue a patch for it.
Does CIP-007 R2 require you to remove the software and/or rewrite it? No, it
says nothing about this situation at all.
Even more importantly, what about an end of life product
like Windows XP? I believe there are some devices in use in OT environments that
are still running XP. Of course, Microsoft is no longer issuing regular patches
for it – they stopped doing that in 2014. If a serious vulnerability appears,
your EOL status may turn into SOL, and your entire ESP might be compromised.
But don’t worry, you won’t be in violation of CIP-007 R2!
These two examples show clearly that the real risk isn’t not
applying a patch, but not mitigating a serious software vulnerability. R2 needs
to be replaced with a risk-based requirement for vulnerability management. That
would require the NERC entity to:
1.
Track vulnerabilities for the software products installed
in their ESP;
2.
Identify those vulnerabilities whose exploitation
might lead to serious damage to the BES, if those vulnerabilities were being exploited
in the wild (perhaps as indicated by the vulnerability’s presence in CISA’s Key
Exploited Vulnerabilities catalog);
3.
Make sure the supplier of the vulnerable product
plans to issue a patch for the vulnerability soon; and
4.
Apply the patch soon after it becomes available.
Of course, if the supplier of the software product doesn’t
issue a patch for the vulnerability, the entity should ideally remove that
product from their environment and replace it with a more secure product.
Unfortunately, that’s easier said than done, since OT software products are
often “one of a kind” and can’t be easily replaced, if at all. In such cases,
the best course of action is to take whatever steps are possible to mitigate the
increased risk caused by the unpatched vulnerability.
There are other problems with a vulnerability management
requirement as well. For example, I’ve learned
that suppliers of intelligent devices often don’t report vulnerabilities for
their devices. Since almost all vulnerabilities (including almost all CVEs) are
reported by the manufacturer of the device or the developer of the software, this
means that the first step above will be impossible in the case of many devices:
any search of a vulnerability database such as the NVD will identify zero
vulnerabilities.
However, the device may be very far from being
vulnerability-free, as was the case for the device described in the post I just
linked: A search of the NVD for that device (which is mostly used in military and
other high assurance environments, by the way) will yield zero applicable CVEs,
when in fact a researcher (Tom Pace of NetRise) estimates there are at least
40,000 firmware vulnerabilities in it.
Without much doubt, the biggest reason why automated
vulnerability management isn’t possible today is that the US National Vulnerability Database, which up
until early this year was the most widely used vulnerability database in the
world, has been experiencing serious problems that call into question its
continued functioning. I’ve described these problems in many recent posts,
including this
one.
Until the NVD fixes its problems and restores its
credibility (which I strongly doubt will happen anytime soon), vulnerability
management will remain what it is now: more of an art than a science. Unfortunately,
this means it’s currently impossible to develop a usable vulnerability
management requirement for NERC CIP.
Does this mean the NERC CIP community has no choice but to
continue to invest big resources in compliance with CIP-007 R2, while realizing
sub-optimal returns on its investment? I’m afraid so. It would be nice if
someone would draft a SAR (Standards Authorization Request) to replace CIP-007
R2 with a true vulnerability management requirement. However, even if a new
requirement were drafted and approved, I don’t think there’s a realistic way to
implement it, given the current problems with the NVD and the ongoing problems
with the CPE identifier.[i]
Currently, the two big focuses of the CIP standards development
effort are virtualization and the cloud. The former process is close to
complete, but the latter
is just getting started. As the “cloud CIP” effort gathers momentum, it’s
likely to suck a lot of the air out of other concerns like CIP-007 R2[ii].
And it’s likely that the cloud effort will take years to be completed.
Of course, if a NERC entity wants to free itself from having
to comply with CIP-007 R2 and CIP-010 R1, one way to do this will be to move as
many medium and high impact BES Cyber Systems as possible to the cloud, as soon
as it becomes “legal” to do so. But if a lot of other NERC entities have the
same idea, some would argue that the result will be that the BES is less
secure, not more secure. Wouldn’t that be ironic?
Are you a vendor of current or
future cloud-based services or software that would like to figure out an
appropriate strategy for selling to customers subject to NERC CIP compliance? Or
are you a NERC entity that is struggling to understand what your current
options are regarding cloud-based software and services? Please drop me an
email so we can set up a time to discuss this!
Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.
[i] The
OWASP SBOM Forum will soon release a proposal for finally addressing the problems
with CPE, by expanding use of the purl
identifier to proprietary software (which purl does not currently address). Stay
tuned to this blog!
[ii] Since
any new “cloud CIP” requirements will need to be completely risk-based, it is
likely that CIP-007 R2 will be replaced with a risk-based vulnerability management
requirement for use by systems deployed in the cloud. However, it’s also
likely that any new CIP standards will have two “branches”: one for use by
cloud systems and one for use by on-premises systems. The latter will bear a close
resemblance to the current CIP standards, if they aren’t virtually identical.
Thus, a prescriptive patch management requirement is likely to remain part of the
CIP standards for years, even after the “cloud problem” in CIP is finally
fixed.
No comments:
Post a Comment