I now believe there are five main problems
that make it hard, if not impossible, for NERC entities to maintain CIP
compliance while deploying certain CIP-related workloads in the cloud. Each of
these problems is unique and requires a unique solution. I will discuss
each of them, as well as their possible solutions, in separate posts soon. They
are (in approximate order of importance):
1.
The “EACMS
problem”.
2.
The medium impact
renewables Control Centers problem.
3.
The SaaS/BCSI problem.
4.
The high and medium
impact utility Control Centers problem.
5.
The low impact Control
Centers problem.
Contrary to what many people
think, it isn’t true that the NERC CIP requirements in any way “forbid” use of
the cloud by assets that fall under the purview of CIP. The current requirements
had their genesis in the years after 2008, when FERC approved CIP version 1. At
that time, the cloud was very new. The idea that assets that control the power
grid might at some point be deployed in the cloud was almost unthinkable. Therefore,
the original CIP requirements said nothing about the cloud, because nobody
thought it was likely this would ever become an issue.
To this day, there is no mention
of the cloud in any CIP requirement or definition. This means that any NERC
entity that wishes to outsource their entire OT environment to the cloud can do
so without fear of being in direct violation of any CIP requirement – as long
as they don’t mind receiving a boatload of Notices of Potential Violation
anyway. How can this happen?
It can happen because, as any NERC
compliance person well knows, remaining in compliance with any NERC requirement
means being able to provide appropriate evidence of compliance. Non-binding
suggestions for that evidence are usually found in the “Measures” column of the
Requirement. The general rule of NERC compliance is: “If you didn’t document it,
you didn’t do it.”
Of course, proving compliance with
any cybersecurity standard always requires some sort of evidence. However,
NERC CIP differs from other standards in that the NERC entity needs to be
prepared to provide evidence that they were compliant with a CIP requirement in
every instance in which compliance was required; it doesn’t matter whether
the systems in question are deployed on premises, in the cloud, or both.
The problem with this is that the
Measures were all developed for the on-premises use case only (except for the
Measures shown in Requirement CIP-004-7 Part R6.1 and Requirement CIP-011-3 Part
R1.2, which were developed with both cloud and on-premises systems in mind).
For many CIP Requirements and
Requirement Parts, evidence for compliance in the cloud does not pose a problem,
since the Requirement merely states the objective to be achieved; usually, the
objective is implementing a policy or procedure. For example, CIP-005-7
Requirement R1 Part R1.5 requires the entity to “Have one or more methods for
detecting known or suspected malicious communications for both inbound and
outbound communications.”
The Measures section of that
Requirement Part reads, “…documentation that malicious communications detection
methods (e.g. intrusion detection system, application layer firewall, etc.) are
implemented.” In other words, the NERC entity needs to document how they
complied with this Requirement Part, but they are allowed to choose the
technology or technologies they implement to achieve the objective. For
on-premises systems, the evidence might be output produced by an IDS or an
application layer firewall. For the cloud, it might be an audit report for ISO
27001 certification or FedRAMP authorization, which describes how the CSP has
complied with a requirement to detect malicious inbound communications. A CIP
auditor might consider both of these to be evidence of compliance with
CIP-005-7 Requirement R1 Part R1.5.[i]
However, some NERC CIP
Requirements are not objectives-based, but instead mandate that particular
actions be performed without regard to achieving a particular objective. For
example, CIP-007-6 Requirement R2 Part R2.2 requires the NERC entity to “At
least once every 35 calendar days, evaluate security patches for applicability
that have been released since the last evaluation from the source or sources
identified in Part 2.1.”
Among other things, this tightly
packed Requirement Part mandates that the NERC entity check with the vendor of
every software product installed on any system within their high or medium
impact Electronic Security Perimeter every 35 days, to determine whether a new
security patch is available for their product. The entity then needs to
determine whether the patch is applicable to their product or environment. If
it is applicable, they need to either apply the patch or develop a mitigation
plan.
A security patch almost always
fixes one or more software vulnerabilities (often identified using a CVE
number). However, according to the well-respected vulnerability intelligence
firm VulnCheck, only 1.1% of publicly known vulnerabilities are observed
being exploited by attackers.
Does this mean that an
organization not subject to CIP compliance could safely deploy just 1.1% of the
security patches that are made available for their systems? Since active
exploitation is always an ex post facto measure, waiting for your system
to be exploited before applying the patch is probably not a great strategy.
However, there are measures of active exploitation available, such as the EPSS score and
CISA’s Known
Exploited Vulnerabilities (KEV) catalog, that all
organizations can use to prioritize their patching efforts.
Since virtually all large
organizations today have a big backlog of patches to apply but nowhere near
enough bandwidth to apply them all, they must triage them. They need to accept the
fact that they won’t be able to apply all patches, and instead divide them into
three groups: patches they definitely will apply, others they definitely won’t
apply, and still others they will apply only if time permits.
However, a NERC entity subject to
compliance with CIP-007 is not allowed to consider information about active
exploitation (or anything else) in deciding whether to apply a security patch. Requirement
CIP-007-6 Part 2.2 does not allow NERC entities with high or medium impact BES
Cyber Systems to ignore any patch because it has very low risk of exploitation.
It doesn’t matter whether the patch mitigates any significant security risk or
not; if it is available and it applies to the NERC entity’s configuration, it
must be applied.[ii]
I don’t honestly know whether CSPs
follow the NERC CIP approach and try to apply every available security patch
regardless of whether it mitigates any risk, or whether they triage patches
based on the risk of exploitation of the vulnerability(ies) that are mitigated by
the patch. However, if they take the latter approach, they are not complying
with the letter of CIP-007 R2, even though I believe a risk-based approach is
best for almost any cybersecurity problem.
But there is a much bigger problem
that prevents platform CSPs from producing compliance evidence for prescriptive
CIP requirements in the cloud like CIP-007-6 R2, CIP-010 R1, and CIP-005 r1: Evidence
for these three requirements must be produced on an individual device
basis, because the requirements can only be complied with on the device level.
And since single cloud workloads migrate from system to system and data center
to data center all the time, a single BCS might reside on hundreds or even
thousands of individual devices during a three-year audit period. There is
simply no way a platform CSP could ever produce the full set of evidence for
any of the prescriptive CIP requirements, even if they were inclined to do so.
However, the platform CSPs could
potentially comply with CIP requirements that just mandate policies or
procedures. If they do comply, it will probably be with three non-negotiable
positions[iii]:
1.
They will not provide CIP
compliance evidence to individual NERC entities, but only to NERC itself (or
perhaps a third party designee). It will be up to NERC to share that evidence
with any NERC entity that can demonstrate a need for it.
2.
The evidence the platform
CSP provides will include selections from audit reports for ISO 27001
certification, FedRAMP authorization, and SOC 2 Type 2 audits. It will be up to
the individual NERC entities to decide what to make of that information; NERC
will never “certify” CSPs for use by NERC entities (indeed, any attempt to do
so might be considered an antitrust violation).
3.
While platform CSPs
are open to answering other questions besides the ones included in frameworks like
the NIST CSF, ISO 27001 and FedRAMP, the questions will need to be agreed on
beforehand by NERC entities. NERC will administer the questions and evaluate
any evidence provided by the CSPs before distributing it to the NERC entities. However,
this will not be a “compliance audit”, since neither NERC nor FERC has any
jurisdiction over the CSPs.
While I think this NERC “audit” of platform CSPs needs to be part of whatever final set of solutions comes out of the current CIP standards drafting effort, I also don’t see any way this can be part of the CIP standards themselves. What I’m describing here will require changes to the NERC Rules of Procedure, even though drafting this will probably require a NERC team separate from the current Project 2023-09 Risk Management for Third-Party Cloud Services team, as well as one or two possible additional years of effort, before all the pieces for the full solution are in place. That’s why the NERC CIP community needs to think now about partial solutions that will allow NERC entities that wish to do so to make as much use of the cloud as possible, without requiring a complete rewrite of the CIP standards.
Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.
[i] Currently,
a NERC CIP auditor is unlikely to accept an audit report as compliance evidence,
since there is nothing in the Rules of Procedure that allows for acceptance of audit
results - other than CIP audit results - as evidence of compliance with a CIP
Requirement. A permanent fix to this problem will probably require changing the
Rules of Procedure, although as a temporary measure a “CMEP Practice Guide”,
which is created by the NERC auditors to address an area of ambiguity in the
current requirements, would probably be sufficient.
[ii] Many
tools now ease the burden of compliance with this Part, although there is
always a large amount of care and feeding involved with CIP-007 R2 compliance,
regardless of the degree of automation.
[iii]
This section just applies to the major platform CSPs, not to SaaS providers. I
think the latter should be subject to a NERC “audit” as well, but it should be
very different from that of the platform CSPs, since their situation is very
different.
No comments:
Post a Comment