In my last
post,
I continued to applaud the CIP Modifications standards drafting team for
breaking out of what had seemed to me to be a dead-end program for introducing
virtualization into the CIP standards. I pointed out that they accomplished
this “breakout” by coming up with two Great Ideas (this is my term. It’s not in
the NERC Glossary). But I also pointed out – very nicely, of course – that one
of their two Great Ideas was not likely to succeed unless another change was
also made.
This
particular Great Idea was that the prescriptive requirements in the
currently-enforced CIP standards (like CIP-007 R2, CIP-005 R1 and CIP-010 R1) should
all be written non-prescriptively, in the manner of CIP-007-6 R3. The question
I asked in the post was “How will these requirements be audited?” And I wasn’t
alone in asking this question. In the SDT’s webinar on virtualization recently,
SDT members said that – or some variation of it – was the most frequently
submitted question.
And well it
should be. The problem is that NERC’s auditing procedures – like those of
almost any other organization that does audits – ultimately come down to
determining whether the entity has or hasn’t done specific things that are
required. This of course works fine for prescriptive requirements, where the
whole point of the audit is determining whether or not the entity has done – or
not done, as the case may be – the particular things specified in the
requirement.
But this
doesn’t work so well for two reasons. First, it doesn’t work well when there is
ambiguity in a prescriptive requirement, or in a definition that underlies
it. For example, CIP-010 R1.1.3 requires
the entity to include in its baseline any “custom software” that is installed
on the system. Of course, a program written for a particular purpose definitely
counts here. But how about other pieces of software like scripts? There isn’t a
definition of “custom software” that can settle this question, so an auditor
can’t issue a PNC (or at least it won’t be upheld) in a case where an entity
isn’t including scripts in their baselines.
The other
reason is because a requirement is non-prescriptive. Three good examples of
this are CIP-014 R1, R4 and R5. In a
post
last year, I discussed two entities (from the same Region) that both told me
the same story: They had been dinged by an auditor for not taking specific
steps to protect transformers located in their substations in scope for
CIP-014. Their mistake was taking the words of these three requirements
literally, since all three only talk about protecting the substation itself,
not any equipment located in it.
Unfortunately,
if you take away all the prescriptive requirements that have ambiguous language
in CIP, as well as all the non-prescriptive requirements, there aren’t many
requirements left! What this means is there are very few current CIP
requirements for which there isn’t any question on how they will be audited.
Of course,
despite this fact, I haven’t heard too many horror stories about an entity and
an auditor being completely at loggerheads about a particular issue, to the
extent that the entity ends up getting a PNC or even just a very stern warning.
Why is this the case?
It certainly
isn’t the case because NERC provided lots of guidance for both the entities and
the auditors. NERC has tried to provide CIP guidance that would be in some way
“binding” for entities and auditors for many years, but in the end it has
always had to be removed
[i]. And for
good reason: There simply is no way, under the NERC Rules of Procedure, for
NERC itself to provide any document – other than the standards themselves -
that would guide the content of an audit. The only way to fix a hole or an
ambiguity in a requirement is to write a Standards Authorization Request to
draft a new requirement or revise an existing one
[ii]; and
that’s part of the CIP Modifications team’s current agenda.
No, the
reason auditing has worked fairly well, despite the many ambiguous and
non-prescriptive requirements in the currently-enforced CIP standards, is that
the Regions have stepped up and provided the “guidance” that NERC can’t
provide. They will almost never do this
in
writing, but you can usually get questions answered verbally (and one
Region takes this a step further, by actively offering “assist visits” to help
entities put in place good cybersecurity practices. And you can be quite sure
that those practices are compliant!).
But this is
by no means an ideal situation. Anytime an auditor speaks to you about a requirement
– or speaks in a Regional workshop – they will make it clear that what they say
is simply their personal opinion, not that of the Region (and perhaps not that
of the other auditors). And they mean this; I’ve heard multiple stories about
entities who were told one thing by an auditor at their office, but another
thing by an auditor (in one case it was the same one), when they came onsite to
do an actual audit. In cases where the auditors themselves are clearly confused
by a requirement, I think it’s highly unlikely that an entity will receive a
PNC for violating it, but I’m sure this wears on CIP compliance professionals
who have to live with this uncertainty.
Why am I
bringing all of this up now? Because the SDT is now proposing to make
prescriptive requirements like CIP-007 R3 and CIP-005 R1 non-prescriptive.
Non-prescriptivity is a good thing, but it needs to be accompanied by a
different audit regimen than NERC has now. With non-prescriptive requirements,
there’s no getting around the fundamental problem: If a requirement just mandates
that you do something like develop or implement a plan, but it doesn’t provide
a specific list of topics that you need to include in your plan, there is
nothing to audit – except whether or not you developed or implemented the plan
at all.
Of course,
if you clearly didn’t develop a serious plan, or you barely tried to implement
it, you could still be held in violation. But for anything else, like missing a
particular element in your plan that the auditor thinks should be there, you
can’t receive a violation – although the auditor can still give you a hard
time. Again, this situation is wearing on CIP compliance professionals. And to
be honest, if the SDT simply changes a few prescriptive requirements like
CIP-007 R2 to non-prescriptive ones, I think they’re going to have a hard time
getting this passed. I don’t think too many CIP professionals are out looking
for another anxiety-producing requirement. They may prefer just to keep the
prescriptive ones, simply because by now they know what the auditors want to
see, and have put all that in place. A prescriptive requirement may require a
lot of work, but at least there isn’t a lot of uncertainty with that work – the
old story of sticking with the devil you know, rather than the devil you don’t
know.
So what’s
the solution to this problem? I’m not at all advocating that the SDT give up
the idea of making CIP standards as non-prescriptive as possible, but I also
don’t want to see more CIP teams having to order Maalox™ by the case load.
The ideal
solution would be if the CIP enforcement process could move from one based on
auditing to one based on the Regions and the entities working together to
secure the BES. I gave some ideas of what I mean in
this
post, and you can be sure it will figure prominently in the book I’m writing.
But I will be quite honest and say I see just about zero chance of NERC making
radical changes in the CIP enforcement procedures anytime soon.
So is there
a Plan B? In other words, is there a way that the SDT could develop
non-prescriptive versions of current prescriptive requirements, that doesn’t
also require changes to the enforcement process? I do have a Plan B, and it’s
based on several observations.
First, non-prescriptive
requirements are definitely the way to go. However, just being non-prescriptive
isn’t enough.
Second, since
CIP v5 was implemented, all subsequent requirements that have been developed
(plus one requirement that was part of v5 itself) have been what I call “
plan-based”.
In these, the entity isn’t told to achieve an objective (which as I said above,
is always unachievable and unmeasurable, when it comes to cybersecurity), such
as “protect your assets against the threats attendant on use of Transient
Electronic Devices and Removable Media”. Instead, they are told to develop and
implement a
plan to manage risks attendant
on TEDs and RM. Completely protecting your assets against threats from TEDs and
RM isn’t achievable or measurable, but developing and implementing a plan is
definitely achievable. Does requiring a plan suffice to make a requirement
auditable?
No it doesn’t.
Just requiring a plan doesn’t make a plan-based requirement auditable in any
meaningful sense. If the requirement just mandates a particular type of plan - say
a supply chain cyber security risk management plan - an entity could put
together a really minimal plan, but as long as the plan addressed supply chain
cyber security risk management in some way, it would have to pass.
[iii]
So how can a
plan-based requirement be made auditable? The requirement (not just a “guidance”
document) needs to provide a list of elements that must be included in the plan
– and it’s best if these elements are threats that the plan should mitigate. CIP-010
R4 is my poster child for a good plan-based requirement. R4 reads “Each
Responsible Entity…shall implement…one or more documented plan(s) for Transient
Cyber Assets and Removable Media that include the sections in Attachment 1.”
Attachment 1
lists three types of devices that must be in scope for the plan, as well as
between two and five “topics” (my term) that must be addressed for each of the
three types. For example, under the Removable Media device type, there are two
topics: Removable Media Authorization and Malicious Code Mitigation. Each topic
lists 2-5 mitigations that must be included in the plan. For example, under
Removable Media Authorization, the entity is required to authorize use of
Removable Media (thumb drives) by both user and location. Under Malicious Code
Mitigation, the user is required to scan the RM for malicious code before using
it with a Medium or High impact BES Cyber System, as well as “Mitigate the
threat of detected malicious code”.
You might
ask, “How does CIP-010 R4 differ from a prescriptive requirement? It includes a
number of steps you need to take; that sounds pretty prescriptive to me.” I’m
glad you asked that question. Here’s how it differs:
- The individual mitigations aren’t prescriptive. For
example, the first mitigation under Malicious Code Mitigation is “3.2.1. Use method(s) to detect
malicious code on Removable Media using a Cyber Asset other than a BES
Cyber System or Protected Cyber Assets.” The methods aren’t specified, as
they would be in a prescriptive requirement.
- The “preambles” to the mitigations under most topics make
clear that the objective in each topic is to mitigate a particular threat.
For example, Section 3.2 reads “Malicious Code Mitigation: To
achieve the objective of mitigating the threat of introducing malicious
code to high impact or medium impact BES Cyber Systems and their
associated Protected Cyber Assets, each Responsible Entity shall…”
- Under most topics, it is made clear that the mitigations
listed aren’t the only ones that can be used to address the threat in
question. For example, the mitigations under Section 2.1 Software
Vulnerabilities Mitigation, read “Review of installed security patch(es); Review
of security patching process used by the party; Review of other
vulnerability mitigation performed by the party; or Other method(s) to
mitigate software vulnerabilities.”
To
summarize, I definitely support the SDT’s idea to make certain prescriptive
requirements non-prescriptive, in order to facilitate incorporating
virtualization into the CIP requirements. However, this is how I would proceed
with each requirement that is going to be rewritten (using CIP-007 R2 as an
example):
a) I
would state its objective, based on threat mitigation. For CIP 7 R2, I would
state the objective as “On a risk-adjusted
[iv] basis, mitigate
the threat of software vulnerabilities”.
b) I
would then probably refer the entity to an attachment (as CIP-010 R4 does),
rather than try to include everything in the requirement itself. But the
important thing is that the attachment has to be called out in the requirement;
if that doesn’t happen, then the attachment is just another piece of guidance,
with no special importance.
c) The
attachment might list a set of “sub-threats” to be addressed. Each sub-threat
would be in its own section, followed by one or more suggested mitigations (but
always saying in the last bullet point “Or any equally effective mitigation
strategy”). In this case, one of the sub-threats might be “The threat of
vulnerabilities in commercial software.” Probably the only mitigation listed
would be “A patch management program, including patch source identification,
patch discovery, patch assessment, patch application or mitigation plan
development, mitigation plan implementation and regular review, etc.” Another sub-threat
would be “The threat of vulnerabilities found in software developed by the
entity itself”; in this case, one mitigation would probably be identification
and implementation of a secure SDLC.
Unfortunately,
I also have a poster child for a flawed plan-based requirement: CIP-013 R1. I
used to
think
that this was the perfect plan-based requirement, because of its ZEN-like
simplicity:
R1.
Each Responsible Entity shall develop one or more documented supply chain cyber
security risk management plan(s) for high and medium impact BES Cyber Systems.
The plan(s) shall include:
1.1.
One or more process(es) used in planning for the procurement of BES Cyber
Systems to identify and assess cyber security risk(s) to the Bulk Electric
System from vendor products or services resulting from: (i) procuring and
installing vendor equipment and software; and (ii) transitions from one
vendor(s) to another vendor(s).
[v]
Let me
critique CIP-013 R1 in terms of the three criteria for a good plan-based CIP
requirement, which I listed above:
a) R1
starts out with a statement of the objective of the requirement, development of
a supply chain cyber security risk management plan. Notice this doesn’t use the
word “threat”, but since a risk management plan should certainly talk about
threat mitigation, this is OK.
b) R1
doesn’t refer to an attachment, but that’s probably because of c).
c) While
R1.1 does list three threats
[vi] that
need to be addressed in the plan, these are quite general. Each of these three threats
(and especially the first one) will of course have many sub-threats. If each of
these threats listed 5-10 sub-threats that must be mitigated in the plan
[vii], then
CIP-013 R1 would receive “Tom’s Seal of Approval for an Auditable Plan-based
Requirement”.
So since
CIP-013 R1.1 doesn’t list specific sub-threats that need to be mitigated in the
supply chain cyber security risk management plan, it doesn’t get my coveted
Seal of Approval – which means it really isn’t auditable, as long as the entity
produces a plan that could be credibly called a supply chain cyber security
risk management plan. I won’t belabor this point now, since I’m going to have a
new post on CIP-013 soon.
Any opinions expressed in this blog post are strictly mine
and are not necessarily shared by any of the clients of Tom Alrich LLC.
If you would like to comment on what you have read here, I
would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that
if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or
challenges like what is discussed in this post – especially on compliance with
CIP-013. And if you’re a security vendor to the power industry, TALLC can help
you by developing marketing materials, delivering webinars, etc. To discuss any
of this, you can email me at the same address.
[i] I
drafted a short history of NERC’s efforts to provide real guidance for
ambiguities in CIP requirements in writing this post, but I’ve decided it
doesn’t need to be in the current post, which is already very long. I’ll post
it separately soon.
[ii] There
is the Request for Interpretation
process. In this process, an entity requests an interpretation; it is drafted
and balloted (multiple times) by a NERC drafting team; then it is approved (or
not, as happened with two CIP Interpretations in 2012) by FERC, and finally it’s
implemented. But this takes almost as much time as the SAR route, and in any
case an Interpretation can never modify the words of a requirement – just
interpret the current wording. The auditing problems we’re talking about come
to pass because the current wording of a requirement doesn’t give enough
information to an auditor to audit it.
[iii] I’m
being a little too absolute here. An auditor who was confronted with a minimal
plan like this would still be justified in giving a PNC for this requirement,
since they can always use “professional judgment”. If an entity has developed
what is clearly an inadequate plan, the auditor can exercise that judgment to
point out that it is grossly inadequate, and therefore doesn’t constitute a
serious plan at all. However, if the entity has actually developed more than a
minimal plan, but has still left out some topic that the auditor thinks should
be in the plan, the auditor can’t give a PNC in this case, and if they do, it
most likely won’t be upheld.
[iv] By “risk-adjusted”,
I mean that the same controls don’t have to apply to all systems or devices in
scope, regardless of the degree of BES risk each one poses. For example, a
relay that operates the circuit breaker for a 345 kV line poses a higher risk
to the BES than one which operates a 138 kV line. For CIP-007 R2, the entity
might decide that a 30-day patching cycle is needed for the former, but a 60-day
cycle is perfectly adequate for the latter (of course, when I talk about risk
here, it has nothing to do with impact level. You could have BCS with three or
five different levels of risk, all installed at a single Medium-impact
substation, so the BCS would all be Mediums); this is, of course, not allowed
in CIP-007 R2 now.
[v] Of
course, R1.2 goes on to list six specific things that need to be included in
the plan. But the plan doesn’t consist of these six things; the six things are
there because they were ordered by FERC in
Order
829. The heart of R1 is R1.1.
[vi] They’re
called risks here, not threats. Since most people would do the same thing, I’m
not going to vehemently object to this, but I’ve developed my own definitions
of what threats, vulnerabilities and risks are and how they relate to each
other. The book I’m writing now is carefully following these definitions. But
since the CIP-013 SDT (and FERC in Order 829) is using a different idea of risk
than mine, that’s fine. But I will keep pointing out that I don’t agree with
that idea, since it leads to confusion when you start talking about the amount
of risk that a threat poses (using the more popular idea of risk, you would
have to start talking about the amount of a risk that a risk poses, which of
course is very confusing).
[vii] Of course,
each of the three threats will have a lot of serious “sub-threats”, perhaps hundreds.
How will the SDT come up with a manageable list of around ten? One way to do
this is to aggregate them into a higher level, and just list those higher-level
sub-threats. Another is to just take the ten most serious sub-threats and list
them – on the idea that the ten of them might account for most of the risk
posed by the overall threat.
