A number of
my posts have started with a mention of something I read in EnergySec’s weekly newsletter. This isn’t
because I’m being paid to promote the organization (although I think very highly
of them), but because the newsletter often sees things that nobody else does
(including me).
One feature
of the newsletter is the Blog Roll, where Brandon Workentin, the very
knowledgeable EnergySec staff member who edits the newsletter, discusses recent
blog posts of interest to the ICS security audience. Two weeks ago, Brandon
wrote about my post
on the issue of whether VOIP phones should be considered BES Cyber Systems. He
pointed out that, while nominally about VOIP, the post “is really about the
need, in some cases, for entities and auditors to interpret a requirement ‘as
if it had been written differently.’" As usual when Brandon writes about
my posts, he hit the nail on the head; in fact, I don’t think I could have
summarized the post any better than he did. This is why I always look forward
to reading what he says about my posts: so I can learn what I meant when I
wrote them!
In the
penultimate paragraph of that post, I concluded (in reference to how VOIP
systems should be treated under CIP-002 R1) that “…both the
entity and the auditor need to insert two words into the (BES Cyber Asset) definition (Note:
I could have gone on to say “in order to have a BCA definition that properly addresses
VOIP phones and other ‘support systems’”).” I continued, “In other cases
(and I will illustrate one case in another post that is coming soon), both the
entity and the auditor need to ignore some of the wording in the
requirements.” Well, the post that is “coming soon” is here! I will now discuss
how one particular requirement in CIP v5/v6 – specifically, CIP-002-5.1 R1 with
Attachment 1 – can only be properly followed by an entity or an auditor if part
of the wording is ignored. But first some background.
In late
April 2013, FERC surprised most in the industry (and certainly me), when they
issued their NOPR
saying they intended to approve CIP version 5 and have it supersede CIP v4,
which was due to come into effect on April 1, 2014. Since they had approved v4
in April 2012 (which was also a surprise to most of us in the industry), I had
been focusing on v4 in my blog posts, because after all v4 was now the law of
the land, and v5 was still struggling to get the votes it needed to get
approved (and there was a serious question whether it would ever be approved).
After the
NOPR came out, I decided to write a series of posts on the CIP v5 standards, starting
at the beginning with CIP-002-5.1. In this
first post, I sat down with CIP-002 R1 and Attachment 1 and tried to piece
together exactly how a NERC entity was to identify its BES Cyber Systems and
then classify them as High, Medium or Low impact, along with identifying its
assets that contain Low impact BCS.
But a funny
thing happened as I worked on this post; no matter how much I scrutinized the
wording of R1 and Attachment 1, I simply couldn’t put together a logical chain
of steps that would produce the required result. I came to a point where no
amount of logic could bridge the wide chasm caused by the contradictory nature
of the wording I found in R1 and Attachment 1. I concluded that this chasm
needed to be closed, in order for CIP v5 to be a set of standards that NERC
entities could comply with and that NERC auditors could audit.
The primary
problem[i] I
identified in this post was in the wording of Attachment 1. This seems to be
written under the implicit assumption that the entity has already identified
BES Cyber Systems at all of its BES
Assets – High, Medium and Low impact – before it even starts to classify them.
In Sections 1 and 2 of Attachment 1, the entity is told to identify those BCS
that meet the High and Medium impact criteria, respectively. And Section 3,
where Lows are identified, seems to follow suit when it tells the entity to
identify “BES Cyber Systems not included in Sections 1 or 2 above…” This
clearly implies that the entity had a comprehensive BCS list at the beginning
of this process. But there’s only one problem with this: The entity isn’t
required to identify BCS at Low assets, so it never had a comprehensive list to
start from (and therefore it can’t identify BCS “not included in Sections 1 or
2”). In fact, Requirement Part 1.3 of CIP-002 R1 (which “sends” the entity to
Section 3 of Attachment 1 to figure out how to identify Low impact assets) says
“A discrete list of low impact BES Cyber Systems is not required.”
This
discovery led to my writing a series of posts over the summer of 2013,
advocating that CIP-002-5.1 R1 and Attachment 1 be rewritten to bridge this
chasm. Since I knew that NERC wouldn’t undertake this task on their own, I put
my hopes in FERC – that they would order NERC to do this when they approved CIP
v5. In fact, I even wrote
out my proposed revised wording, and submitted that to FERC during the
comment period on v5.
Of course,
FERC didn’t take me up on my suggestion when they approved v5 in Order 791
on November 22, 2013 (50 years to the day – in fact, almost to the hour - after
the assassination of President Kennedy in 1963. Just coincidence, you say?
Well….). Instead, they ordered the four changes that led to CIP v6, none of
which addressed this issue. I then pinned my
hopes on the idea that NERC would find some way to “interpret” CIP-002 so
that it would make sense, even though the actual wording wouldn’t change.
NERC did
issue some “Lessons Learned” in 2014 and 2015 that tried to resolve ambiguities
(although not the big one I was concerned about), but these were all withdrawn.
But NERC came up against the reality that the Rules of Procedure only allow one
way to change a standard: write a Standards Authorization Request, seat a
Standards Drafting Team, and get to work drafting the new standard (which takes
easily 3-4 years to produce results). And there is only one official way to
interpret a standard: go through the Request for Interpretation process (which
takes easily 2-3 years, with no guarantee of ultimate success. FERC remanded
the last two CIP RFIs that were presented to it in 2012, although there may be
a test soon when EnergySec’s RFI on Criterion 2.1 in Attachment 1 of
CIP-002-5.1 is presented to them).
In the end,
NERC admitted there needed to be a new version of CIP (although the fact that
it is a new version seems never to be
stated publicly. Instead, the drafting team to this day is called the “CIP
Revisions” SDT, presumably to demonstrate that NERC has a great sense of humor),
and indeed the new SDT started work last spring. The results of their work will
be implemented in pieces over probably the next 2-6 years, although the
fundamental problem I’m concerned with will most likely never be addressed at
all.[ii]
You may wonder
why, if there is such a basic contradiction in the wording of CIP-002-5.1 R1,
the whole edifice of CIP v5 hasn’t come crashing down. After all, R1 can very
easily be said to be the fundamental requirement in CIP v5 and v6 (and it will
be in v7 as well). This is the requirement where the entity figures out what
all the other requirements apply to. If the entity is lucky, their list of
applicable assets and cyber assets is very small. If they’re not lucky, there
is a huge list, and the entity had better figure on spending some significant
portion of their total annual budget on CIP compliance for at least the next
5-10 years (a number of entities are doing exactly this. In fact, I know of one
medium-sized entity that has allocated over ten percent of their entire annual revenues to cyber security and NERC CIP
compliance, at least for this year).
So why
didn’t CIP v5 collapse? There is certainly a lot of confusion over this
requirement, and there continues to be confusion today; in my opinion, this
confusion alone probably delayed
many entities’ implementation of v5 for at least a year. But NERC entities and
NERC auditors seem to be in fairly broad agreement now on how to comply with
R1, as well as on what constitutes non-compliance with the requirement.
How did this
agreement come about, given that applying strict logic doesn’t allow a single
consistent interpretation of the wording of CIP-002 R1 and Attachment 1? It’s
actually very simple: people reverted back to the overall framework for the
asset identification process in CIP v1 – v4, even though that wasn’t directly supported
by the language in CIP-002-5.1. To briefly (briefly for me, anyway!) summarize
what happened:
- The bright-line criteria in Attachment 1 made their first
appearance in CIP-002
version 4. If you read the v4 criteria, you will recognize the
predecessors to most of the High and Medium impact criteria in v5. In v4,
the criteria applied directly to assets, not cyber assets or BCS. BES
assets that met one of the v4 criteria were Critical Assets and thus in
scope for CIP v4; those assets that didn’t were out of scope for CIP v4
altogether (of course, there was no distinction between High and Medium
impact assets. An asset was either critical or it wasn’t in scope for CIP
v4 at all). Naturally, there were no criteria applying to Low impact
assets, although some assets that would have been Critical Assets under v4
ended up as Lows under v5 (notably blackstart assets and Facilities).
- When the SDT started work on CIP v5 after completing v4
(the same SDT developed v2, v3, v4 and v5, although there were a number of
personnel changes over the four years the team was in existence), they
more or less imported the criteria directly from v4, even though in theory
the whole purpose of Attachment 1 had changed. In CIP-002 version 5,
Attachment 1 in theory provides criteria for identifying BES Cyber
Systems, not Critical Assets as in v4. But instead of finding new criteria
that applied to cyber systems, the SDT decided essentially to reuse the
asset-based criteria from v4 for the High and Medium criteria in v5.
However, they “front-ended” the criteria with language that tried to make
them conform with the new purpose of Attachment 1 – to classify BES Cyber
Systems.
- In retrospect, doing this was a big mistake. The SDT was
using asset-based criteria to classify BCS, but the wording of Attachment
1 (and some of the wording of R1) sounded like the BCS were really being
classified on their intrinsic BES impact. Question: How could the SDT have
developed meaningful, measurable criteria that actually applied to BCS,
not assets/Facilities? Answer: They would have had to bring in some
measure of how important each BCS was to the BES.
In CIP v1-4, a Critical Cyber Asset was
defined as one that is “essential to the operation of” a Critical Asset – so it
was the impact on the asset that was important, not on the BES itself. In order
to achieve the stated goal of CIP-002-5.1 R1, the v5 criteria (in Attachment 1)
should have ranked the intrinsic “essentialness” of each BCS to the BES itself,
not to an asset. I admit this would have been very hard to do. But of course,
the fact that it would have been hard simply reflects a point that I argued
with SDT members in 2011 and 2012: It doesn’t make much sense to talk about the
intrinsic impact of a BCS on the BES.[iii]
Rather, the impact of a BCS is almost always only through the asset or Facility
it controls or supports. It is those assets or Facilities, not the BCS
themselves, that should be classified, as was the case in v1-4. As it is, I
will demonstrate below that literally the entire NERC community has reverted to
the v1-4 method of classifying assets, not BCS, even though this involves
ignoring a lot of the wording in CIP-002-5.1 R1 and Attachment 1.
- I think the SDT made this mistake because they knew it was
now verboten[iv]
to require entities to identify BCS at Low impact assets[v];
so they had to let the entity simply identify Low assets, not Low BCS.
This meant they had to make all the Attachment 1 criteria (High, Medium
and Low) apply to assets, not BCS. If they didn’t do this, how could an
entity possibly identify its Low assets, since it hasn’t previously
identified its High and Medium assets? Unfortunately, the SDT at the same
time tried to maintain the fiction that Attachment 1 was really about
classifying BCS, not assets, which is why the literal wording of
Attachment 1 is in direct contradiction to the way that 99% of NERC
entities and auditors are interpreting it (and the way that IMHO the SDT
members intended it to be interpreted).
- What the SDT should have done was go back to the CIP v1-4
model: First the entity identifies its most important assets in scope.
These were Critical Assets in v1-4, but High or Medium assets/Facilities
in v5 (and in v5, the entity uses the criteria in Attachment 1 to classify
High vs. Medium assets). Next, the entity identifies the Critical Cyber
Assets (in v1-4) or Medium or High BES Cyber Systems (in v5) associated
with those assets (with High or Medium BCS defined as those associated
with/located at High or Medium assets/Facilities respectively). Finally,
the entity subtracts its Medium and High assets from its total list of BES
assets; the remaining assets are Lows (of course, there is no requirement
to identify BCS at Lows). This wouldn’t have been terribly elegant, but it
would have been logically consistent.
- However, even though the sequence I just described in the
previous paragraph isn’t in the actual wording of CIP-002-5.1, it is in
fact how literally every entity (that I know of) is complying with CIP
v5/v6, and it is how literally every auditor (again, that I know of) is
auditing CIP-002-5.1 R1. Many, if not most, of the entities, and probably
more than a few auditors, are quite unaware that the compliance
methodology they are following (or auditing against) isn’t in the actual
wording of the standard. But that doesn’t really matter. There isn’t much
confusion over this issue now, since this consensus has developed in spite
of the wording of R1 and Attachment 1 (although there was a lot of
confusion over these and other parts of CIP v5 in 2014, when many entities
lost part or all of a year in their v5 compliance efforts, as they tried
to figure out what the requirements actually meant, and waited for
definitive guidance from NERC, which never came).
To finish my
long sermon, this is another case where both entities and auditors cannot
simply follow the strict wording of the CIP v5/v6 standards in order to comply
with and audit them. In the VOIP case in my earlier post, I stated that what
almost all entities and auditors are doing is implicitly inserting two words
into the BES Cyber Asset definition – when they do that, confusion over how to
classify VOIP and other “support systems” melts away. In the case of CIP-002 R1
and Attachment 1, I have just tried to show that the only way to overcome the
logical inconsistency of the wording is for entities and auditors to ignore a
lot of the actual wording and substitute alternate wording – and they have done
that with a remarkable degree of unanimity.
So what’s
the big deal with all of this? Since neither of these issues is causing
entities to be out of CIP compliance (or to devote a lot of resources to
something which they have mistakenly identified as required for compliance),
what’s the problem? Aren’t these both simply dead issues?
One problem
is that – in my opinion – the need to either supplement or ignore some of the
wording of CIP v5 (and especially in CIP-002 R1, the fundamental requirement of
CIP v5 and v6) makes the standards unenforceable in what I call the strict
sense: If an entity is fined for not properly identifying BCS and challenges
that fine on the grounds that the wording of CIP-002-5.1 R1 is contradictory, I
think the fine will be thrown out by any judge who spends ten minutes reading
the requirement. But since no NERC entity has actually ever challenged a CIP
fine in court, I admit this is a fairly theoretical issue.[vi]
The real
problem is that sometimes entities take the words of CIP-002 R1 and Attachment
1 literally, and end up mis-identifying BCS and assets for that reason. In my
next post, I will provide an example that I recently came across, where this is
exactly what happened.
The views and opinions expressed here are my own and don’t
necessarily represent the views or opinions of Deloitte Advisory.
[i]
I actually pointed out two fundamental problems with CIP-002-5.1 R1 in the
post. The first (which I said was of lesser importance) had to do with what I
thought at the time was inconsistent use of the terms “asset” and “Facility”. I
later learned that this wording is actually not inconsistent, although it is
terribly confusing.
In practice, very few entities – and perhaps very few auditors – understand
that these two terms actually refer to different things. This without doubt has
led many entities to over-identify Medium BCS at Transmission substations,
although a few large Transmission entities have told me that the greatly
increased effort that would be required, to properly take the word “Facilities”
into account to reduce the number of BCS, outweighs the benefit that would be
derived from having fewer Medium BCS in the first place.
[ii]
I haven’t even advocated that the SDT take up the issue discussed in this post.
It would require a huge effort and multiple ballots, and this effort alone
would probably push the final implementation of v7 back another year or so. I
have the greatest admiration for this SDT, but they have a huge amount on their
plate as it is, and I am becoming convinced that they will never succeed in
addressing one of the biggest items on that plate (as long as the CIP standards
remain basically prescriptive). More on that topic coming soon to a blog near
you.
[iii]
In fact, I remember a discussion on an SDT call in 2012, when I asked the
chairman of the SDT to name a cyber asset that directly impacted the BES - not
indirectly through an asset or Facility. He came up with “leak detectors”, a device
I hadn’t heard of. Evidently, these sit directly on a line and are not located within
a substation or other asset. However, to require that all cyber assets be
evaluated for their BES impact, not their impact on an asset/Facility, solely
because there are one or two BCS that actually do directly impact the BES, is
the textbook example of the tail wagging the dog. Yet this is in fact the
stated purpose of CIP-002 R1: identify
and classify BES Cyber Systems based on their impact on the BES. Ironically,
those leak detectors wouldn’t be in scope for CIP v5/v6 anyway, since the only
BCS that are in scope are those located at one of the six asset types listed in
R1.
[iv]
Here is why it was forbidden: In 2009, the SDT put out a “concept
paper” that first introduced the ideas of BES Cyber System and the BES
Reliability Operating Services (although the latter were called “BES
Reliability Functions” in the paper). While I think both of these are good
concepts, I think that pretending that the impact of a BCS on an asset has
nothing to do with whether a Cyber Asset impacts the BES – while at the same
time including criteria in Attachment 1 that are entirely asset (or
Facility)-based – is the Fundamental Sin of CIP v5, and is the reason it will
never be enforceable in the strong sense.
[v]
To comply with the first draft of CIP-002 v5, which was posted for comment in
November 2011, the entity would literally have had to start their BCS classification
effort by identifying BCS at all BES
assets – High, Medium and Low impact. Then they would go through Attachment 1
and classify each BCS as High, Medium or Low impact. Of course, NERC entities
weren’t too excited about this since it meant identifying all Low BCS. The
first draft went down to resounding defeat, with none of the standards
receiving much more than a 20% positive vote (I wrote a post pointing this out
in December 2011. It was posted on a Honeywell blog that is no longer available
online. If you would like to see that post, send me an email at talrich@deloitte.com). At this point,
the SDT realized they had to make it explicit that BCS didn’t have to be
identified at Lows. Unfortunately, rather than completely rewriting CIP-002 R1
and Attachment 1 so that they took account of that new reality, the SDT kind of
tinkered around the edges but left the basic structure in place. The result was
the logically inconsistent R1 and Attachment 1 that we know and love so dearly
today.
[vi]
Although I do believe that if this ever happens, the result will be disastrous.
Think of what would happen if a judge suddenly invalidated CIP-002-5.1 R1, and
there was no longer a legally-approved way to identify and classify BES Cyber
Systems. In my opinion, this would effectively invalidate all of CIP v5 and v6
(and, by the time that happens, probably v7). What would NERC do then? Go back
to v3? Throw in the towel on regulating cyber security? Beats me.