I have been
writing about problems with CIP-002-5.1 R1 and Attachment 1 for more than three
years, starting with this
post in April 2013; I’m sure I have written more than 50 posts on various
aspects of this topic. I recently came indirectly to have a discussion on this problem
with Lew Folkerth of RF. I had just written a post
discussing the two main ways I’m using the term “enforceable” regarding CIP v5
and v6 and how I suspect that, in what I call the strict sense of the term,
many of the requirements are in fact unenforceable. The day after the post
appeared, Lew emailed me with a question on what I had said.
My answer
led to a back-and-forth discussion that I think will be interesting to the
three or four of you who also have nothing better to do than to ponder the
subtleties of the wording of CIP-002-5 R1 - the most important requirement in
CIP v5 and v6, and IMHO the most poorly written. In this post, I will reproduce
my conversation with Lew almost in full (removing only some extraneous
comments, and adding a few clarifying phrases to my statements. I have left
Lew’s as he wrote them). I will add comments, which will all be in italics.
Lew’s Original Email:
This is
where you lose me (referring to my post):
“For an
example of this, consider the fact that CIP-002-5.1 R1 never requires the
entity to document a methodology for identifying BES Cyber Systems. In fact, it
never requires the entity to identify BCS in the first place, although that
requirement is certainly implied by the fact that all the other requirements
apply to BCS.”
CIP-002-5.1
R1 Part 1.1 states, “Identify each of the high impact BES Cyber Systems
according to Attachment 1, Section 1, if any, at each asset.”
What am I
missing?
My Reply:
Good point,
Lew! The short answer is the word “Identify” in R1.1-1.3 really means
“classify”. You’re sent to Attachment 1 to do your BCS identification, but
Attachment 1 just assumes you’ve already
identified your BCS and you now need to classify them. Of course, R1 (or
Attachment 1) itself gives zero guidance on identification (you have the
discussion of the BROS in the Guidance, but there is nothing in R1 itself that
would lead you to believe this has any applicability to the BCS identification
process); this is why entities need to develop their own methodology for
identification and classification of BCS.
(Tom’s later note) I believe there are two
basic approaches to BCS identification. One is the “bottom up” approach, where
the entity identifies Cyber Assets using that definition, then BES Cyber Assets
according to that definition; finally, they aggregate BCAs into BCS (or just
call each BCA a BCS, as I know some entities have done, perhaps out of an
over-abundance of caution). The other is the “top-down” approach, where the
entity uses the list of BES Reliability Operating Services performed by the
asset in question to identify the BCS that fulfill those BROS. For a
description of these two approaches (and how they can – and should – be combined),
see this
post from early 2015.
Lew’s Response:
- CIP-002-5.1 R1 Part 1.1 requires an entity to identify
each of the high impact BES Cyber Systems, if any, at each asset. The
phrase “according to Attachment 1, Section 1” modifies the imperative verb
“identify.” Think of it like this:
a) Identify:
i.
What: each of the high impact BES Cyber Systems,
if any;
ii.
How: according to Attachment 1 Section 1;
iii.
Where: at each asset.
•
You are correct in saying that by the time you classify a BES Cyber
System, you should have already identified it. You should do this based on the
Glossary definition. You need to show that each BES Cyber System is one or more
BES Cyber Assets, that the BES Cyber Assets are logically grouped, and that the
logical grouping of BES Cyber Assets performs one or more reliability tasks.
•
While the Requirement does not discuss identifying BES Cyber Assets
explicitly, this identification is strongly implied by the cascading Glossary
definitions, and by the Purpose section of CIP-002-5.1: “To identify and
categorize BES Cyber Systems and their associated BES Cyber Assets.”
My Response
Thanks, Lew.
I’ll agree with you that the SDT combined “identify” and “classify” into
“identify” in R1.1-1.2. But let’s think through the actual steps that are literally
implied by this. As far as I can see, the only interpretation of R1 and
Attachment 1 that complies with the wording is the following – and I don’t
think any NERC entity would be happy following these steps, nor would any
auditors be comfortable auditing them.
1. The list of 6 asset types in R1 is like
a FORTRAN FOR loop[i].
“For each of the following asset types, do 1.1-1.3.” Alternatively, you can
interpret this as saying “For each of R1.1 to 1.3, iterate through each of the
six asset types.”
2. You start on 1.1. To comply with this,
you have to first go to every control center you own, then every transmission
substation, every plant, etc. For each of these, you first have to identify
every BCS (using the nested definitions, as you said. Of course the Guidance
talks about the BROS, but there is nothing in R1 to lead you to believe that
you could actually use the BROS to identify BCS. So you have to assume you need
to use what I call the bottom-up approach, starting with the definitions).
3. At every one of the six asset types
(regardless of classification, because as you know assets officially aren’t
classified in CIP v5!), you will have to identify every Cyber Asset (assuming
you know what “programmable” means), determine whether it’s a BCA, then group
those into BCS. Then you have to run each of those BCS through the criteria in
Section 1 of Attachment 1, to see if it’s a High.
4. I emphasize you have to do this for every BES asset you own – identify all
Cyber Assets, then BES Cyber Assets, and finally BES Cyber Systems. It doesn’t
matter whether the asset will eventually turn out to be High, Medium or Low
impact. There is nothing in R1 or Attachment 1 that tells you that R1.1 applies
just to High impact assets (and, of course, there is no such thing as High,
Medium or Low impact assets, again according to the literal wording of R1 and
Attachment 1). So an entity with 500 Transmission substations and 100
generating plants has to do this for all 600 of those assets, before it can
determine what its High impact BCS are. Obviously, this is a huge task, but
strictly following the wording leaves you no other choice.
5. You have to do the same process for 1.2,
except that you don’t have to redo the BCS identification step. You do have to
run each BCS, that wasn’t already classified as High impact, through the Medium
criteria, to identify Medium impact BCS.
6. So far, you’ve had to identify BCS at
all BES assets, then identify the ones that are High impact under R1.1 and
Medium impact under R1.2. Of course, you are now left with a list of Low BCS;
low BCS are simply all the BCS that aren’t High or Medium. Of course, this conflicts
with the statement that a list of Low BCS isn’t required. But you will
inevitably develop a complete list of your Low BCS by going through the
above process. That is, if you’re inclined to comply with the literal wording
of R1 and Attachment 1.
7. You now go to R1.3, which sends you to
Section 3 of Attachment 1. This says you’re to identify “BES Cyber Systems not
included in Sections 1 or 2 above…” The clear implication of this phrase is
that you started out Attachment 1 with your complete list of BCS at all assets,
then subtracted out the Highs and Mediums, leaving your list of Low BCS. How
could you possibly literally comply with the wording without doing that?
8. Of course, R1.3 itself says you’re
supposed to identify each “asset containing a Low BCS”, and as we all know, this
doesn’t require literally identifying all BCS at Low assets. In practice, what
everyone I know is doing to comply with this requirement part is starting with
their total list of BES assets (ones that correspond to one of the six asset
types in R1), removing the High and Medium impact ones, and considering the
rest as Lows.[ii] Then, they identify BCS at the High assets and consider them High BCS. Finally, they identify BCS at the Medium assets and consider them Medium BCS. At the Low assets, they don't do anything more than list them, complying with R1.3. But this ignores the fact that the phrases “High impact asset” and “Medium
impact asset” have no explicit meaning in R1, since assets themselves are never
officially classified. The criteria in Attachment 1 are for BES Cyber Systems
only, although they do refer to assets and thus are widely believed to be
classifying the assets themselves.
So strictly
speaking, you’re right about “identification” being required by R1.1 to 1.3
(although the implied identification process, based on the Cyber Asset and BCA definitions
only, constrains the entity to completely ignore what I call the top-down,
BROS-based approach), since that is the word used. But those requirement parts
also require classification of BCS, since there is no other place in R1 that
you are called on to do that.[iii]
Of course,
as I’ve said repeatedly in my blog, most of this doesn't cause problems in real life,
since entities (and almost all auditors) are treating the v5 process as just a
variation of CIP-002-3 R1-3. In other words, you first classify the assets as
High, Medium or Low impact (in v3, there were just two asset classifications:
Critical and non-Critical; but it’s the same idea). Then you identify your BCS
at those assets (and BCS are somewhat equivalent to Critical Cyber Assets in v3). This process works fairly well, and it is how - consciously or unconsciously - almost all entities identified and classified their BCS; it is also, I believe, how most regional auditors would describe the process of BCS identification and classification. So it's great that there is widespread agreement on this; but it's not so great that this process doesn't correspond at all with the wording of R1 and Attachment 1!
I do want to
point out that one good innovation in v5 is that it does allow classification
based on Facilities (lines, transformers, etc), especially in substations (in
fact, that is the correct way to classify BCS at substations, since criteria
2.4 to 2.8 all have “Facilities” as their subject, not “Substations”). On the
other hand, I have yet to find an entity that is actually classifying based on
Facilities, except in the case of mixed-ownership substations, where it’s
almost imperative that you do this.
So the big
problem with CIP-002-5.1 R1 is that it is impossible to comply with the
requirement as written – without devoting a substantial fraction of the entity’s
total revenues to visiting all BES assets and identifying all BCS there.
Fortunately, both NERC entities and auditors have come up with a workable
“interpretation” of this requirement that allows them to have an agreed-upon
methodology for BCS identification and classification. But that interpretation
doesn’t correspond to the wording of R1 and Attachment 1. IMHO, this is why CIP-002-5.1
R1 is completely unenforceable in the strict sense; and it may possibly make
all of v5 and v6 unenforceable as well. If NERC ever wants to have v5 and v6 be
strictly enforceable, it will have to completely rewrite R1 and Attachment 1
(although I doubt even that would make v5 and v6 completely enforceable. There’s
a separate issue that I won’t get into here, but that I’ve discussed
before); but I don’t see that ever happening.
Lew’s Final Reply
I agree with
everything except the last statement, about the strict enforceability of
CIP-002-5.1. I do agree with your post from last Wednesday where you said we
are unlikely to ever find out. And if we do find out that CIP-002-5.1 isn’t
strictly enforceable, I expect governmental authority will step in and require
changes that are enforceable.
Tom: Some people have suggested that the
idea that CIP v5 may be “unenforceable in the strict sense” means that an
entity would actually have to file a lawsuit contesting a CIP fine, and win the
suit, before there would be any practical impact of that unenforceability. Of
course, if that were to happen, it would be many years in the future.
However, I believe CIP v5’s unenforceability
will be evident long before that happens. If auditors come to believe that a
requirement isn’t strictly enforceable, they’re not likely to write PVs for it
in the first place, unless the entity has simply not bothered to comply with
the requirement at all. For example, I see no way an entity will ever receive a
PV for not correctly applying the phrase “associated with” in Section 2 of Attachment
1, since there is no definition of the phrase. More broadly, I don't see any PVs being issued for not following proper procedure in identifying and classifying BCS under R1. Since the only procedure that can be said to be "proper" is the one described above - which almost literally no NERC entity would be willing to go through - there is simply no way the courts would uphold a fine based on such a "violation".
I see it as
the job of the Regions to ensure the reliability of the BES; RF’s official mission
statement is “ReliabilityFirst preserves and enhances bulk power system
reliability and security across 13 states and the District of Columbia.” To
support this mission we have a wide range of tools, including various forms of
outreach, risk assessments, and controls evaluations. These non-CMEP tools are,
of course, backed up by the full power of the CMEP if that is needed.
As for your
analysis of the language that says the “bottom-up” method is strongly implied
by the Standard, I agree. But we’re training the audit teams to accept the
“top-down” as well as the “hybrid” approach (top-down with a cross check of
systems bottom-up style). Since, for a large entity, the “bottom-up” approach
can take orders of magnitude more resources to complete than “top-down,” this
just makes good business sense.
(Tom) I totally agree it makes good business
sense. But it is unfortunate that R1 (which should really be broken into at a minimum three
or four requirements) requires so much interpretation and “gap-filling” in
order to comply with it.
The views and opinions expressed here are my own and don’t
necessarily represent the views or opinions of Deloitte Advisory.
[i]
The fact that I had to refer to FORTRAN to explain my point, and the fact that
Lew understood the reference, is a rather sad commentary on both of our ages.
[ii]
If an asset that isn’t High or Medium doesn’t contain any control systems, it
won’t be an “asset containing a Low impact BCS”, and it won’t be High, Medium
or Low. It will just be out of scope.
[iii]
I have pointed out several times that one of the biggest problems with the
wording of CIP-002-5 is that all of the process of identifying cyber assets in
scope for the standards is compressed into one requirement; in contrast, in CIP
v3 the three main tasks – developing a risk-based assessment methodology,
applying that methodology to identify Critical Assets, and identifying Critical
Cyber Assets, which are defined as Cyber Assets “essential to the operation of”
the Critical Assets – each had their own requirement. This makes both
compliance and auditing much easier.
Last fall, I identified 15 separate steps
that are required to identify and classify BES Cyber Systems, which is of
course the main goal of CIP-002-5. A huge amount of the current confusion about
asset identification in CIP v5 could have been eliminated had each of these
been addressed in its own requirement. Instead, they were all collapsed into a
single requirement, and at least seven or eight (probably more) of the tasks
are never actually called out in that requirement; instead, they are simply
implied by definitions. It’s as if the Standards Drafting Team was temporarily
replaced by a group of haiku masters,
for whom compression of as much meaning into as few words as possible was the
highest goal. Of course, compression of meaning should be the absolutely last
goal of enforceable standards, not the first. Clarity should be first, not
somewhere near the end.
No comments:
Post a Comment