Monday, June 6, 2016

A Conversation with Lew Folkerth on CIP-002-5 R1

I have been writing about problems with CIP-002-5.1 R1 and Attachment 1 for more than three years, starting with this post in April 2013; I’m sure I have written more than 50 posts on various aspects of this topic. I recently came indirectly to have a discussion on this problem with Lew Folkerth of RF. I had just written a post discussing the two main ways I’m using the term “enforceable” regarding CIP v5 and v6 and how I suspect that, in what I call the strict sense of the term, many of the requirements are in fact unenforceable. The day after the post appeared, Lew emailed me with a question on what I had said.

My answer led to a back-and-forth discussion that I think will be interesting to the three or four of you who also have nothing better to do than to ponder the subtleties of the wording of CIP-002-5 R1 - the most important requirement in CIP v5 and v6, and IMHO the most poorly written. In this post, I will reproduce my conversation with Lew almost in full (removing only some extraneous comments, and adding a few clarifying phrases to my statements. I have left Lew’s as he wrote them). I will add comments, which will all be in italics.

Lew’s Original Email:
This is where you lose me (referring to my post):

“For an example of this, consider the fact that CIP-002-5.1 R1 never requires the entity to document a methodology for identifying BES Cyber Systems. In fact, it never requires the entity to identify BCS in the first place, although that requirement is certainly implied by the fact that all the other requirements apply to BCS.”

CIP-002-5.1 R1 Part 1.1 states, “Identify each of the high impact BES Cyber Systems according to Attachment 1, Section 1, if any, at each asset.”

What am I missing?

My Reply:
Good point, Lew! The short answer is the word “Identify” in R1.1-1.3 really means “classify”. You’re sent to Attachment 1 to do your BCS identification, but Attachment 1 just assumes you’ve already identified your BCS and you now need to classify them. Of course, R1 (or Attachment 1) itself gives zero guidance on identification (you have the discussion of the BROS in the Guidance, but there is nothing in R1 itself that would lead you to believe this has any applicability to the BCS identification process); this is why entities need to develop their own methodology for identification and classification of BCS.

(Tom’s later note) I believe there are two basic approaches to BCS identification. One is the “bottom up” approach, where the entity identifies Cyber Assets using that definition, then BES Cyber Assets according to that definition; finally, they aggregate BCAs into BCS (or just call each BCA a BCS, as I know some entities have done, perhaps out of an over-abundance of caution). The other is the “top-down” approach, where the entity uses the list of BES Reliability Operating Services performed by the asset in question to identify the BCS that fulfill those BROS. For a description of these two approaches (and how they can – and should – be combined), see this post from early 2015.

Lew’s Response:
  • CIP-002-5.1 R1 Part 1.1 requires an entity to identify each of the high impact BES Cyber Systems, if any, at each asset. The phrase “according to Attachment 1, Section 1” modifies the imperative verb “identify.” Think of it like this:
a)      Identify:
                                             i.            What: each of the high impact BES Cyber Systems, if any;
                                           ii.            How: according to Attachment 1 Section 1;
                                         iii.            Where: at each asset.
•       You are correct in saying that by the time you classify a BES Cyber System, you should have already identified it. You should do this based on the Glossary definition. You need to show that each BES Cyber System is one or more BES Cyber Assets, that the BES Cyber Assets are logically grouped, and that the logical grouping of BES Cyber Assets performs one or more reliability tasks.
•       While the Requirement does not discuss identifying BES Cyber Assets explicitly, this identification is strongly implied by the cascading Glossary definitions, and by the Purpose section of CIP-002-5.1: “To identify and categorize BES Cyber Systems and their associated BES Cyber Assets.”

My Response
Thanks, Lew. I’ll agree with you that the SDT combined “identify” and “classify” into “identify” in R1.1-1.2. But let’s think through the actual steps that are literally implied by this. As far as I can see, the only interpretation of R1 and Attachment 1 that complies with the wording is the following – and I don’t think any NERC entity would be happy following these steps, nor would any auditors be comfortable auditing them.

1.       The list of 6 asset types in R1 is like a FORTRAN FOR loop[i]. “For each of the following asset types, do 1.1-1.3.” Alternatively, you can interpret this as saying “For each of R1.1 to 1.3, iterate through each of the six asset types.”
2.       You start on 1.1. To comply with this, you have to first go to every control center you own, then every transmission substation, every plant, etc. For each of these, you first have to identify every BCS (using the nested definitions, as you said. Of course the Guidance talks about the BROS, but there is nothing in R1 to lead you to believe that you could actually use the BROS to identify BCS. So you have to assume you need to use what I call the bottom-up approach, starting with the definitions).
3.       At every one of the six asset types (regardless of classification, because as you know assets officially aren’t classified in CIP v5!), you will have to identify every Cyber Asset (assuming you know what “programmable” means), determine whether it’s a BCA, then group those into BCS. Then you have to run each of those BCS through the criteria in Section 1 of Attachment 1, to see if it’s a High.
4.       I emphasize you have to do this for every BES asset you own – identify all Cyber Assets, then BES Cyber Assets, and finally BES Cyber Systems. It doesn’t matter whether the asset will eventually turn out to be High, Medium or Low impact. There is nothing in R1 or Attachment 1 that tells you that R1.1 applies just to High impact assets (and, of course, there is no such thing as High, Medium or Low impact assets, again according to the literal wording of R1 and Attachment 1). So an entity with 500 Transmission substations and 100 generating plants has to do this for all 600 of those assets, before it can determine what its High impact BCS are. Obviously, this is a huge task, but strictly following the wording leaves you no other choice.
5.       You have to do the same process for 1.2, except that you don’t have to redo the BCS identification step. You do have to run each BCS, that wasn’t already classified as High impact, through the Medium criteria, to identify Medium impact BCS.
6.       So far, you’ve had to identify BCS at all BES assets, then identify the ones that are High impact under R1.1 and Medium impact under R1.2. Of course, you are now left with a list of Low BCS; low BCS are simply all the BCS that aren’t High or Medium. Of course, this conflicts with the statement that a list of Low BCS isn’t required. But you will inevitably develop a complete list of your Low BCS by going through the above process. That is, if you’re inclined to comply with the literal wording of R1 and Attachment 1.
7.       You now go to R1.3, which sends you to Section 3 of Attachment 1. This says you’re to identify “BES Cyber Systems not included in Sections 1 or 2 above…” The clear implication of this phrase is that you started out Attachment 1 with your complete list of BCS at all assets, then subtracted out the Highs and Mediums, leaving your list of Low BCS. How could you possibly literally comply with the wording without doing that?
8.       Of course, R1.3 itself says you’re supposed to identify each “asset containing a Low BCS”, and as we all know, this doesn’t require literally identifying all BCS at Low assets. In practice, what everyone I know is doing to comply with this requirement part is starting with their total list of BES assets (ones that correspond to one of the six asset types in R1), removing the High and Medium impact ones, and considering the rest as Lows.[ii] Then, they identify BCS at the High assets and consider them High BCS. Finally, they identify BCS at the Medium assets and consider them Medium BCS. At the Low assets, they don't do anything more than list them, complying with R1.3. But this ignores the fact that the phrases “High impact asset” and “Medium impact asset” have no explicit meaning in R1, since assets themselves are never officially classified. The criteria in Attachment 1 are for BES Cyber Systems only, although they do refer to assets and thus are widely believed to be classifying the assets themselves.

So strictly speaking, you’re right about “identification” being required by R1.1 to 1.3 (although the implied identification process, based on the Cyber Asset and BCA definitions only, constrains the entity to completely ignore what I call the top-down, BROS-based approach), since that is the word used. But those requirement parts also require classification of BCS, since there is no other place in R1 that you are called on to do that.[iii]

Of course, as I’ve said repeatedly in my blog, most of this doesn't cause problems in real life, since entities (and almost all auditors) are treating the v5 process as just a variation of CIP-002-3 R1-3. In other words, you first classify the assets as High, Medium or Low impact (in v3, there were just two asset classifications: Critical and non-Critical; but it’s the same idea). Then you identify your BCS at those assets (and BCS are somewhat equivalent to Critical Cyber Assets in v3). This process works fairly well, and it is how - consciously or unconsciously - almost all entities identified and classified their BCS; it is also, I believe, how most regional auditors would describe the process of BCS identification and classification. So it's great that there is widespread agreement on this; but it's not so great that this process doesn't correspond at all with the wording of R1 and Attachment 1! 

I do want to point out that one good innovation in v5 is that it does allow classification based on Facilities (lines, transformers, etc), especially in substations (in fact, that is the correct way to classify BCS at substations, since criteria 2.4 to 2.8 all have “Facilities” as their subject, not “Substations”). On the other hand, I have yet to find an entity that is actually classifying based on Facilities, except in the case of mixed-ownership substations, where it’s almost imperative that you do this.

So the big problem with CIP-002-5.1 R1 is that it is impossible to comply with the requirement as written – without devoting a substantial fraction of the entity’s total revenues to visiting all BES assets and identifying all BCS there. Fortunately, both NERC entities and auditors have come up with a workable “interpretation” of this requirement that allows them to have an agreed-upon methodology for BCS identification and classification. But that interpretation doesn’t correspond to the wording of R1 and Attachment 1. IMHO, this is why CIP-002-5.1 R1 is completely unenforceable in the strict sense; and it may possibly make all of v5 and v6 unenforceable as well. If NERC ever wants to have v5 and v6 be strictly enforceable, it will have to completely rewrite R1 and Attachment 1 (although I doubt even that would make v5 and v6 completely enforceable. There’s a separate issue that I won’t get into here, but that I’ve discussed before); but I don’t see that ever happening.

Lew’s Final Reply
I agree with everything except the last statement, about the strict enforceability of CIP-002-5.1. I do agree with your post from last Wednesday where you said we are unlikely to ever find out. And if we do find out that CIP-002-5.1 isn’t strictly enforceable, I expect governmental authority will step in and require changes that are enforceable.

Tom: Some people have suggested that the idea that CIP v5 may be “unenforceable in the strict sense” means that an entity would actually have to file a lawsuit contesting a CIP fine, and win the suit, before there would be any practical impact of that unenforceability. Of course, if that were to happen, it would be many years in the future.

However, I believe CIP v5’s unenforceability will be evident long before that happens. If auditors come to believe that a requirement isn’t strictly enforceable, they’re not likely to write PVs for it in the first place, unless the entity has simply not bothered to comply with the requirement at all. For example, I see no way an entity will ever receive a PV for not correctly applying the phrase “associated with” in Section 2 of Attachment 1, since there is no definition of the phrase. More broadly, I don't see any PVs being issued for not following proper procedure in identifying and classifying BCS under R1. Since the only procedure that can be said to be "proper" is the one described above - which almost literally no NERC entity would be willing to go through - there is simply no way the courts would uphold a fine based on such a "violation".

I see it as the job of the Regions to ensure the reliability of the BES; RF’s official mission statement is “ReliabilityFirst preserves and enhances bulk power system reliability and security across 13 states and the District of Columbia.” To support this mission we have a wide range of tools, including various forms of outreach, risk assessments, and controls evaluations. These non-CMEP tools are, of course, backed up by the full power of the CMEP if that is needed.

As for your analysis of the language that says the “bottom-up” method is strongly implied by the Standard, I agree. But we’re training the audit teams to accept the “top-down” as well as the “hybrid” approach (top-down with a cross check of systems bottom-up style). Since, for a large entity, the “bottom-up” approach can take orders of magnitude more resources to complete than “top-down,” this just makes good business sense.

(Tom) I totally agree it makes good business sense. But it is unfortunate that R1 (which should really be broken into at a minimum three or four requirements) requires so much interpretation and “gap-filling” in order to comply with it.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

[i] The fact that I had to refer to FORTRAN to explain my point, and the fact that Lew understood the reference, is a rather sad commentary on both of our ages.

[ii] If an asset that isn’t High or Medium doesn’t contain any control systems, it won’t be an “asset containing a Low impact BCS”, and it won’t be High, Medium or Low. It will just be out of scope.

[iii] I have pointed out several times that one of the biggest problems with the wording of CIP-002-5 is that all of the process of identifying cyber assets in scope for the standards is compressed into one requirement; in contrast, in CIP v3 the three main tasks – developing a risk-based assessment methodology, applying that methodology to identify Critical Assets, and identifying Critical Cyber Assets, which are defined as Cyber Assets “essential to the operation of” the Critical Assets – each had their own requirement. This makes both compliance and auditing much easier.

Last fall, I identified 15 separate steps that are required to identify and classify BES Cyber Systems, which is of course the main goal of CIP-002-5. A huge amount of the current confusion about asset identification in CIP v5 could have been eliminated had each of these been addressed in its own requirement. Instead, they were all collapsed into a single requirement, and at least seven or eight (probably more) of the tasks are never actually called out in that requirement; instead, they are simply implied by definitions. It’s as if the Standards Drafting Team was temporarily replaced by a group of haiku masters, for whom compression of as much meaning into as few words as possible was the highest goal. Of course, compression of meaning should be the absolutely last goal of enforceable standards, not the first. Clarity should be first, not somewhere near the end.

No comments:

Post a Comment