Corrections to Yesterday's Post

Correction No. 1: In my post on the question of the "top-down" vs. "bottom-up" approaches to BES Cyber System identification, I stated "I actually know of only two auditor presentations that have squarely addressed the BCS identification issue.  Kevin Perry of SPP, in his webinar on CIP v5 last February, only talked about the top-down approach.  On the other hand, Joe Baugh of WECC, in his presentation linked above (starting on slide 40), only discusses what I call the bottom-up approach, and never even mentions the BROS."

Kevin informed me by email this morning that I didn't read his webinar narrative closely enough, since he does state in there that  he is open to the entity's using either approach; in fact, he suggests they use both, as I also suggest.

I further went on to say "Does this mean that SPP entities should use top-down, WECC entities should use bottom-up, and all other entities are simply SOL?  At the moment, I’d say yes..." Obviously, I was wrong to suggest SPP entities need to use top-down.  But I was also wrong to suggest WECC entities need to use bottom-up.  Just because Joe Baugh didn't mention top-down in his presentation that I referenced, this doesn't mean WECC won't allow either approach - or a blended one like I and SPP suggested.

However, my statement about other entities being SOL stands, and should be generalized. All NERC entities are SOL until there is some sort of comprehensive interpretation of CIP-002-5 R1 published; and I really think NERC should do it, not the individual regions.

Correction No. 2: This isn't a correction per se, but an enhancement suggested by the generation CIP compliance person that I mentioned in the first post.  He says:

The fix to SPP's concern about missing systems such as environmental is to apply criteria for misuse, misoperation, or failure to operate from the BCA definition to evaluation/classification of the BES Asset's systems.

So if the system's misuse, misoperation, or failure to operate adversely affects performance of a BROS within the 15 minute window and it has cyber assets, it is a BCS.

This should be written into the required Process and applied before identifying and classifying cyber assets.


The upshot of this is that, once an entity has done the top-down approach and come up with a list of BES Cyber Systems, it should then consider all of the other systems at the facility - i.e. the remainder of systems that weren't considered so far because they don't fulfill a reliability purpose. The entity should consider whether misuse, etc. of each of these systems can affect a BROS within 15 minutes. If so, the system should also be considered a BCS.  This provides a rule that would capture the environmental system discussed in the original post, which wouldn't be identified by the top-down approach since it doesn't fulfill a BROS.   If someone were to fool the environmental system into thinking there had been an environmental excursion, the plant would shut down in ten minutes in this hypothetical scenario.

I should also point out that this works both ways.  I recommend that, once an entity has developed their list of BCS using the top-down approach, they should consider whether each BCS so identified can have a 15-minute impact on reliability if misused, etc. (as required by the definition of BES Cyber Asset).  If the answer to that question is no, then it shouldn't be listed as a BES Cyber System.  The top-down analysis by itself won't catch this, since it doesn't build up from BES Cyber Assets as the bottom-up approach does.

Of course, if you actually use both the top-down and bottom-up approaches, then you don't have to add these two qualifications just discussed; they are both part of the bottom-up approach.  In most cases, I don't think it's a huge burden to use both approaches.  The big exception is large generating stations, where there are often literally thousands of cyber assets, and where - as I discussed in the first post - there can be many Low impact BCS, which shouldn't have to be inventoried.


The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

Top-Down vs. Bottom-Up

This is a post I’ve wanted to write for a while.  I have been talking about the “top-down” and “bottom-up” approaches to BES Cyber System identification since last fall, but – as with everything else I write – my views have evolved since that time.  So this post is hopefully my definitive take on this topic, and I can get on to others I want to write[i].

Before I start, I want to point out that there is another use of the words “top-down” and “bottom-up” in regards to CIP version 5.  This is exemplified by Dr. Joe Baugh of WECC, who has used (in the CIP-002 presentation found at this link, slide 20) the words to mean how one approaches CIP-002-5 R1 in the first place: either by evaluating an inventory of BES assets (substations, etc) against the criteria in Attachment 1 (“top-down”), or by evaluating an inventory of BES cyber assets against those same criteria (“bottom-up”). 

Joe says either approach is acceptable, but entities will find the latter to be much more burdensome, since they will have to first evaluate all of their cyber assets as BES Cyber Assets.  This means they’ll have to have a complete inventory of High, Medium and Low cyber assets (since at this point they don’t know how they’re classified).  As a result, just about all entities are using what he calls the “top-down” approach.

This is not how I’m using the terms.  I have been using them – since last October – in a much narrower sense: they are two different approaches to identifying BES Cyber Systems at a Medium or High-impact asset/Facility.  In the bottom-up approach as I define it, the entity starts by applying the definition of BES Cyber Asset to each of its cyber assets.  The next step is to combine those cyber assets into BES Cyber Systems, with no hard-and-fast rule about how to do so – other than the rule that every BCA needs to be part of at least one BCS.

What I call the top-down approach is to start with the BES Reliability Operating Services (BROS), discussed in the Guidance and Technical Basis section of CIP-002-5.  The entity needs to determine which BROS apply to the asset being evaluated (e.g. in a generating station, it is likely that the “Controlling Frequency” BROS will apply, while the “Controlling Voltage” BROS may not).  Then it needs to identify the systems that support one or more BROS at the asset (e.g. the DCS in the generating station).  These will then be the BES Cyber Systems (they need to be further broken down into their component BES Cyber Assets for completeness’ sake, but the unit of compliance is BCS, not BCA).

When I first realized that both of these approaches were viable, I thought it was an either/or question: which one was required by CIP-002-5 R1?  And the answer to that was clear: the bottom-up approach is actually built into the requirement, whereas the top-down approach only comes from reading the Guidance, which is of course not part of the standard for compliance purposes. 

When I say the bottom-up approach is built in, I mean it is implicitly built in.  One of the many endearing features of CIP-002-5 R1 is that it never explicitly orders the entity to identify their BES Cyber Systems in the first place; it starts right out by telling you to classify your BCS[ii].  But obviously you can’t classify what you haven’t identified, so first you have to figure out how to identify BCS.  And since the BCS definition refers to BCAs, you logically have to start with BCAs – then group them into BCS.  So the bottom-up approach is the only one that you can derive from simply reading the requirement.

However, the BROS are discussed at length in the Guidance section, and I know that many auditors consider the top-down approach to be the approach for BCS identification.  I actually know of only two auditor presentations that have squarely addressed the BCS identification issue.  Kevin Perry of SPP, in his webinar on CIP v5 last February, only talked about the top-down approach.  On the other hand, Joe Baugh of WECC, in his presentation linked above (starting on slide 40), only discusses what I call the bottom-up approach, and never even mentions the BROS. 

Does this mean that SPP entities should use top-down, WECC entities should use bottom-up, and all other entities are simply SOL[iii]?  At the moment, I’d say yes, but as I’ve said many times, I’m hoping there will be a comprehensive re-interpretation of CIP-002-5, almost certainly by NERC.  I would think that re-interpretation would address this issue, along with the many other issues I’ve been writing about for more than a year.  And if this doesn’t happen?  Well, we can all take comfort in the fact that McDonald’s is still hiring.

And what do I believe is the right approach?  I have been saying for a while (e.g. in this article in the June issue of Power magazine) that it is best to use both approaches, one as a check on the other.  Practically, I’ve been saying the entity needs to start with the top-down approach, which of course yields a list of BES Cyber Systems.  However, the entity needs to then run the bottom-up approach, going at least as far as the step of identifying BES Cyber Assets.  Then the entity needs to confirm that each BCA is contained in one or more BCS.

I was fairly happy with that idea until I had lunch recently with the CIP manager on the generation side of a large IOU.  He pointed out to me that, in large generating plants (those over 1500MW and subject to criterion 2.1), this will place a big burden on the entity.  It does this because the bottom-up approach requires a complete inventory of cyber assets, and large plants can literally have thousands of cyber assets – “programmable electronic devices”.

You may say at this point (especially if you’re on the Transmission side of the house), “Well that’s too bad, but CIP-002-5 R1 clearly requires the entity to consider every cyber asset at a Medium plant against the definition of BES Cyber Asset; this can only be done if there is a complete inventory.”  However, this misses one important point (and I can say that I missed it until my friend reminded me): not all BES Cyber Assets at a criterion 2.1 plant will be Medium BCAs.  Those that don’t affect 1500MWwill be Lows[iv], and CIP-002-5 says in two places that an inventory of Low cyber assets isn’t required[v].

Does this mean that, at least in a criterion 2.1 plant, the bottom-up approach really isn’t feasible?  In general, I’d say yes.  The entity first needs to do the top-down approach to produce the list of BCS.  It then needs to determine which if any of these BCS affect[vi] 1500MW.  Finally, the entity needs to identify the component BES Cyber Assets of each BCS - as well as those cyber assets that are networked with one or more BCAs, since they will be Protected Cyber Assets.  Any cyber assets that aren’t identified as BCA or PCA by these steps will be Low impact and don’t need to be inventoried.

However, I can think of an example where the top-down approach clearly isn’t enough in a 1500MW+ plant.  At SPP’s BES Cyber System identification exercise I attended in Dallas in February, they of course advocated the top-down approach.  But they also pointed out the case of an environmental system in a large plant that is designed to trip the plant if there is an environmental excursion of more than ten minutes.  Since environmental protection isn’t one of the BROS, this system wouldn’t be identified by the top-down approach; yet it clearly would be identified in the bottom-up approach, since it has an effect in under 15 minutes.  SPP clearly expects such systems to be included in the entity’s list of Medium impact BES Cyber Systems.

Therefore, my modified rule for the criterion 2.1 generating stations is: a) Use the top-down approach to get your list of BCS; and b) Augment that list with any other systems that may not fulfill a BROS but clearly can have a fifteen-minute impact.  In any case, you shouldn’t have to inventory all of your cyber assets at the plant, as long as you can show that cyber assets not inventoried are all on separate networks from those that contain Medium BCS. 

So at least in a criterion 2.1 generating station, the top-down approach (with the slight modification just described) is the only feasible one.  Does that mean it’s the only feasible approach across the board – for substations, control centers, etc?

I say the answer to this is no, for two reasons.  The first reason – the weightier from a “legal” point of view – is that IMHO criterion 2.1 is the only one that can lead to both Medium and Low BCA/BCS at a single asset/Facility.[vii]  If there can be only one classification of BCS, then every cyber asset at a Medium asset/Facility will need to be considered as a BCA, meaning it will need to be inventoried.  The second reason is that control centers and substations (and smaller generating stations) have much more manageable numbers of cyber assets; at them, I don’t believe it’s a great burden to do a complete inventory.

So here is my final “ruling” on the question of the top-down vs bottom-up approaches to BES Cyber System identification:

1. At criterion 2.1 plants, use the modified top-down approach I outlined above.
2. At all other High or Medium impact assets/Facilities, combine the two approaches so one checks the other.

Of course, as with everything else having to do with CIP v5, your friendly local Regional Entity auditor will have to decide this question for you (and hopefully he/she will get some guidance from NERC, as I think I’ve already said ten times in this post).

July 28, 2014: Please note I just posted a correction to this post. 

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

---

[i] What I’m hoping will come soon is a new post in which I “rewrite” CIP-002-5, the way I think it should have been written in the first place.  Of course, this is for heuristic value only, since there is no longer any chance that CIP-002-5 can actually be changed (unlike the last time I rewrote CIP-002-5 - in my comments to FERC last June - when I really did hope that FERC would direct NERC to rewrite the standard.  Of course, that didn’t happen).  But since I still think that somebody – almost certainly NERC – needs to take an extraordinary action and come out with a comprehensive re-interpretation of CIP-002-5, by rewriting the standard now I’ll at least provide my view on what that re-interpretation should look like.

[ii] You may find this statement confusing, since CIP-002-5 R1.1 and R1.2 clearly order the entity to “Identify” BCS, not classify them.  But since this is only done in the context of classifying BCS (i.e. R1.1 is about “identifying” High BCS and R1.2 is about “identifying” Medium BCS), this is a misuse of the term ‘identify’.  There should first be a separate requirement saying you should identify your BCS (using the top-down or bottom-up approaches) at High and Medium assets/Facilities, followed by a requirement something like the current R1 (but much more clearly written), in which you determine which of those BCS you’ve identified are in fact either High or Medium ones (I’m skating over a whole bunch of other issues here.  I’m hoping I can address them all when I “rewrite” CIP-002-5 in a future post).

[iii] I believe this is a NERC Glossary term, but if not I’m sure it’s in Webster’s.

[iv] This is my interpretation.  Criterion 2.1 doesn’t explicitly say that any cyber asset that doesn’t affect 1500MW will be a Low.  This is one of the many areas in CIP-002-5 R1 and Attachment 1 where some sort of ruling needs to be provided by NERC.

[v] It actually says a list of Low BES Cyber Systems isn’t required, but you couldn’t have that list without having a list of Low BCAs.

[vi] I’m simplifying by saying “affect”; see criterion 2.1 for the full story.

[vii] I’m glossing over a whole bunch of considerations here, of course.  I’ve addressed them in previous posts, but when I do my rewrite of CIP-002-5 R1 I hope to address them all in a logical, consistent fashion.

The Timeline for CIP Version 5.5

When FERC said they were going to approve CIP version 5 in their April 2013 NOPR, I immediately drew the conclusion that CIP v5 wouldn’t be the next version that entities had to comply with; it would be a new version 6.  My reasoning, outlined in this post, was this:

1. Given the tone of the NOPR, it was clear that FERC wasn’t going to approve v5 unchanged. 
2. Since NERC’s Rules of Procedure don’t allow changes to be made to a standard after it has been approved by the Board of Trustees (and all of the v5 standards were so approved at the end of January 2013), the changes would have to be in a “compliance filing” ordered by FERC when they approved 2013.  The revised standards would have to have a new version number, which would almost certainly be v6.
3. I also didn’t think it was likely that v5 would ever be allowed to come into effect before v6.  I felt the changes ordered by FERC (especially removal of the “Identify, Assess and Correct” language, which FERC strongly suggested they would order when they wrote the NOPR, and indeed they did mandate in Order 791) would make it almost impossible to have a smooth transition from v5 to v6.  I therefore thought that v6 would “sunset” v5, just as v5 would “sunset” v4 when it was approved, and the next compliance version would be v6.

I continued to hold this position when Order 791 came out last November.  In a post two days later, I argued that it was likely that the v6 standards would come into effect on the same dates as the v5 standards, so they would clearly supersede v5. 

However, after attending the first meeting of the “CIP Version 5 Revisions” drafting team in February, I changed my opinion.  As discussed in this post, I was convinced at the meeting, by the smooth-tongued Scott Mix, that there was really no problem with v5 coming into effect before v6; all NERC has to do is let it be known they won’t consider IAC in their auditing, and all will be well.

At that point, my official position became that v5 would be the next version, followed after some period of time by v6.[i]  That is, until I saw the official first drafts of the drafting team’s work (now being balloted by NERC).  Then I realized two things.  First, the drafting team hasn’t produced new versions of all the v5 standards.  Specifically, they have only revised CIP-003, -004, -006, -007, -009, -010, and -011.  They have left CIP-002, -005, and -008 exactly as they were.  You might wonder why this is unusual, since the latter three standards must not have needed changing.

But this hasn’t been the previous practice.  For example, in the CIP v2 to v3 transition, the only standard that was changed in v3 (I believe) was CIP-006; yet all of the standards were revised so that there was a “-3” at the end of them.  Similarly, when CIP v4 was developed, only CIP-002 was changed in substance, but all of the other standards were revved to v4.  The fact that this hasn’t been done this time means that the next compliance version of CIP won’t be v5 or v6; rather, it will be the mixture of v5 and v6 standards I’ve just described.

Is this a problem?  No, it isn’t in theory.  But it does mean that NERC entities need to be careful with the version of the standard they’re using.  With versions 1-3, you could tell simply by the “-1”, “-2” or “-3” which version you were dealing with.  Now, you need to have a little cheat sheet telling you that you should always be using the “-6” versions of CIP-003, -004, -006, -007, -009, and the “-5” versions of CIP-002, -005 and -008.  And how about CIP-010 and -011?  Those were “-1” in the original v5, but they have both been revved to “-2”.  So you need to use the latter.  And since the next CIP compliance version will be a mixture of 5 and 6, it may be better to just call it v5.5 (except I know everyone – including me – will continue to call it v5).

At this point, you may ask (and if not, I’ll ask for you) “Why don’t we just simplify things and bring CIP-002, -005 and -008 to v6?  Then there will be a complete set of v6 standards, and we’ll just comply with those.”  I have to say, that’s an excellent question.  The answer is simple: After the debacle with CIP Version 4 – then v5 – then v4 – and finally v5, I think most people at NERC are convinced that if they introduce a new improved CIP version, they’ll all be strung up from the nearest lamppost by an angry crowd of NERC entities. 

And there is reason for that fear, since the CIP v4 debacle did result in real costs to a number of entities.  On the other hand, it simply isn’t the case that v6 is being foisted on the industry as part of some nefarious plan to generate more paperwork (and perhaps justify more jobs at NERC?).  V6 has only come about because FERC required changes in v5, and these can’t be included in the current version.  V6 is nothing more or less than what FERC ordered (I actually wish it were more than what FERC ordered.  For example, I would like to see CIP-002-5 R1 and Attachment 1 completely rewritten.  I believe I’ve mentioned this once or twice before – like in over 20 posts.  However, it ain’t happening, meaning we have to go to plan B.  Or C.  Or D….But something needs to be done).

So I can guarantee you won’t hear a NERC employee refer to CIP version 6, even though most of the standards currently being balloted have a “-6” suffix.  Instead, it’s “CIP Version 5 Revisions”.  Whatever…

The second thing I noticed is that, unlike what I expected, the compliance dates for the new standards are almost all exactly the same as they were for the “classic” version 5 standards – that is, High and Medium impact facilities need to comply on April 1, 2016, while Lows need to comply on April 1, 2017.  This may be hard to see from the wording of the new Implementation Plan, but hear me out.  For most of the standards, the plan reads like this:

Reliability Standard CIP-003-6 shall become effective on the later of April 1, 2016 or the first day of the first calendar quarter that is three months after the date that the standard is approved by an applicable governmental authority.

So CIP-003-6 will become effective on April 1, 2016 (the same day as CIP-003-5, of course), unless FERC doesn’t approve it until January 1, 2016 or later.  And what is the chance that the latter will happen?  Very small.  Since the new standards will all be presented to FERC by the beginning of February of next year, and since FERC is likely to be happy with them (after all, these new standards are simply what FERC ordered, and FERC staff members have been riding herd on the SDT to make sure they produced something the Commissioners would like, although I’m sure they were always careful to say they didn’t know what the Commissioners would like), it is very unlikely that FERC won’t approve them fairly quickly, and certainly before January 1, 2016.

In other words, there will be no v5 standards that will come into effect, only to be superseded by v6 standards.  The three v5 standards that don’t have v6 versions will remain in effect, and in fact the compliance dates for v5.5 are for the most part unchanged from those of v5.

Of course, this being NERC, there are exceptions to the statement that the compliance dates are the same for the v6 standards as for their v6 predecessors.  Here are the exceptions:

1. This really isn’t an exception, but compliance with CIP-003-6 R2 – the only requirement that applies to Low impact assets – is due on April 1, 2017 (the same date as CIP-003-5 R2) or nine months after the effective date of CIP-003-6 itself.  This means that, if FERC doesn’t approve CIP-003-6 until April 1, 2016 or later, the compliance date for CIP-003-6 R2 will be later than April 1, 2017.[ii]  But there is almost no chance that will happen.[iii]
2. CIP-004-6 is the only v6 standard that requires FERC approval six months before April 1, 2016, in order for it to be effective on that date.  Since FERC will get the v6 standards from NERC in early February, 2015, this means they would have to approve them before October 1, 2015; if they approve them in the fourth quarter, then CIP-004-6 will become effective July 1, 2016 (while the other v6 standards will still come into effect April 1).  I won’t say this is impossible, although I still think it is less likely than that FERC will approve all of the v6 standards before October 1.
3. There is an exception regarding CIP-006-6 R1.10[iv]:

For new high or medium impact BES Cyber Systems at Control Centers identified by CIP-002-5.1 which were not identified as Critical Cyber Assets in CIP Version 3, Registered Entities shall not be required to comply with Reliability Standard CIP-006-6, Requirement R1, Part 1.10 until nine calendar months after the effective date of Reliability Standard CIP-006-6.

This states that control centers that weren’t Critical Assets under v3 have an additional nine months to comply with this one requirement part – so their date is January 1, 2017.

4. The next exception may seem small but is probably huge: 

Registered Entities shall not be required to comply with the elements of Reliability Standard CIP-007-6, Requirement R1, Part 1.2 that apply to PCAs and nonprogrammable communication components located inside a PSP and inside an ESP and associated with High and Medium Impact BES Cyber Systems until six calendar months after the effective date of Reliability Standard CIP-007-6.

This states that, for the purposes of compliance with CIP-007-6 R1.2, Protected Cyber Assets and “nonprogrammable communication components” inside an ESP have an additional six months to comply, for an effective date of October 1, 2016.

5. Here is the last exception:

Registered Entities shall not be required to comply with Reliability Standard CIP-010-2, Requirement R4 until nine calendar months after the effective date of Reliability Standard CIP-010-2.

CIP-010-2 R4 is the new requirement for Transient Cyber Assets and Removable Media.  This says that the effective date for this requirement – for all entities with High and Medium assets (the only ones subject to the requirement) – is January 1, 2017.  This is of course a big exception, but probably reflects the SDT’s estimate of the effort that will be required to comply with this requirement.

So there you have it – the timeline for CIP version 5.5.  If somebody creative wants to put this into an actual visual timeline, I’ll publish it.  Otherwise, this is left as an exercise for the reader.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

---

[i] There is a precedent for a smooth transition like the above.  When FERC approved CIP version 2 in the fall of 2009, they ordered a single change – a requirement for providing escorted physical access to non-authorized visitors to a Critical Asset.  NERC developed this requirement, added it to CIP-006-2, and changed the version number of all the v2 standards to v3.  CIP version 2 came into effect on April 1, 2010, while v3 came into effect on October 1, 2010.  This transition was possible because there was nothing changed or removed from v2; just the one requirement was added.  In the case of the v5-v6 transition, that was certainly not the case, so I didn’t see that it would be possible to have a similar transition happen as had with v2-v3.

[ii] Here’s why I say this: As has just been said, the compliance date for CIP-003-6 will be April 1, 2016 or three months after FERC approval.  If FERC waits until April 1, 2016 or later to approve the standard, then the effective date for the standard will be July 1, 2016 or later.  Since that date is nine months before April 1, 2017, in this case CIP-003-6 R2 would have an effective date later than April 1, 2017.  However, there is just about zero chance that FERC will dawdle this long in approving CIP-003-6.

[iii] And this is a pretty significant point.  I had really expected that the new requirement for Lows would have a compliance date 9 months or even longer after April 1, 2017.  Given the number of Low impact assets that most NERC entities have, it really behooves them to start planning for the Low compliance effort now.  I have had a couple large entities tell me that, regardless of problems they see with the High and Medium impact assets, it’s the Lows that scare them – simply because of their number.

[iv] This requirement is for protection of cabling that connects devices in an ESP, which itself exits the PSP.  It was one of the four items mandated by FERC in Order 791.

More Utility Bashing

In this recent post, I discussed three particularly egregious instances of people reflexively dumping on electric utilities for poor cyber security, without bothering to back up what they said with any evidence.  The first example was an article on the Smart Grid News website, which had attacked the new NERC physical security standard without bothering to understand how and why the standard was developed.

I regret to say that SGN is at it again.  In this recent article – whose overall premise I basically agree with – Jesse Berst takes a pot shot (so to speak) at Pacific Gas and Electric, owner of the Metcalf substation, by saying “Metcalf is the California substation that was attacked last year (though the incident was not revealed until the following spring).” 

I found this quite interesting, since I remember reading about the incident when it happened in April 2013.  Sure enough, a quick Google search found several contemporaneous articles, including this one published the same day as the attack.

It is true that the publicity became much more widespread this year, when the former FERC chairman began raising the issue repeatedly, and some US Senators and Congressmen wrote FERC to do something about this (even though they had been doing a lot) – this led to the new physical security standard, CIP-014-1.   

But to say that this was never disclosed last year is just another example of the reflexive utility bashing that seems to be one of the biggest sports in the media nowadays.  I’m certainly not saying the utilities can’t be criticized for poor cyber security practices.  But make your case, guys – based on facts.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

Chasing our Own Tails

This post is actually an extended footnote to the previous post on the CIP-002-5 RSAW.  But it’s lengthy enough –and relevant in its own right – that I decided to make it a separate post.  It has to do with the discussion at the end of what I call “Issue 4” – i.e. the big issue with this RSAW that I think could and will lead to the final collapse of CIP v5, if nothing is done to change this.[i]

In the section on Issue 4, I brought up the first note in the “Notes to Auditor” section at the end of the RSAW:

Results-based Requirement: The auditor should note that this is a results-based Requirement. As such, the entity has great latitude in determining how the result is achieved. The auditor should focus on verifying that the result is complete and correct.

I pointed out in the previous post that this sounds wonderful, but it only works if the “result” aimed for in the requirement is clear and unambiguous; this is certainly not the case here.

However, on further reflection I realized there is a basic logical flaw in this statement.  You can see that by asking how the auditor will determine if the intended result of CIP-002-5 R1 – the correct identification and classification of BES Cyber Systems – was achieved. 

Let’s imagine an easy case:  Suppose the auditors were given special glasses that made each actual BES Cyber System appear blue.  All they would have to do is put these glasses on, look around the facilities being audited, note the blue systems, and compare this to the list already provided by the entity being audited.  Of course, any discrepancies will be instantly identified, and appropriate PV’s can be issued.

But there are no special glasses in this case.  So how does the auditor determine if a system in front of him/her is a BCS?  You would probably say, “Of course, it’s a BCS if it meets the definition of a BCS.”  So what is the definition of a BCS?  It’s linked to the BES Cyber Asset definition[ii]:

 A Cyber Asset that if rendered unavailable, degraded, or misused would, within 15 minutes of its required operation, misoperation, or non-operation, adversely impact one or more Facilities, systems, or equipment, which, if destroyed, degraded, or otherwise rendered unavailable when needed, would affect the reliable operation of the Bulk Electric System.

Is an auditor going to be able to tell if an alleged BCS meets this definition simply by looking at it?  Certainly not.  He/she is going to ask the entity to show how they arrived at the determination that this is a BES Cyber System.  In other words, they will look at the documented process for this determination, meaning they will look for a discussion of how this result was achieved.

Now let’s go back to the RSAW quote above, but substitute in what we have just learned:

Results-based Requirement: The auditor should note that this is a results-based Requirement. As such, the entity has great latitude in determining how the result is achieved. The auditor should focus on verifying that the result was achieved by a correct process.

Do you see the problem?  Even though this is supposedly a “results-based” requirement, the only way to determine whether the result is correct is to look at how it was achieved.  And as I said repeatedly in the previous post, the RSAW says nothing about what the process of BCS identification / classification should be – except that it should be one which achieves this “result”.  So we’re just chasing our own tail here. 

What do I recommend to fix the problem?  The same thing I’ve been saying for a year: CIP-002-5 R1 needs to be “reworded”.  It’s no longer possible to change the actual wording, but there needs to be some interpretation, probably from NERC, of what the requirement means.  Unfortunately, the RSAW isn’t that interpretation.

But no matter what happens, R1 isn’t ever going to be a “results-based” requirement; there will never be a way an auditor can determine if an entity has correctly identified BES Cyber Systems other than by looking at the process they used to identify them.  But the process itself needs to be made clear, and at the moment it is anything but clear.

  

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

---

[i] And this isn’t saying that the RSAW is the cause of this problem, but rather the symptom of it.  The problem is the inconsistent and ambiguous wording of CIP-002-5 R1.  Some people thought the RSAW might fix the problem, but instead it makes it worse by not even attempting to clarify the ambiguity.  Instead, it pretends that this is in some way a virtue and that the auditors can and want to step into the role of interpreter/judge/executioner to make all this better.  They can’t do that, and they certainly don’t want to.

[ii] This ignores a complication in that I believe almost all auditors will say that an alternative “definition” for BCS is a system that assists with the performance of a BES Reliability Operating Service – discussed in the Guidance section of CIP-002-5 R1 but not in the requirement itself.  This just reinforces my argument, so I won’t pursue it further now, for fear of violating what Little League teams call the “Slaughter Rule”.

The CIP-002-5 RSAW: Waiting for Godot

I have a confession to make: I often start writing a post without knowing clearly what I’m going to say.  This is because I don’t start really analyzing a topic until I write a post on it – like most people, I’m fine with using my initial impression as a guide going forward.  You can’t analyze everything in depth, or you’d never be able to get out of bed in the morning.  This has led many times to a post that started out by saying one thing but ended up saying something quite different.

And so it is with this post.  Last week, NERC released drafts of the RSAWs (Reliability Standard Audit Worksheets) for CIP Version 5.  I of course immediately downloaded the CIP-002-5 RSAW and quickly reviewed it.  I was eager to see whether and how it might shed some light on the ambiguities and inconsistencies in CIP-002-5 R1 that I’ve written so much about.  My initial impression was this: I didn’t expect much from the RSAW, and sure enough it didn’t disappoint me.  If you’re looking for guidance with the problems in the language of CIP-002-5 R1, you won’t find it here.  Meanwhile, it doesn’t even do a good job of accomplishing its limited purpose: translating the words of R1 and Attachment 1 into a format that auditors can use for audits.[i]

The good news is that, as I started working on this post in earnest, I moved away from this initial impression of mild disappointment.  The bad news is that I moved to something much more like deep despair.  Not only does the RSAW not do a good job of accomplishing its very limited purpose, but it creates huge holes which – if NERC does nothing to fill them – will result in chaos when it comes time to audit compliance with CIP-002-5 R1.  And as I’ve said many times before, R1 is the foundation for all of CIP Version 5 (and v6, for that matter).  If the foundation is rotten, the house will fall.  I believe the whole v5 house will ultimately fall unless something is done about this.

Before I continue, I need to repeat the warning I’ve used a couple times before.  It was never more needed than for this post:

Warning: Reading this post may cause unintended side effects in NERC compliance professionals, including loss of sleep, depression, excessive consumption of alcoholic beverages, and thoughts of suicide.

Now to the argument.  In a post in March, I tried to look at the different possibilities for help with the problems with CIP-002-5 R1, and concluded that the RSAWs weren’t likely to provide much assistance (in spite of the fact that NERC staff members had hinted at the March CIPC meeting that this might be the case).  I said this because I know the RSAWs are intended to provide guidance to auditors based on the literal wording of the standard, and the NERC lawyers will never let them do anything more than that.  Anyone who believes (or implies) that the RSAWs can provide interpretation of the requirements is deluding themselves, others, or both.

But there’s a bigger problem, even if you accept the idea that the best an RSAW can do is simply tell an auditor how to audit based on the literal wording of the standard:  Because of the many problems with the wording of CIP-002-5 R1 and the fact that – as I’ve said multiple times before – there can be no consistent interpretation of R1 that doesn’t contradict at least some of the wording, just telling auditors how to audit based on the literal wording (if this is even possible) doesn’t in fact give them a consistent, clear methodology for conducting the audit.  And it also doesn’t give the NERC entities a consistent, clear methodology for complying with R1 in a manner that will keep them free of violations.  The inevitable result of this, unhappily, will be a HUGE increase in the use of auditor discretion in audits – and supposedly one of the goals of v5 was to decrease or even eliminate this problem.  The auditors will have to use some methodology, and this RSAW doesn’t provide it.  More on this below.

Also, I’m not at all sure this RSAW even does a good job of reproducing the wording of the standard. As I said, there are lots of ambiguities and contradictions in that wording, and you have to do some sort of interpretation even in order to just translate the wording in a consistent manner (since it isn’t consistent in the first place).  I’m sure the people writing the RSAW have tried their best, but some of what they’ve come up with is as contradictory as the requirement itself. 

I will first discuss three specific issues, in the order in which I encountered them in the RSAW.  Note that these were all discussed in mind-numbing detail in my recent 7,000-word post on the problems in CIP-002-5 R1.  Finally I will conclude with a discussion of a huge overall issue that pretty much trivializes everything else.

Issue 1: Assets/Facilities

First, there’s the issue of what the criteria in Attachment 1 apply to.  I had originally thought that the six asset types listed in R1 were what you applied the criteria to, in order to identify High, Medium and Low impacts; in fact, I know there are to this day many in NERC and the Regional Entities that believe this.  However, an Interested Party convinced me earlier this year that the six asset types are simply the locations at which BES Cyber Assets/Systems can be found, and the Attachment 1 criteria really refer to other things.  Indeed, when you go through the criteria, you will see that they apply to a lot of things that aren’t included in the list of six asset types. 

Most importantly, criteria 2.3 – 2.8 apply to something called “Facilities”, which in a substation means each line, each transformer, each bus, etc.  If a Transmission entity assumes that Facilities is just another word for the six types of assets in R1, they may well end up over-identifying BES Cyber Systems.  This is because there could be both Medium and Low impact Facilities at a substation, but if you assume that “Facility” means the substation itself, this would indicate that all of the BCS are Medium as well[ii] – and this could well not be the case. 

However, I know at least some of the Regional Entities are saying that “Facility” means the substation  itself in criteria 2.4 – 2.8.  And since the substation is clearly Medium impact in those criteria, all of the BCS at the substation have to be classified Medium as well.[iii]

How does the RSAW deal with this issue?  It doesn’t.  Consider this wording under Evidence Set 1 on page 5:

2. A list of all high impact BES Cyber Systems identified, and the asset(s) with which the BES Cyber System is associated.

3. A list of all medium impact BES Cyber Systems identified, and the asset(s) with which the BES Cyber System is associated.

These clearly imply that BES Cyber Systems are only classified by their association with assets.  Of course, “assets” isn’t a NERC defined term, but in this context I believe it means a member of one of the six types of assets listed in R1.  And there is definitely no mention of Facilities.  The upshot of this is that a Transmission entity won’t be able to have both Medium and Low BCS at a Medium substation; all BCS will be Medium – unless the RSAW is changed, of course.[iv]

Issue 2: “At” / ”Associated with” / Whatever

I think I could write a small book by now on the question of the use of “at” vs. “associated with” in Attachment 1.  I did write a whole post on this in February.  Unfortunately, there is a lot of confusion on this issue in the RSAW.

The first example is item 2 under Evidence Set 1, which I quoted above.   Here, “associated” is used to refer to High BCS, when the wording in Attachment 1 Section 1 is “used by and located at”.  A BCS is classified as High because it is used by and located at a High control center, not “associated with” it.  If the SDT had used “associated”, this would have meant that every RTU in a substation controlled by the control center could be a High impact BCS – and entities would have to spend a fortune protecting substations as High impact assets.   

Fortunately, the writer(s) of the RSAW wasn’t consistent in this regard, since line 1a under Evidence Set 2 (page 5) says

a. A list of high impact BES Cyber Systems, for which this entity has full or partial compliance responsibility, used by and located at the asset

Here, they got it right.  But don’t celebrate yet, since near the top of page 7 we find this line:

Verify that the high and medium impact BES Cyber Assets associated with the sampled asset have been correctly identified.

Now they’re back to using “associated with” for both High and Medium BES Cyber Assets (and this should read “BES Cyber Systems”, of course, not “BES Cyber Assets”.  The BCAs themselves aren’t rated, which I would hope the NERC staff understands).

There’s now a new issue related to the “at / associated with” question.  That is what I am now officially dubbing the “Whitney Doctrine”.  This doctrine states that “Physical location IS a determinant factor for impact classification.”  It has the effect of solving the “transfer-trip” relay question in a way that made a lot of Transmission entities quite happy, as described in my previous post.  It has no basis in the wording of CIP-002-5 that I can see, but as I said in that post, I’m perfectly fine with this – somebody has to push the words of R1 around so that there is a consistent interpretation.[v]

However, it does mean that “associated with” now has no meaning, since because of the Whitney Doctrine both High and Medium BES Cyber Systems need to be located at the asset (vs. just High BCS in the dark days before the Doctrine was promulgated, as described in my last post).  And it also probably means that you won’t be able to rate a BCS at a criterion 2.5 substation according to the line (i.e. Facility) it’s associated with.  You can’t say that a relay is “at” a line, but you can say it’s "at" a substation.  So the relay will have to take the classification of the substation (Medium in this case). 

However, maybe Tobias will figure out a way to make the Whitney Doctrine compatible with the Facilities principle, thus saving his “transfer-trip” ruling (and possibly his life, since the TO/TOP’s wouldn’t be pleased to hear he was going back on what he said at the CIPC meeting, just for the sake of consistency).  He’s a smart guy.[vi]  But this does show the whack-a-mole effort that is required when trying to fix problems in the Attachment 1 criteria; you solve one problem (to actual applause, in the case of Tobias) and another pops up somewhere else because of your “solution”.

Issue 3: “The Phantom Low Impact BES Cyber System” and other Tales of Mystery and Horror

I have several times discussed the fact that there is an inherent contradiction in CIP-002-5 between Attachment 1 and Requirement 1.  The former assumes you start out with an inventory of all your BES Cyber Systems; you subtract out the Highs and Mediums, and voila the rest are Lows.  Meanwhile, R1 makes clear you don’t have to inventory cyber assets at Lows.  So how would you ever get the comprehensive list that Attachment 1 clearly implies you need?

One result of this contradiction is that there is a big asymmetry between R1.1 and 1.2 on the one hand, and R1.3 on the other:

1.1. Identify each of the high impact BES Cyber Systems according to

Attachment 1, Section 1, if any, at each asset;

1.2. Identify each of the medium impact BES Cyber Systems according to

Attachment 1, Section 2, if any, at each asset; and

1.3. Identify each asset that contains a low impact BES Cyber System

according to Attachment 1, Section 3, if any (a discrete list of low impact

BES Cyber Systems is not required).

1.1 and 1.2 require you to identify BCS, but 1.3 just requires you to identify an asset that “contains” a low BCS.  It immediately goes on to tell you that you don’t need a list of those low impact BCS.  So how on earth can you know whether an asset contains one of these mythical beasts?  Of course, what would make much more sense would be if 1.3 read something like “Identify each Low impact asset”; in fact, every NERC entity I’ve talked to about CIP v5 asset identification says this is exactly how they’re interpreting 1.3.

So why did the SDT twist their words into knots in R1.3?  It’s because the entire premise of R1 and Attachment 1 is that assets (i.e. the “big iron”) are never classified; just the BES Cyber Systems (the “little iron”) are.  The only way they could say something that means “Low impact asset”, without using those dreaded words, was to use the circumlocution in R1.3.   This in spite of the fact that nobody is interpreting the words this way – in fact, nobody could interpret them that way, given the contradiction between the first part of R1.3 and the parenthetical expression at the end of it.

All of this might just be amusing, but the fact that this wording is used in the RSAW (which it is) could potentially cause trouble.  I refer now to two lines on page 7:

Verify that the asset has been correctly identified as containing a low impact BES Cyber System.

and

Verify that the sampled asset does not contain a BES Cyber System.

In the first line, the auditor is being asked in effect to make sure the entity has properly identified Low impact assets; in the second line, he/she is being asked to make sure the entity isn’t lying when they say an asset isn’t even a Low[vii].  But because “Low asset” is a verboten term in CIP v5, the RSAW uses the language of R1.3.

But how is the auditor going to verify this?  If I were an auditor new to CIP v5 and I saw these two lines, the first thing I would ask the entity is to show me their list of all Low BCS; of course, that doesn’t exist.  So how does the auditor verify that the asset contains or doesn’t contain a Low BCS?  The answer is that the auditor needs to know that saying an asset contains a Low BCS is the same thing as saying it is a Low asset – wink wink, nudge nudge.  So he verifies whether or not these are Low assets by doing what every entity is now doing anyway: assuming the criteria in Attachment 1 refer to assets (or Facilities) and identifying Low assets as those that aren’t High or Medium.[viii]  But shh…don’t tell anybody they did this.[ix]

I hope I don’t have to tell you that requiring knowledge that is nowhere stated in the requirements isn’t a wonderful way to have auditors audit a standard with million-dollar-a-day penalties.

Issue 4: The BIG One

I really shouldn’t call this just one of four issues.  That’s like saying there were a number of issues on the Titanic the night of April 14 1912, including a shortage of party hats and also an iceberg they’d just hit.  And like that iceberg, this issue could literally sink CIP Version 5, and probably will if nothing is done to address it.

Unfortunately, this isn’t just a problem found in the wording in one or two places in the RSAW; it’s found just about everywhere.  Let’s start with Evidence Set 1 on pages 5 and 6.  There are three phrases that contain the words “the process required by R1”, for example

Evidence that the process required by R1 was implemented to determine the list of high and medium impact BES Cyber Systems.

What’s wrong with these words?  Tell me, what is the “the process required by R1?”  You might say, “That’s simple, it’s the methodology you use to comply with Requirement 1”.  I say, “Great, now tell me what that methodology is.”  Then what do you say?  I know that I certainly couldn’t tell you that methodology.  I recently wrote a post where I started out thinking I was going to finally nail down this methodology.  I ended up breaking it off unfinished, and finally came back to admit that I couldn’t finish it.  This is because I have come to the conclusion that there is no consistent, complete methodology that can be stated for compliance with R1, which will at the same time not do some violence to the wording of either R1 or Attachment 1.[x]

So why does the RSAW blithely imply that “the process required by R1” is something that any child could recite?  The whole point of the RSAW is to tell the auditors how to audit the requirements, not to leave the hardest part as an exercise for the reader.  It’s as if you read a cake recipe for the first time.  It first carefully lists all the ingredients – that’s good.  Then it says, “Now bake the cake according to the required process.”  Wouldn’t you expect the recipe to tell you what that process was?

Let’s go on to page 6.  There we read several phrases with the words “can be reasonably expected”, as in

Verify that the process can be reasonably expected to identify all high and medium impact BES Cyber Systems at each asset.

This is really great.  The most important parts of R1 (classifying BCS in this case) are being judged not on the basis of whether they were properly followed by the entity, but on whether they’re “reasonable”.  And whose reason is going to decide this?  Why, the auditor’s!  In the end, the way you will be judged on how well you classified BCS (as well as Low impact assets) is by whether your auditor thinks what you did was reasonable.

Does anyone else think this just might be a tiny little problem?  Like perhaps the sound of that iceberg hitting in 1912?  I have nothing against the auditors – they’re probably much more “reasonable” than the rest of us.  But I thought one of the cardinal principles for CIP v5 was that it was going to eliminate the need for auditors to use their discretion – and now we’re not only not eliminating that, we’re requiring them to do so!  I know a number of auditors, and the last thing they want to do is have to use their own reason to bridge a wording gap in the requirements.  They sometimes do have to do this, but for a fairly well-defined issue like what “routable” means.  Here, they’re being asked to each read the great works of Western philosophy and regulatory compliance, so that they can each come to their own understanding of what CIP-002-5 R1 means.

Let’s go on.  In this same section, we now find multiple uses of the words “correct” or “correctly”, for example

1. Verify that the high and medium impact BES Cyber Assets associated with the sampled asset have been correctly identified.

2. Verify that the impact rating of each identified BES Cyber System is correct.

And what exactly is the “correct” way to identify BCAs, or the “correct” way to assign impact ratings to BCS?  Of course, there’s no clue provided.  The auditors are clearly required to use their “reason”.  Wonderful.

Finally, we come to the “Notes to Auditor” section at the end and read this blockbuster:

Results-based Requirement: The auditor should note that this is a results-based Requirement. As such, the entity has great latitude in determining how the result is achieved. The auditor should focus on verifying that the result is complete and correct.

This makes sense on the surface.  After all, wasn’t that one of the big problems with the previous CIP versions?  The requirements dictated how you should do something rather than just saying what you should do and letting you figure out how to do it? 

This would make sense here, too, if there were some clear definition of the results that need to be achieved.  But what are those results?  They're things like the “correct” classification of BCS, a “reasonable” methodology for complying with R1, etc. – and these are left totally undefined. 

Here’s an analogy.  Let’s say you were being audited on your navigation abilities.  The auditor might say, “Drive to City Hall.”   It would make sense that she wouldn’t care how you got to City Hall, just that you found the place.  However, now suppose she is told to audit your navigation abilities, but not tell you where to go.  She is just to tell you to go “somewhere reasonable”, and make sure you did a reasonably good job of getting “somewhere reasonable”.  So you drive wherever you want, and as long as she’s sure you got somewhere, she has to pass you.  Obviously, this would make the entire auditing process meaningless, since there would be no objective standard for deciding you had gotten “somewhere reasonable”.

(June 29: I have written a long footnote to this discussion that I decided to make a new post. You can find it here).

And friends, that is exactly the problem here.  The RSAW spends most of its time addressing the simple wording problems of CIP-002-5 R1 (and doesn’t even do a good job of that, as shown in Issues 1-3 above), and hides the really thorny problems behind words like “reasonable” and “correct”.  The result is a requirement that can’t be objectively audited at all.  The only two ways an auditor can audit R1 are a) Simply give everybody a pass, or b) Decide on their own what R1 means, and apply that meaning mercilessly in their audits. 

To be honest, I don’t think there’s too much danger of b); I doubt there’s a single auditor who wouldn’t commit suicide or change careers if he had to be continually telling people that they were getting PV’s because his “reason” told him they should get them.  Instead, what will happen is that nobody will ever be judged wrong on how they identify and classify their BES Cyber Systems. This is exactly like the RBAM situation in CIP v1-3, where there was virtually no way an entity could have been found to have identified its Critical Assets incorrectly – except perhaps by turning in a game of Hangman and saying that was the RBAM.[xi]

And as I’ve said previously, if CIP-002-5 R1 is rendered unauditable by a lack of objective criteria to guide an audit, then the rest of the v5 requirements become unauditable as well.  I’m more convinced than ever that, if nothing is done about this and an entity receives a PV for R1 and challenges it in court (which they can do – CIP is regulatory law), the judge will take 15 minutes to read the requirement, exclaim “What is this ___ (stuff)?”, and throw CIP-002-5 out.  That will of course also invalidate CIP-003-5 through CIP-011-1.  This might actually be a good thing were it to happen say in the next six months.  Unfortunately, there’s no way it can happen before about five years from now, at which point there will be a huge investment in complying with CIP v5.  For that to all be put into question…well, it won’t be pretty, that’s for sure.

And now, Our Conclusion

I have been writing about this fundamental problem with CIP Version 5 for over a year; others have written and spoken about this as well.  But since human beings are averse to contemplating great upheavals, people have been waiting for Godot to come and fix everything.  I myself hoped FERC would do that in Order 791, and later hoped (against hope) that the new SDT would decide to address the problem.  More recently, people have been looking to the RSAWs as their hope; and I’m sure there are many who think the upcoming Transition Study Lessons Learned cases will be the answer.

Folks, I’m telling you: Godot ain’t coming.  This problem has to be solved by the NERC community.  The best solution is for NERC to step in and do something.  The second best is for FERC to demand NERC do something (or do it themselves, although that would be a very radical step that I don’t think anyone wants to see, since it would set a very bad precedent).  I also used to think that the regions could get together and fix this, but I no longer see that as possible or likely.  Who knows, maybe even an industry group like the EEI or the Transmission Forum could pull this off.

But I do know this is a disaster waiting to happen.  We’re heading full speed toward the cliff, smiling all the way over.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

---

[i] I will point out that this failure to even accomplish the limited purpose of the RSAW isn’t ultimately the fault of the people who wrote it, but of the wording of R1 and Attachment 1.  The many inconsistencies and contradictions in their wording make it literally impossible to craft an RSAW that accomplishes even its minimal purpose.

[ii] An exception to this would be BCS associated with an SPS that might be located in the substation.  If the SPS is Low and the BCS is only a BCS because of its association with the SPS (not with the substation itself), then the BCS would also be Low impact.

[iii] This post discusses this point in more detail.

[iv] I should point out that this is actually an interpretation of CIP-002-5 R1.  It seems the decision has been made by NERC that, no matter how the Attachment 1 criteria read, the only things they can apply to are the six asset types.  So “Facilities” means “assets”, and this means the different lines in a substation (and their BCS) meeting criterion 2.5 are all Medium impact.  And it means that criterion 2.3 now applies to an entire plant, not just to a single generating unit (a “Facility”).  So if a single unit at a plant has been designated “Reliability Must Run”, the entire plant will have to be Medium impact, not just the unit.  There are other consequences of this as well.  The upshot: I can’t fault NERC for making an interpretation in the RSAW, since I’ve been urging them to do this.  But I think a number of entities will be unhappy with what this interpretation means for their compliance costs. 

And I really don’t think this was a deliberate interpretation, either.  It is interesting that Tobias Whitney’s presentation at the NERC CIPC meeting two weeks ago quite clearly states that BCS at substations meeting criterion 2.5 are classified according to the Facility (i.e. line) they’re associated with (see slides 29 and 30).  I think this is the correct interpretation, but somehow this insight wasn’t passed on to the person writing the RSAW, even though they probably work for Tobias. 

[v] One of my favorite quotes from Lewis Carroll’s Through the Looking Glass is this one (I know, I’ve already used it in a previous post.  Hey, I’m all about sustainability):

"When I use a word," Humpty Dumpty said, in rather a scornful tone, "it means just what I choose it to mean- neither more nor less."

"The question is," said Alice, "whether you can make words mean so many different things."

"The question is," said Humpty Dumpty, "which is to be master, you or the words.  That's all."

[vi] I wish to point out another anomaly in the use of “associated with” in the RSAW.  This seems to be just the result of someone’s laziness, but it needs to be corrected.  In the discussion of Evidence Set 2 on page 5, item 1c says

c. A list of all BES Cyber Assets associated with each high or medium impact BES Cyber System identified in a. or b. above.

Can you see what’s wrong with this?  BES Cyber Assets are what makes up a BES Cyber System; they certainly aren’t “associated with” the BCS in the sense that Medium BCS are associated with assets or Facilities that meet one of the criteria in Section 2 of Attachment 1.  Fortunately, the writer found the right term on page 7, where we find this:

Verify that the BES Cyber Assets (and Cyber Assets, if any) comprising the BES Cyber System are identified.

So “associated with” needs to be replaced with “comprising” in the previous quote, and I’ll be happy – about this one point, anyway.

[vii] An example of such an asset is a generating station or Transmission substation that doesn’t contain any cyber assets (in a control capacity) at all.  It clearly won’t contain any BCS, either.

[viii] Unfortunately, even this procedure doesn’t work to identify assets that aren’t High, Medium or Low, such as those that contain no control cyber assets at all.  

[ix] I was told that at least two auditors say they will require a list of Low BCS, precisely so they can make sure the entity correctly identified assets “that contain a Low impact BES Cyber System.”  At this point, the problem has moved beyond being simply amusing.

[x] Actually, I knew that when I started to write the post in question.  But I did think I could write a consistent methodology that would conform to what the wording of R1 would be, if the SDT had taken more time to get it right.  I don’t even believe that anymore.

[xi] I realize there were some entities that did get assessed for having an incorrect RBAM in CIP v1-3 (I believe these mostly happened in the later years).  And I also realize that most NERC entities are going to do the right thing for BCS identification, even if they are given a free pass on CIP-002-5.  But the big problem is that, if everybody gets a pass on CIP-002-5 R1, it undermines the legitimacy of the rest of CIP v5.  How can you really be penalized for violations of let's say CIP-007-5, when everybody knows that you could have simply said you didn't have BES Cyber Systems?  If CIP-002-5 R1 is unauditable, then that spreads to the rest of the standards as well.  As I said, R1 is the foundation of the CIP v5 "house".  If the foundation is rotten, the house falls.