Monday, July 23, 2018

Debate with an auditor on CIP-014

In my last post, I lauded the NERC CIP Modifications drafting team for coming up with two great ideas for incorporating virtualization (or any new technology that affects fundamental definitions – the cloud is another example) into the CIP standards. That post was about the second of those ideas, the idea that the truly prescriptive CIP requirements need to be made non-prescriptive (although I don’t agree with their use of the term “objectives-based”, since that requires a measurable objective, and cyber security objectives aren’t measurable). Since almost all new CIP requirements since CIP v5 have been plan-based, I think that is the correct term to use now.

My concern with the SDT’s idea was that just making a prescriptive requirement (like CIP-007 R2 or CIP-005 R1) non-prescriptive isn’t the whole story on what needs to be done. It is important to keep in mind – as if anyone has forgotten! – that NERC’s auditing procedures are very prescriptive; you either did exactly what the requirement says or you didn’t. This works well for the 693 standards (in fact, it’s really the only way you could audit those). But it really misses the mark on the non-prescriptive CIP requirements, since if one of those isn’t written carefully, it becomes un-auditable.

The first example I used was CIP-014. I said “Three good examples of this are CIP-014 R1, R4 and R5. In a post last year, I discussed two entities (from the same Region) that both told me the same story: They had been dinged by an auditor for not taking specific steps to protect transformers located in their substations in scope for CIP-014. Their mistake was taking the words of these three requirements literally, since all three only talk about protecting the substation itself, not any equipment located in it.”

In the post I referred to, I had pointed out that auditors (from the same region) gave one of these entities a Potential Non-Compliance (PNC) finding (which can lead to a violation finding), and the other an Area of Concern (which is a non-mandatory recommendation to remediate a problem discovered by the auditor) because they had focused on protecting the whole substation, not particular pieces of equipment in it (in particular transformers). The problem is that all three of these requirements refer only to protecting the substation; nothing about equipment in it. Each of these entities had engaged an outside firm (different ones) to develop their threat and vulnerability assessment (mandated by CIP-014 R4), and the threats identified in that were all just to the total substation. So their physical security plans (mandated by R5) just focused on mitigating those threats.

Both of these entities were cited for not specifically including the transformers in their physical security plans. Yet R5 just states that the entity needs to develop a physical security plan “that covers their respective Transmission station(s), Transmission substation(s), and primary control center(s).” Notice there’s nothing about protecting transformers or other equipment here.

I got an email the next day from an auditor, who said that “CIP-014 requires a risk assessment and then a physical security plan for those assets that are identified in the risk assessment.  The plan has to address physical security measures that ‘deter, detect, delay, assess, communicate, and respond’ to potential physical threats and vulnerabilities that were identified during the vulnerability assessment conducted upon the identified assets.”

He then went on to describe specific physical threats against transformers, and said these need to be protected against in the physical security plan. Since what he said sounds like good advice for anyone protecting a substation, I am reproducing it below. But in response to his sentence I just quoted, I responded that, while both entities deserved to receive an Area of Concern notice (since CIP-014 came about because of the Metcalf attack, which disabled transformers), neither of them had violated the strict wording of any of the CIP-014 requirements, so a PNC should be out of the question.[i]

The auditor’s reply led with the assertion that “the substation is simply a container of stuff, and CIP-014 expects you to protect the stuff.” He went on to give some good physical security observations (which I also reproduce below), and then concluded “Just remember, administrative law is largely based on what a reasonable, qualified person would do.  The auditor has to determine if what the entity did was enough to meet the stated objective of the requirement.  If the auditor finds that the entity failed to achieve the objective, the auditor will find a PNC.  We really do not need highly prescriptive requirements in order to audit.”

My reply simply said that either he or I might be right, but that my point in writing the last post (which I now realize I didn’t actually state in the post – my bad) was to provide advice to the SDT, that might let them avoid a mistake like the CIP-014 SDT seems to have made[ii] by not explicitly stating in R4 that the threat and vulnerability assessment needs to look at threats to the Facilities (i.e. the equipment) in the substation, not just the total substation itself. If they had just included a sentence to that effect in CIP-014 R1, R4 and R5, we wouldn’t have to talk about these auditing problems with plan-based requirements like these, and probably in a couple years with CIP-013-1 R1.1 (see the second end note for more discussion on this).

The auditor’s CIP-014 compliance advice
(from the auditor’s first email)
“CIP-014 requires a risk assessment and then a physical security plan for those assets that are identified in the risk assessment.  The plan has to address physical security measures that “deter, detect, delay, assess, communicate, and respond” to potential physical threats and vulnerabilities that were identified during the vulnerability assessment conducted upon the identified assets.

“So, what does that mean.  Yes, you need to mitigate against the threat of a malicious actor entering the physical confines of the substation.  However, you also have to consider and address threats and vulnerabilities that can be exploited from outside the perimeter fence line.  For example, I can take a .50 cal Barret rifle and punch holes in a transformer from a considerable standoff distance, as long as I have line of sight target acquisition ability (although a good old AK-47 works quite well as was demonstrated in the Metcalf attack).  That is a vulnerability.  How do I address that?  By blocking or preventing the line of sight in some manner.

“Transformers are, in a sense, big boxes.  And, they are high dollar, extremely long lead time items to replace if destroyed (last I heard, it can take 18 months or longer to get a new 500 kV transformer, and they are not built in the USA).  My substation perimeter fence will deter and delay someone from gaining physical access to the transformer.  And I can have sensors and camera systems to detect a breach of the fence line.  The fence will not deter a standoff shooter who can see the “box” in the weapon’s sights.  So, I have to somehow prevent the line of sight target acquisition to mitigate that vulnerability.  I do that with tall ballistic barriers (e.g., concrete walls) around the substation perimeter if the terrain is flat and there are no high points that can peer over the barrier.  But if there are hills, trees, or other high points offering a shooter a look down – shoot down advantage, I have to move the barriers closer to the potential target (it is all about the angles).  Of course, if I own the land, I can cut down the trees.  I can put anti-climb devices on the nearby transmission towers.  I have to do something to deter a shooter from afar.  The vulnerability assessment, if properly performed, will have identified the target lanes where a shooter can acquire the transformer as a target.  If I do not address that vulnerability, then my plan is inadequate.”

(from the auditor’s second email)
“As a furtherance of my comments, I would point out that the entity if required to perform a vulnerability assessment.  So what are some possible vulnerabilities?  Immediately coming to mind are:

·         Physical intrusion into the substation yard (with or without gaining entry into the control house)
·         Can include climbing the fence, cutting the fence
·         Can include vehicle-based breach
·         Weapons discharge into the substation yard from outside the fence line
·         Lofted bombs (explosive, incendiary) from outside the fence line
·         VBIED (Vehicle-Born Improvised Explosive Device) – inside the perimeter after penetration, or outside the fence line where the blast perimeter reaches to critical equipment
·         Airborne (drone) delivered explosives
·         Launched or airborne delivered metallic material designed to short out equipment

Some things you can protect from by mitigating measures (airborne threats not so much).  Relying on local law enforcement response is a non-starter since response time far exceeds the exploit time.  Response time is very much dependent on where the substation is in relation to LEO and could easily exceed 30 minutes in many places.  Therefore relying on cameras and sensors to watch an attack unfold as your primary defense is not all that helpful.  To the extent that you can deter an attack, the better off you are.  That means penetration-resistant perimeter barriers, line-of-sight obscuration, outward-looking camera systems with analytics capability, lighting considerations (including normally dark, only lighting up when a perimeter breach is detected), two-way voice communication from the SOC, etc.  Wall heights and barrier placement are driven by local conditions.  Lighting and audible alarms/communications may be limited by local ordinances).  But the bottom line is that a chain link fence with a padlock is effective for a few seconds at best.  Hardly a delaying property and certainly not a deterrent.”

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at Please keep in mind that if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post – especially on compliance with CIP-013. And if you’re a security vendor to the power industry, TALLC can help you by developing marketing materials, delivering webinars, etc. To discuss any of this, you can email me at the same address.         

[i] I believe the entity that got an AoC was originally going to receive a PNC, but they successfully fought back against this auditor. The other entity either was cowed into not doing this (I believe the auditor was the same person), or perhaps the fact that they were one of the first entities audited for CIP-014 in this region worked against them.

[ii] Although I pointed out at the end of the post from last year that this mistake – and another I discussed briefly – can readily be excused by the fact that FERC gave NERC only 90 days to draft, ballot and approve the standard, and have it to FERC to sign. When you set a very aggressive deadline like that, it’s almost inevitable that mistakes will be made – and in this case the biggest mistake seems to have made CIP-014 R1, R4 and R5 mostly, if not completely, un-auditable.

Unfortunately, the CIP-013 drafting team seems to have made a very similar mistake. As I’ve pointed out previously, CIP-013 R1.1 mandates that the entity “identify and assess cyber security risk(s) to the Bulk Electric System from vendor products or services…” – but it doesn’t provide any guidelines on what types of risk need to be addressed.

So an auditor who thought that a particular risk like “the vendor will buy chips from the cheapest source, without fully vetting those sources for trustworthiness” should be addressed in the entity’s plan will be in the same position as the regional auditor who felt strongly that the two entities I wrote about last year should have included measures to protect their transformers, not the whole substation. The auditor might be absolutely right from a security point of view, but if the requirement doesn’t state particular classes of risk that need to be addressed (as is done in CIP-010 R4 Attachment 1, which I think is the best plan-based requirement so far), then there is nothing that can be audited, other than whether or not the entity produced any sort of credible plan.

No comments:

Post a Comment