Kevin Perry on auditing CIP-013

Kevin Perry, who was Vice Chairman of the CSO706 Standards Drafting Team (which drafted CIP version 2-5, although Kevin left the team when he became an auditor) and went on to become the Chief CIP Auditor of the SPP Regional Entity for ten years until he retired in 2018, provided a very insightful response to my post last week asking how CIP-013-1 R1.1 would be audited. I’m reproducing it below, followed by my comments. Kevin wants you to know that “I am no longer a Regional auditor and my thoughts do not necessarily reflect what NERC and the Regions will do.  I just know how I would approach the subject.” He said:

I do not have any insider information or insights, so I am not claiming to know how NERC and the Regions will conduct the audits.  I would hope that this has been discussed and trained upon at great length among the Regions so that there is consistency across the board.  Of course, I shall not hold my breath that Nirvana has been achieved in that regard.  I do know that if I were to audit CIP-013, R1.1, I would ask “have you considered…” and listen to their answer.  Maybe they did, maybe they did not.  I cannot imagine a documented plan that enumerates and discusses every possible risk for every possible vendor or provider.  And, of course, every entity is different and the applicable risks that need to be addressed will be different as well. 

Now, if they stare at me like deer caught in the headlights, then I can see a PNC coming.  But if they are able to discuss what they considered and why they chose to leave a particular concern off the list, then even if I disagree with them, I would likely not find a PNC.  Even if they admit that they hadn’t thought about that particular issue.  I might issue an Area of Concern or a strong recommendation, but I do not, as an external auditor on a limited engagement, have sufficient risk profile information for the entity to assert the entity flat out missed the target.  And I do not expect every entity to be overstocked with risk management experts.  But they should have a reasonable understanding of their risk profile and they need to be able to articulate that risk profile information and their rationale for choosing the risks that they addressed.

This strikes me as exactly the right approach. R1.1 requires the entity to “identify and assess” supply chain cybersecurity risks to the BES (it doesn’t say “to the BES”, but since the CIP standards deal only with cyber risks to the BES, this should be understood here as well). So why not find out if the entity did actually identify and assess risks by asking them what they did? If it never even occurred to the entity to do what the requirement said in plain English, that’s a problem.

However, I don’t think even Kevin’s “deer-in-the-headlights” entities should get PNCs, unless they simply refuse to consider this at all (as seems to be the case with the entity I wrote about in this post. The management that made that decision was specifically warned that this was a mistake, but they wouldn’t even make a small gesture toward identifying and assessing risks. IMO, they deserve to have the book thrown at them), or if their plan sounds like they did identify and assess risks, but in fact they can’t explain what they did.  

I say this because neither NERC nor the Regional Entities have made any official effort to explain to the NERC entities what “identify and assess” means in R1.1; so the entities that were legitimately confused shouldn’t be fined because of their confusion. NERC could change this situation at any time. As I said at the end of the original post, it makes no sense to let NERC entities misinterpret a CIP Requirement, but not let them know this until their next audit. Believe it or not, the CIP requirements (all of them) are enforced in order to mitigate cybersecurity risks to the BES. Not explaining to NERC entities what they need to do to comply with them just leaves the BES unprotected from those risks for years longer than it needs to be.

In other words, I think it’s time for NERC (actually, the whole NERC community) to rethink what auditing means when it comes to cybersecurity standards. I realize that they interpret GAGAS as prohibiting NERC and the Regions from providing anything that smacks of compliance guidance. I think that is a mistaken interpretation of GAGAS, but it especially doesn’t make sense when we’re talking about a requirement that simply requires “identifying and assessing” risks.

 The ultimate goal should be protecting the BES.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.