I recently
wrote a series of posts about “plan-based” requirements (e.g. CIP-010 R4 and
CIP-013 R1) and raised two main questions about them. The first was whether
they could be strictly audited using the standard NERC auditing framework
(which is embodied in the Compliance Monitoring and Enforcement Plan, or CMEP).
My answer to that question was that they are auditable to various degrees
(depending on how they are written), but none of them are auditable in the
strict sense that the prescriptive CIP requirements (like CIP-007 R2) or the
Operations and Planning requirements (like FAC-003 R1) are auditable.
The second
question was more important; I only delved into this question in the last
post in the series. This question is effectively
[i] “Given
that the real goal of auditing is promoting the reliability and security of the
Bulk Electric System, is it possible that trying to audit plan-based
requirements (which, as I’ve pointed out several times, are the wave of the
future for NERC. In fact, all of the important new CIP requirements developed
since CIP v5 have been plan-based) using the standard NERC auditing framework
will actually hinder this goal?”
And my
answer to that question was yes. In that post, I referenced a previous
post
I had written on CIP-014 enforcement.
CIP-014 was the first plan-based standard to be approved by NERC, and the NERC
Regional Entities are already auditing it. In this post, I recounted a
conversation I’d had with a CIP physical security compliance person at a very
large utility, who had been rebuffed by his regional auditors when he asked
them a question about whether a particular technology – that this entity
proposed to implement in their substations subject to CIP-014 – would likely be
determined to be appropriate to include in the Physical Security Plan required
by CIP-014 R5.
He was
flatly turned down when he asked this question, on the grounds that answering
it would constitute a violation of the principle of auditor independence: If
the auditors answered it for him now, when they came to audit him later
(perhaps years later), they would in effect simply be auditing themselves. On
the face of it, this seemed to be the only possible answer that the auditors
could have given. But the unfortunate result of this was that the utility he
worked for would most likely cancel their plans to implement this technology
(which would cost $80 million to deploy to all of their CIP-014 substations).
In my post, I
pointed out that this clearly seemed to be a case where the standard NERC audit
framework was actually working against the goal of enhancing the physical
security of the BES. I further hinted that, given the choice between
maintaining that standard framework and securing the BES, I would choose the
latter any day. I was thus preparing to suggest that the standard NERC auditing
framework – which works very well for the NERC Operations and Planning
standards, but not for the CIP standards, and especially the plan-based ones –
be replaced with a new auditing program just for the CIP standards.
However, as
has happened before, an auditor had written in to me about this issue, and in
an email dialogue he pointed out that not only is it not necessary that there
be a new auditing framework, but that the elements required to deal with this
problem are already in place in at least two of the NERC Regions, and could be
implemented in the other Regions as well. In the rest of this post, I won’t
usually quote the auditor directly, but I will include his ideas, as well as my
interpretation of them, without necessarily saying at every point whether they
are his or my ideas. After I initially wrote this post, I sent it to the
auditor to review for any mis-interpretations on my part, and he corrected
those.
Before I
discuss this further, I want to make sure we all understand what the big
problem is. It is
not that plan-based
requirements are not very auditable (if they are auditable at all) under NERC’s
CMEP; I’ve already stated that I consider auditability to be a distant second
to the main concern. The main concern is the security of the BES, and the problem
is that, as exemplified in the case I just discussed, that goal will not be
aided if, for plan-based requirements, NERC entities can’t get their NERC
Region to review their plan before they implement it. Additionally, when it
comes to implementing the plan, the entities would be greatly helped if they
could ask their Region to review the implementation while it’s in progress and
point out potential problems. The case just discussed is an example of how
auditing concerns can prevent NERC entities from getting the advice they need on
complying with plan-based requirements. As we have just seen, this problem has
already appeared for CIP-014, and it will appear in spades when FERC approves
CIP-013 and entities start working seriously on their supply chain cyber
security risk management plans.
[ii]
In my
opinion, here is what is needed to address this problem:
- NERC entities, when faced with plan-based CIP
requirements, will of course first have to develop the plan mandated by
the requirement (the Physical Security Plan mandated by CIP-014 R5, the
supply chain security plan mandated by CIP-013 R1, the Transient Cyber
Asset/Removable Media plan mandated by CIP-010 R4, etc). In the process of
developing the plan, they need to be able to ask their Region questions
about what should be in the plan, what are best practices for mitigating
the threats addressed by the plan, etc.[iii]
- Once they have developed their plan, the entity needs to
be able to take it to their Region and ask them to review it. The review
won’t tell the entity whether the plan is “compliant” or not; rather, the
reviewer will point out whether the plan doesn’t address any threats that
should be addressed in the plan, and whether the mitigations proposed
follow best practices as the Region understands them. If the entity can’t
get this review, they will have to take a deep breath, hope the plan
they’ve developed is one their region will think is good, and then
implement the plan. The danger is that they may go a long way down the
road to implementation (or even finish it) before their next CIP audit,
and that the auditors will then tell them the plan had a lot of problems
and needs to be redone. Of course, that could possibly lead to a lot, or
even most, of the work the entity has done implementing the plan needing
to be re-done as well.
- If the Region does the review and sees problems with the
plan, they will point those out to the entity. At that point, the entity
could elect to re-work the plan to fix the problems, or else to dismiss the
Region’s advice if they think it isn’t well-founded for some reason.
- If the entity did re-work their plan, they would be
well-advised to ask for a new review from the Region, to make sure they
have addressed whatever objections the Region brought up.[iv]
- Once the entity was satisfied that it had a good plan, it
would start implementing it. However, at any point during the
implementation, the entity could request of their Region that they review
the entity’s implementation work so far, and let them know of any
developing problems they see (e.g. that the entity isn’t implementing
everything in the plan, or that they are implementing parts of it badly).
- If the Region does point out problems with the
implementation, the entity has the choice either to try to address these
problems or to dismiss them if they don’t think they actually are problems
that need to be fixed – just as in the case of the plan review.
In reading
this, you may have already thought of the objection that first came into my
mind when I realized these steps would be required in order for the entity to
be sure they had a good plan (and that they were implementing it correctly):
How would it be possible for the Region to provide these services to the
entity, then turn around and audit them later without having their auditor
independence completely compromised?
The key to resolving
this question is that the Region will need to have what is known as an Entity
Development program in place; currently, one Region does have such a program,
and I was told another Region is now putting one in place. The point of this
program is for the Region to have a formal way of providing advice, like the
above, to entities outside of an actual audit
[v]. In
general the staff members for this program will be separate from the auditors,
although the auditor pointed out to me that it isn’t impossible for the CIP
auditors to also provide Entity Development services, assuming the entities
trust them not to mix the two functions.
One absolute
requirement for the Entity Development staff is that they be knowledgeable in
the subject matter of the plans they are providing advice on – for example, if
they are providing advice on the Physical Security Plan in CIP-014, they need
to understand physical security for substations. Of course, this requirement isn’t
something to be taken lightly, since such people – with electric utility
experience – may be hard to find. So putting together this staff may be a multi-year
process.
While the
Entity Development staff will in theory just be providing best practices-type
information to the entity, it is likely that they will sometimes, in the
process of reviewing an entity’s plan, discover some element of non-compliance.
When this happens, they will point this out to the entity, but they won’t be
able to provide advice on what the entity should do to remediate this
non-compliance; that would probably
compromise auditor independence.
Of course,
the Entity Development staff wouldn’t report these instances of possible
non-compliance to the auditors, and the report wouldn’t become part of the
record for the entity’s next audit. However, when receiving a report like this
from Entity Development, the entity that requested the plan review will need to
decide whether to self-report a violation; and if they do self-report, they
need to also “discover the scope and extent of the non-compliance” (to use the
auditor’s words), as well as mitigate the non-compliance
[vi].
If the
entity does self-report, and the issue that was the subject of the report is
discovered as part of the next audit, the entity won’t be subject to a
Potential Non-Compliance (PNC) finding for the same issue, covering the same
time period that was self-reported. Of course, if the entity doesn’t
self-report and the potential violation is discovered, the entity would be subject to a PNC, which of
course will be more serious since it was discovered at audit.
The auditor
did also point out that there
is a
way that the Region can provide advice on whether a plan is likely to be found
compliant or not, but they can only do this during the period before a standard
becomes enforceable. If the entity develops their plan and specifically asks
the Region for a “readiness assessment” of the plan, then, depending on
available time and resources, Regional staff (either auditors themselves or
Entity Development staff) can perform, to quote the auditor, “a non-binding, no
risk, no consequence ‘audit’” of the requirement for developing the plan.
[vii]
For example,
suppose your entity wishes to have your Region review your CIP-013 supply chain
cyber security risk management plan before the CIP-013 compliance date (which I
am currently expecting to be toward the end of 2019, assuming FERC approves
CIP-013 this spring). You would of course have to first develop the plan (and
this would have to be done well before the compliance date), then request a
readiness assessment of your compliance with CIP-013 R1.
Of course,
developing your CIP-013 plan in the first place will require having some idea
of what should be in the plan. There is the 13-page Implementation Guidance
produced by the CIP-013 drafting team; this is a very good document as far as
it goes, but it is nowhere near a comprehensive guide to developing the plan. There
will also be more guidance coming from other sources (including specifically
the North American Transmission Forum, although I’m not sure that will be
available to non-members), and NERC is now considering a CIP-013 “Transition
Study” similar to the one for CIP v5. In this study, a small number of early
adopters will share their experience with NERC, to help them develop Lessons
Learned (remember those?) and other guidance.
And there
will be another source of guidance: I have been thinking about what should be in
the CIP-013 plan and discussing this with an auditor, and I plan to write a
series of posts (probably not consecutive posts, of course) about this
question. Unlike the NATF, I’m not on NERC’s list of approved guidance
providers, so you’ll of course have to take whatever I say with a grain of salt
(but in fact the same applies to the approved providers like NATF, since NERC
specifically says that no guidance these approved providers turn out they will
itself be “approved” by NERC). But I hope you’ll find my posts on this subject
to be helpful, and I’ll welcome any feedback you have on them.
The views and opinions expressed here are my own, and do
not reflect those of any organization I work with. If you would like to comment
on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.
[i]
I say “effectively” because I now have a better way of wording the question
than I did when I wrote that post.
[ii]
This is because the requirements in CIP-013 provide very little “guidance” on
what should be in a good supply chain security plan, even less so than the
guidance that CIP-014 R5 provides to guide entities in developing their
physical security plans.
[iii]
Of course there is, and will be, guidance provided on these questions by
various industry organizations. However, the guidance NERC provides will be
very limited due to NERC’s limited interpretation of what guidance they are allowed
to provide. NERC entities will need to weigh all the guidance they find, but in
the end what will count most for them is what their Region says. This is the
way it has been since CIP version 1, although this trend intensified with CIP
v5.
[iv]
It bears repeating that the Region’s review of the plan won’t be for the
purpose of saying whether it is compliant with the requirement or not, but only
a) whether all of the threats that should be mitigated in the plan are in fact
addressed; and b) whether the proposed threat mitigations actually follow best
practices. It is possible that whoever reviews the plan will notice something
in the plan that is non-compliant and will point this out to the entity; it
would then be up to the entity to decide whether they want to revise their plan
to address this concern, or whether they think the observation is mistaken for
some reason. In any case, the observation made by the Region wouldn’t become
part of the audit record, and wouldn’t be passed to the auditors when the
entity was next audited.
[v]
The auditors have always been able to point out Areas of Concern covering cyber
security practices that aren’t within the scope of CIP, when they notice
something regarding the entity’s practices during the course of an audit. But
there has never been an official way for them to provide such advice outside of
an audit – and, as I pointed out earlier, an audit will often come a year or
two after the entity has started implementing their plan. It will be much
better if the plan can be reviewed as soon as it is developed.
[vi]
The auditor did explicitly warn against what might be a temptation: agreeing
with the opinion that your plan was non-compliant in some way and fixing
whatever the problem was, but then still not self-reporting the issue. While it
is true that the friendly advice of possible non-compliance that you receive
from Entity Development will not in any way be reported to the auditors (and
even if it were, they would ignore it), it is still very likely the auditors
will discover that at a certain point your documentation changed, from
reflecting the old non-compliant wording in the plan to reflecting the new
compliant wording. Of course, the penalty for non-compliance discovered in an
audit is likely to be much more severe than for a self-report.
[vii]
Of course, the readiness assessments are nothing new. The Regions conducted a
number of these during the run-up to the CIP v5 enforcement date. They were
very helpful, both to the entities that received them and to the auditors that
conducted them. However, the auditor did caution that there is no way that the
readiness assessment will be able to issue an opinion that the plan seems to be
compliant. The team will point out gaps in compliance and recommend steps for
remediation; after that, the entity is on its own to determine what it should
do with the information.