I have
already said
that Reliability First’s CIP workshop two weeks ago was the best regional CIP
meeting I have attended. Probably the highlight of the meeting for me was the
joint presentation by Lew Folkerth of RF and Felek Abbas of NERC (you can find
their slides here,
in the single file that includes all of the day’s presentations. Their slides
begin at slide 19). I’ll have at least a few more posts on points that were
made by Lew or Felek.
Lew addressed
a number of interesting topics, including the RSAW for the new CIP-003-7
standard; of course, the standard itself is awaiting FERC approval. One of his
points was a real lightbulb moment for me, which I’d like to share here. On
slide 68 in the second section, Lew listed what the new RSAW says regarding
auditing Attachment 1, Section 5 of CIP-003-7 (this is the new requirement that
addresses Transient Cyber Assets used at Low impact assets): “For Transient
Cyber Assets managed by the Responsible Entity in an ongoing manner, verify
that the Transient Cyber Assets have an effective means of mitigating the risk
of the introduction of malicious code onto the Transient Cyber Asset.”
Lew
emphasized the word “effective”, then pointed out that he thought this is
really the key to auditing non-prescriptive, results-based requirements
(although I prefer the term “objectives-based[i]”), such
as this one. That is, since this type of requirement only specifies an
objective that needs to be met, not the method to achieve it, there has to be
some criterion that the auditor uses to determine what is an acceptable method
and what is not.
For example,
in CIP-007 R3 (another objectives-based requirement), the entity is required to
achieve the objective of mitigating the threat posed by malware to BCS. Suppose
an SME at an entity told the auditor that, based on the advice of his
brother-in-law, his method of mitigating the malware threat to one or more BCS
is to say a certain chant every morning at 7 AM. I think the auditor would be
justified in finding the entity in violation - not just issue an Area of
Concern, as might be the case if the entity had chosen IDS signatures over
anti-virus or application whitelisting methodologies. In the latter case, the auditor
might issue an Area of Concern and ask the entity to either justify this
decision or implement a different solution. IDS signatures are a plausible
methodology for effectively mitigating the malware threat, whereas chants are
not (and please don’t send me emails arguing why chants are probably likely to
be as effective as IDS signatures! I pride myself on having a fairly open mind,
but I do have my limits).
Lew wrote an
article about auditing non-prescriptive CIP requirements for the January/February
RF newsletter, and I wrote about that article in my own post.
I just checked to see how the use of “effective” as a criterion fits into what
he said in that article. He lists four components of a good evidence package
for the requirement he wrote about in that article, CIP-010-2 R4 (of course,
another non-prescriptive requirement). The third component is that the plan must show “how methods documented in the plan achieve the
objectives” (the “plan” Lew refers to is the one required by R4. You could say
that the plan is the same thing as the objective of this requirement).
Of course,
the word “effective” isn’t in here, but I would argue that “methods that
achieve the objective” is the same thing as saying “effective methods for
achieving the objective”. So I call this a match (not that I would hold it
against Lew if his thinking had evolved since he wrote the article. My thinking
is always evolving - to put it kindly - and my unofficial motto is “Often
wrong, but never in doubt!”).
To sum up
this post, I think that the word “effective” (or an equivalent word or phrase)
should be understood (and if possible, explicitly stated) in every
non-prescriptive, objective-based requirement. This will effectively (I
couldn’t help that one. Sorry) indicate that the entity must not just utilize
one or more methods to achieve the objective, but that the chosen method must
be effective. Of course, none of the current non-prescriptive CIP requirements
(such as CIP-010 R4 and CIP-007 R3) currently use this word, but I imagine the
RSAWs effectively (OK, I did it again!) remediate that omission. In any case,
you should always understand that this word is at least implicitly in place.
As a postscript, I want to point out that one
questioner at the RF CIP workshop implied to Lew that the use of the word
“effective” would increase use of “auditor discretion”, and thus was a bad
thing. I can’t remember Lew’s answer, but I know my answer – if I were in Lew’s
place - would be: “The fact that this requirement is non-prescriptive means
auditor discretion will definitely be required, whether or not the word
‘effective’ (or its equivalent) is present in the requirement – and the
decision to make the requirement non-prescriptive was made by the Standards
Drafting Team, not me. However, as I discussed in this
post, auditor discretion is already required
to audit most of the current CIP v5 and v6 requirements – both prescriptive and
non-prescriptive - due to the presence of many ambiguities and missing
definitions. The auditor is expected (perhaps with assistance from the Regional
Entity) to use whatever training they have in legal logic to audit in spite of
these flaws.
“The difference with non-prescriptive
requirements is that the auditor is required to use discretion regarding
matters of cyber security, including making judgments about whether the entity
has used an effective methodology for addressing a particular requirement.
Since the auditors are chosen for their posts in part because of cyber security
expertise, not legal training, I think it is much preferable to have them
exercising judgment in cyber issues, rather than legal/logical ones. But in any
case, non-prescriptive requirements are clearly here to stay. No CIP drafting
team has drafted prescriptive requirements since v5; I predict that no more
will be drafted, regardless of what happens with the current prescriptive
requirements[ii] and
the current compliance
regime.”
Note: When I showed a draft of this post to Lew,
he commented that he wasn’t sure what his answer was, but he should have
referred to GAGAS, which requires use of professional judgment when performing
audits. So his answer would be something like “Use of professional judgment isn’t
the exception in auditing, but the rule. Even the “Bible” of our profession
requires that we exercise professional judgment, since no requirement ever
perfectly addresses every possible case you may throw at it.” Amen.
The views and opinions expressed here are my own and don’t
necessarily represent the views or opinions of Deloitte.
[i]
After reading a draft of this post, Lew commented that he prefers this term as
well.
[ii]
I used to view the current CIP v5 and v6 requirements as being almost entirely
prescriptive, except for a few notable exceptions like CIP-007 R3 and CIP-010
R4. I now think that the majority of the current requirements and requirement
parts are non-prescriptive, perhaps the great majority. I hope to sit down in
the not-too-distant future and determine whether each requirement and/or
requirement part is prescriptive or not. However, in my opinion there are a few
very prescriptive requirements – including CIP-007 R2 and CIP-010 R1 – that require
NERC entities to devote inordinate amounts of resources to them, way out of
proportion to whatever benefits they provide.
No comments:
Post a Comment