My most
recent post
discussed the important question of the role of procurement contract language
in CIP-013. My general point in the post was that I think it is a big mistake
for a NERC entity to focus their CIP-013 compliance efforts solely, or even
primarily, on getting the vendor to agree to incorporate particular language in
their contract. After that post came out, an auditor wrote in to suggest some
of my statements needed clarification. We exchanged a number of emails, and I’ve
identified four important points that he raised.
I.
Tales
from Beyond the Scope
Early in the
post, I quoted from a note that appears after CIP-013 R2: “the following issues
are beyond the scope of Requirement R2: (1) the actual terms and conditions of
a procurement contract; and (2) vendor performance and adherence to a
contract.” Regarding (1), I had this to say: “The first part of the second sentence…says
that the terms and conditions of a procurement contract are ‘beyond the scope’
of R2. What does this phrase mean? It specifically means that contract language
is not the subject of R2 – and if an auditor asks to see an entity’s vendor
contract, they can be turned down because it isn’t in scope.”
It turns out
the truth is more nuanced. To quote from the auditor’s email: “For compliance
finding purposes (i.e., determination of a PNC) the contract is out of
scope. However, if the entity states
that the vendor will (or has promised to) do something, such as advise its
customers within x days of identification of a vulnerability in its products, I
want to see something from the vendor that demonstrates that, not just a verbal
statement of assurance from the audited entity.
In many cases, the MSA, SLA, or contract T&C will address that
expectation and it is appropriate for the entity to provide the written vendor
assurances.”
What does
this mean? It means that, if you assert that a vendor has promised to do X in a
contract, the auditor can (and most likely will) ask for evidence of this. If
you refuse to show him the contact and point to phrase (1) above, the auditor
will probably say “OK, Mr. Smart Guy. Show me some other evidence they’ve made
this promise.” At this point, you’ll wish you had a letter or email from the
vendor making the same promise; presumably, you won’t feel you’re violating
some legal principle by not showing him one of these.
But what if
your only evidence is the contract – and you’ve been told by your lawyers never
to show the contract to the auditors, even if they jump up and down and promise
divine retribution for this refusal? In this case, the auditor technically
can’t issue a PNC (potential non-compliance) finding, but they can certainly
list in their report that you were unable to substantiate your assertion
regarding the promise by vendor Y to do X.
So if you
are going to be at all constrained in your ability to show a contract to the
auditors, why even take the contract route in the first place? Get a letter
from your vendor. This constitutes obtaining a vendor promise just as much as
contract language does, and it will make your audit go much more smoothly. As the
auditor pointed out above, other documents from the procurement process, such
as an RFP stating terms the vendor agreed to, a Master Services Agreement or a
Service Level Agreement, might also constitute evidence of the vendor’s
promise.
II.
Trying
is Everything
What happens
if you beg your vendor to promise to do something and they refuse – yet
dropping them as a vendor is out of the question because of the cost of making
the change, the unavailability of close substitutes, etc? Will you receive a
PNC because of this? The auditor further states in the same email: “The key
point in the contractual language exclusion is that the auditor cannot hold the
entity at fault with a PNC if the entity was not successful in getting the
vendor to agree to a CIP-013 requirement expectation. And, failure of the vendor to comply with its
contractual obligations or promises does not constitute a compliance failure on
the part of the audited entity. But the
entity needs to demonstrate how it addressed the expectation during its
procurement process. The Plan should
address how the entity will go about asking and the pre-contract negotiations
and perhaps elements of the contract itself will demonstrate the Plan was
followed.”
In other
words, your supply chain cyber security risk management plan (which R1 requires
you to develop, of course) must describe how you will go about requesting that
your vendor promise to do something, as well as the different vehicles
available for the vendor to document that promise. For each vendor, you should
document how you tried to get them to make the promise. If they agreed to make
it, then you need some document to verify that. If they didn’t agree, you
should document that fact as well. To quote the auditor again: “The entity does
need to keep records of any communication with the vendor; preferably date,
time, with whom, and a summary of the discussion or agreement. I like emails better than phone logs for the
obvious reason; emails are better evidence than one-sided hand-written records.”
The auditor
also points out that you don’t have to simply throw up your hands if the vendor
tells you “no” the first time you ask for something, even though you know that
finding another vendor for the same product isn’t an option, for whatever
reason. How about negotiating, using tools that you (as a compliance or cyber
security person, not a supply chain or legal person) have access to? The
auditor also says “…if I cannot get, for example, prompt notifications of
vendor staff terminations, then maybe I should reconsider my willingness to
allow electronic or unescorted physical access to my BES Cyber Assets. That is called a negotiation point.” Good
idea!
Before I
leave this topic, I want to point out something else: Your supply chain cyber
security risk management plan must list risks and how you intend to mitigate
them. Your first step to mitigate a particular vendor risk may well be getting
the vendor to promise to take certain steps that will mitigate the risk. But if
the vendor absolutely refuses to do this, are you now off the hook for that
particular risk? Will your mitigation be “Well, we tried our best, but that
vendor is as stubborn as a senior citizen mule!”?
Unfortunately,
this isn’t mitigation – this is making an excuse for non-mitigation. There
should usually be some step you can take to mitigate the risk in question,
which doesn’t require the vendor to do anything at all. Let’s go back to the
example the auditor just used, the risk that a vendor won’t provide prompt
notification when an employee leaves who had access to your systems (this is
the risk addressed by R1.2.3). The auditor suggested that, if the vendor
refused to promise to give you this notification, you might then refuse to give
their employees electronic access and/or unescorted physical access to their
systems at your facilities. This isn’t just a negotiation tactic. It’s also a
legitimate mitigation for this risk, since an employee who had just been fired
and wanted to take out his anger on your systems would have a hard time doing
so without electronic or unescorted physical access. He/she would have to come
onsite and be escorted to the systems!
III.
Just
Do It!
In my last
post, I made this assertion: “I don’t think the entity needs to show any
procurement documents at all, because I don’t think the entity needs to prove
that they ‘initiated correspondence’ with the vendor to implement certain
controls, if they can show other evidence of compliance. And what other
evidence can the entity show? The best evidence will demonstrate that the
vendor is actually doing what they were supposed to do! After all, isn’t the goal of CIP-013 to
protect the BES against supply chain risks, not simply document that you’ve
asked your vendors to do certain things?”
In this
statement, it seems I was flat out wrong. While it is important that you gather
some evidence to show the vendor is actually keeping a particular promise[i], that
evidence alone isn’t sufficient. In his email, the auditor further stated “I do
have a problem with a “handshake” agreement.
I cannot evaluate that – it is essentially an attestation in the absence
of supporting evidence. Yes, the entity
can show me that they received a notification of a vendor staff
termination. But, what prompted the
notification? Was there a contractual
expectation or a vendor declaration that such (notification) would be sent by
the vendor consistently and in a timely manner?
How do I know that the vendor will inform me of every applicable
termination or reassignment? Similarly,
how do I know that the vendor will apprise me of every security vulnerability
in their products?”
In other
words, if a vendor makes a promise, it needs to be made in writing in some way
(although contract language is just one of those ways). You also need to
document either a) that the vendor is keeping that promise (using emails, etc);
b) that you tried to get the vendor to promise something but you were turned
down; or c) the vendor promised to do something but didn’t keep the promise. If
either b) or c) is the case, you also need to document some other mitigation
steps you took to address the risk in question – assuming there is a way you
could mitigate the risk without the vendor’s cooperation (obviously, as
required in R1.2.4, if the vendor absolutely refuses to tell customers about
vulnerabilities it knows about but hasn’t yet made public, you will not be able
to take steps to address those vulnerabilities - except perhaps by simply
tightening up the normal security measures that would be applied to that
system).
IV.
The
Big Guys
Finally, the
auditor also made a good point about how you mitigate vendor risks when it
comes to big vendors like Microsoft, who obviously aren’t going to change their
contract language for a single electric utility, no matter how large it might
be. He says “…the plan can certainly address how procurement from the huge
international players like Microsoft, Red Hat, Cisco, and Oracle will be
handled. I would prefer to see you get
your networking gear from a Cisco channel partner, not eBay or some unknown
used equipment reseller. I would expect
you to download software from a known, reputable source, not Sammy’s Software
Shack. Hopefully the vendor provides a
digital signature (MD5 or SHA(2)-256 hash) of its software for verification
purposes. It is all about risk
management. You cannot eliminate all
risk, but you can certainly reasonably mitigate it.”
If you would like to comment on what you have read here, I
would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that
Tom Alrich LLC can help you with NERC CIP issues or challenges like what is
discussed in this post. To discuss this, you can email me at the same address or
call me at 312-515-8996.
[i]
Although you definitely don’t have to document every instance where the vendor
kept the particular promise in question, as would be the case if we were
talking about a prescriptive requirement like CIP-007 R2.
No comments:
Post a Comment