I’ve been
working almost full time on CIP-013 compliance since the beginning of this
year, and I’ve always been very concerned about making sure that clients
carefully target their supply chain risk mitigation efforts, so they don’t
waste money.
Believe it
or not, wasting money on CIP-013 compliance is really a new concern in the NERC
CIP world. When complying with the prescriptive CIP requirements (with CIP-007
R2 and CIP-010 R1 being the poster children for that), waste really isn’t the
question. You are required to do certain things at certain times, and you need
to invest whatever it costs to be assured that you’re doing the right things
and you’re doing them at the right times. If you pass the audit for a
particular requirement, not a single penny that you spent on compliance with
that requirement was wasted. If you don’t pass, the whole amount you spent was
wasted.
But CIP-013
is very different, of course. R1.1 requires you to “identify and assess” supply
chain security risks, and then (implicitly) requires you to mitigate the most
important of those risks (since there’s no way any entity could ever mitigate
all, or even a significant fraction of its supply chain security risks). But,
other than the fact that you need to include in your plan the six risks (really
8) mandated by R1.2.1-R1.2.6, CIP-013 provides no further guidance on:
- What risks you should identify, or even how you should go
about identifying risks;
- How you should decide, of the risks you’ve identified,
which ones to mitigate and which to accept; and
- To what degree you should mitigate each risk. Should you
apply so many mitigations that the risk level asymptotically approaches
zero? Or should you just do a pro
forma mitigation of each risk (such as simply sending an email to each
vendor giving them a list of things you’d like them to do, with no effort
later on to verify that they’re actually doing it), which will in fact
mitigate almost no real risk?
But just
because there isn’t any guidance from NERC on these questions
[i] doesn’t
mean you shouldn’t think about them. Granted, you can’t be held in violation if
you don’t address them, but your goal should be more than compliance: It should
be making sure that, no matter what amount of resources (staff time and funds)
you have available to you for CIP-013 compliance, you are using them as
efficiently and
effectively as possible. Because after all, supply chain risks are
probably the fastest growing set of risks in cybersecurity today (just ask the
Russians, who are
routing
close to 100% - if not 100% - of their attacks on the US electric power grid
through vendors). You (and the BES) can’t afford to waste resources in this
effort, especially if you don’t have adequate resources to begin with (and
raise your hand if you have all the resources you need for supply chain
security…yup, I didn’t think I’d see any hands).
I see three
main ways in which a NERC entity can waste resources on CIP-013 compliance:
- Scatter their resources on partially mitigating a lot of
supply chain security threats (probably without realizing they’re doing
that), without actually mitigating any of them (and I define mitigation to
be lowering the level of risk posed by a particular threat from medium or
high to low, for a particular vendor, product or service).
- Pour resources into over-mitigating a small number of
threats, while barely addressing other threats at all, even though they
may pose the same – or even a higher – level of risk.
- Devote resources to mitigating risks that don’t even apply
to BES Cyber Systems.
However,
while I’ve been sure for a long time that it’s possible to waste serious money
on CIP-013 compliance, until fairly recently I struggled to identify what was
the most likely way this might happen. Then, a month or so ago, a lawyer with a
very large IOU contacted me about a problem she had – and voila! In one
document I found great examples of all three ways to waste money. Moreover,
it’s a document that is likely to be put into practice by a number of large
NERC entities.
The document
is EEI’s “Model Procurement Contract Language Addressing Cybersecurity Supply
Chain Risk”. EEI released it in March, and while I certainly looked at it at
the time, I admit that it didn’t make much of an impression on me, good or bad.
However, the lawyer said her team was working on procurement language for
CIP-013 compliance, and she was being asked to base that procurement language
on this document. She said she thought the document was kind of “over the top”
and would require her to devote a lot of resources to CIP-013 contracts, as
well as delay procurements by months. She wondered if I agreed with her opinion.
So for the
first time, I went through the EEI document in detail, and I came to the
realization: While some parts of the document are OK and might even be
worthwhile examples to follow, others would require a huge effort and lots of
resources to implement properly. In fact, the document provides great examples
of all three wasteful practices listed above. Before I elaborate on that
statement, here’s a little background on the document.
As I’ve
already pointed out, CIP-013 R1.1 requires NERC entities to identify and assess
supply chain cyber security risks, but it doesn’t provide direct guidance on
what risks should be considered, or even where they can be found. However,
R1.2.1 – R1.2.6 state six mitigations that must be included in your plan (and
since R1.2.5 and R1.2.6 include two mitigations each, it’s really eight in
total). With a little wordsmithing, you can “go behind” each of these
mitigations, to identify the threats they’re intended to mitigate (I prefer the
term threat to risk, since what is needed here is simply a statement of
something bad that can happen, not referring to its likelihood and impact. When
you later consider likelihood and impact to determine the risk posed by the
threat, then it becomes a risk. This approach is consonant with NIST 800-30,
although I’ll admit I didn’t develop it based on that).
For example,
R1.2.1 reads “Notification by the vendor of vendor-identified incidents related
to the products or services provided to the Responsible Entity that pose cyber
security risk to the Responsible Entity.” The threat behind this mitigation can
be found by inverting the wording: “A vendor won’t notify the Responsible
Entity of an incident related to products or services it provides to the RE.
Because the RE doesn’t know about this incident, it isn’t able to take adequate
steps to mitigate the risk(s) to the BES that were revealed by the incident.
This leads to a compromise of a BCS and damage to the BES.” Note that your
statement of a threat for CIP-013 should always end with an adverse impact on
the BES. If you can’t think of a BES impact for this threat, then you shouldn’t
be considering it under CIP-013 (although you might consider it in your supply
chain cyber security risk management plan for IT systems).
When you
consider supply chain threats in general for CIP-013 plan, you need to include
the eight R1.2 threats in the plan, regardless of whether you would do that if
they weren’t mandatory. Fortunately, all of the R1.2 threats are important
ones, so they would almost all probably make the cut anyway. This means that
you need to describe in your plan how you will mitigate each of the R1.2
threats.
There are
various tools you can use for supply chain risk mitigation. One tool is RFP
language; you can include risk mitigation measures as deliverables of the RFP
itself. Another is vendor questionnaires, in which you ask the vendor to
describe how they have mitigated each of the vendor (or
supplier)
threats; you can then use their responses to develop a risk score for each
threat, for each vendor. These let you precisely target your procurement
mitigations, instead of basing them on an overall vendor risk score, which
applies to all threats and requires the same mitigations for any vendor with
the same overall score.
Of course,
contract language is another tool for mitigating vendor risk, and the EEI
document lays out what they think is appropriate language for mitigating the
eight R1.2.1 – R1.2.6 threats. I do want to mention that simply getting the
vendor to agree to contract language doesn’t in itself mitigate any risk; you
need to take some steps to verify they’re doing what they promised to do – just
like you would do if the vendor makes their promise in a letter, an email, or
simply a phone call.
So how will
adapting the EEI contract language as your own lead to your wasting money?
Let’s start with the first way described in item A above: scattering your
resources among a wide variety of supply chain risks, and not mitigating any of
them very well.
You may
notice a contradiction here: I’m saying the EEI document encourages you to
waste your resources by addressing a wide variety of supply chain threats, yet
I also said the document was developed to provide contract language
specifically for the eight threats implied in R1.2.1 through R1.2.6. Which is
it?
The problem
is that the people who wrote the EEI document included a lot of language that
addresses threats other than those included in R1.2. For example, R1.2.5
requires mitigation of two, and only two, threats: a) the threat that a malicious third party will intercept a patch while you're downloading it from a trusted site and insert a
malware-laden substitute; and b) the threat that you will download inauthentic software or patches, either from a fake site that's a good facsimile of the trusted site or from the
trusted site itself, which has been compromised - with inauthentic patches
inserted – by a third party.
However, EEI's language for R1.2.5 includes this:
Contractor shall identify the country
(or countries) of origin of the procured product and its components (including
hardware, software, and firmware). Contractor will identify the countries where
the development, manufacturing, maintenance, and service for the product are
provided. Contractor will notify Company of changes in the list of countries
where product maintenance or other services are provided in support of the
procured product. This notification shall occur 180 days prior to initiating a
change in the list of countries.
This one
paragraph (and there are many more like it throughout the document) identifies
four threats that are different from the two threats that are the subject of
R1.2.5. These threats are:
a) A
supplier will provide the entity with a product or product component that has
been manufactured or developed in a suspect foreign country.
b) A
product that is shipped back to the supplier for service will in fact be
serviced in a suspect country.
c) Even
though a supplier has informed the entity of the country(ies) where service is
performed and they were acceptable to the NERC entity, the supplier
will in fact send a product to a different country that is a suspect one (“Oh, we
forgot to tell you. We normally ship our products to Japan for service, but
because we got a great deal this one time, we sent your product to North Korea
instead. Hope you don’t mind…”).
d) The
supplier will notify the entity of an upcoming change in service countries,
from an acceptable country to an unacceptable one, close to the date when that
change is effective, when it will be too late for the entity to do anything
about it.
Now, I’m not
disputing that all of these threats might be worth addressing in contract
language (or with other mitigation tools). But the NERC entity needs to identify the
threats it wants to mitigate at the beginning of the CIP-013 process, when they
“identify and assess” risks in compliance with R1.1 - EEI shouldn't have made this choice for the entity. I’m sure that whoever
wrote the EEI document didn’t understand that they were addressing a lot more
threats than advertised, but that’s exactly what they did (I haven’t counted
them, but I’m sure there are at least 50 threats addressed in EEI’s language,
that don’t have anything to do with the R1.2 threats that are the ostensible
object of this document).
You might
ask “Why is it such a big deal if you have a bunch of other threats addressed
in your contract? Maybe the supplier will actually take them seriously; maybe
they won’t. But there’s no harm in just including them.” To be honest, I might
have said something like that, before my email conversation with the lawyer.
She said “What I don’t want to do is overburden the procurement process with
unnecessary language, or have to evaluate the acceptability of each and every
deviation proposed by a vendor. That might add months to the procurement
process.”
From experience,
she knows that the supplier’s lawyers will always try to minimize the
compliance burden on their employer, by proposing deviations from the entity’s
proposed language – after all, that’s what they’re paid to do. There will
almost inevitably be a back-and-forth between the two sides, until they have
arrived at an acceptable term (or the NERC entity has been convinced to remove the
term altogether). This will have to be done for every term. Again, if the term is there to mitigate a threat that
the entity has decided is important to mitigate, that’s fine. It’s worth the
lawyers’ time (and the delay in the procurement process) to do this
negotiation. But I really don’t believe your lawyers will be too happy when
they’re asked to negotiate terms relating to threats that you don’t think are
important enough to mitigate in the first place!
And the
lawyer didn’t mention another big cost to having superfluous language in your
contract: You need to take steps to verify that the supplier is actually
keeping their promises. This doesn’t mean you have to do an expensive audit of each supplier.
I think a questionnaire is fine for this purpose, as long as it was made clear
in the RFP or contract that the supplier will need to fill out a security
questionnaire at some regular interval.
But keep in
mind that there is a cost to the supplier for every question they answer. Since
it’s very unlikely that the person answering the questions will have all of the
answers at their fingertips, they will have to spend a lot of time trying to
find the right person to answer each question, asking the question multiple
times if they’re very busy, etc. While it’s not likely that the supplier will
tell you they’re just too busy to answer your questionnaire and they wish you
good luck with the next sucker…er, supplier…that you choose to buy from, you
can be sure that, especially if your questionnaire has lots of questions,
you’ll sooner or later pay the price for this in one way or the other. Plus,
you will need to evaluate the supplier’s response to every question in the
questionnaire, and that requires a lot of valuable time on your end.
Similarly to
the case with the lawyers, if you’re sure that every question in the questionnaire
is precisely targeted to identify how the supplier has mitigated a threat that
you’ve decided you need to mitigate, this cost should be acceptable to you,
since the questionnaire is the best way to determine what steps the supplier
has taken to mitigate each threat. But if you have a bunch of questions that
are only there because of some language that you blindly copied from the EEI
document into your contract, you’re once again wasting your resources.
In part 2 of
this post, I’ll address even more exciting ways that the EEI document can help
you waste money! Please try to contain your excitement.
Any opinions expressed in this blog post are strictly mine
and are not necessarily shared by any of the clients of Tom Alrich LLC.
If you would like to comment on what you have read here, I
would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that
if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or challenges
like what is discussed in this post – especially on compliance with CIP-013. My
offer of a free
webinar on CIP-013, specifically for your organization, has received a
great response, and remains open to NERC entities and vendors of hardware or
software components of BES Cyber Systems. To discuss this, you can email me at
the same address.
[i]
One of the
five
white papers recently published on NERC’s web site by the Supply Chain
Working Group of the NERC CIP Committee discusses these issues, but doesn’t
constitute compliance guidance (as the disclaimer says at the beginning of the
paper). It’s the one called “Supply Chain Cyber Security Risk Management
Lifecycle”.