You too can waste BIG money on CIP-013 compliance! (part two)

Part one of this series used the example of EEI’s way-over-the-top supply chain security contract terms (and how one big IOU almost made the mistake of trying to use them!) to show how a NERC entity can impose huge unnecessary costs on itself (in this case, costs to their legal staff) in the name of CIP-013 compliance. However, it turns out I way understated the over-the-topness of this document.

The problem I discussed in the first post was that EEI’s language went far beyond what was needed for compliance with CIP-013 R1.2.5 and addressed a number of other threats, besides the one that R1.2.5 requires the entity to mitigate. Of course, there’s nothing wrong with going beyond what is strictly required – in fact, R1.1 requires that you do go beyond the six (really eight) threats in R1.2.5, and “identify and assess” other threats as well. As I’ve discussed in several posts, this means the entity needs to identify other threats, then decide for itself which ones it will mitigate (and that decision can be based on cost, which is the first time you’ve been allowed to make such decisions in NERC CIP).

But the point is that you need to decide for yourself (i.e. for your organization) what supply chain risks you’ll mitigate in CIP-013; you don’t want to have the decision made for you by somebody at EEI who thought it would be kinda nice to insert some extra contract language. As I pointed out in the post, every term you include in a contract is going to require a lot of lawyer time to respond to vendor requests to modify that language; negotiate on what acceptable language is; get approval for this exception, etc. etc. And if there are 100 vendors this applies to, you might conceivably have to do this for a large number of those – and that is for every term that you put in the contract.

In my post I identified four unneeded terms included in the EEI language, and I estimated that there are at least 50 total unneeded terms in the whole document, although that’s probably low. But let’s say that, for each of those 50 terms, 20 vendors out of the 100 will object to the language and try to get it modified (as I also pointed out in my last post, this is what the vendor’s lawyers are paid to do. They’re there to reduce the vendor’s contract compliance burden, and they’ll reflexively object to almost anything. In fact, 20 out of 100 might be a low estimate for the number that would object to a term). And assume a lawyer will need to spend at least 10 hours dealing with each objection (again, probably low). 20 vendors X 50 terms X 10 hours means you’ll be investing 10,000 legal hours, if you seriously try to implement all of the unneeded terms in the EEI document. And let’s say they get paid $100 an hour fully burdened (probably that’s low, of course).

So this is $1 million to implement the unneeded language in the EEI document (on top of what you’d spend to implement the language that is needed, although see my comment below about relying on contract language as a last resort for vendor risk mitigation, never as a first resort). And God help you if you have a problem down the road with a vendor and you decide you need to audit their compliance with these terms – which keep in mind aren’t needed for compliance in the first place.

But there’s another cost – perhaps more serious – in tying up so many lawyers for so much time, which the lawyer who prompted the first post brought up in her email: negotiating with all of these vendors will take lawyers away from their other work like approving contracts. So this will delay all procurements (she estimated 2-3 months, which would be huge in itself. But I’m sure she was thinking of just a small number of unneeded terms, not at least 50, like I see). And that is going to have an impact on BES reliability, as well as all the other functions of your organization.

Of course, since there are a number of threats your organization will decide to mitigate in CIP-013, you will probably need to address at least some of these in contract language (although I always think of contract language as the last resort. If you can get the vendor to agree to do something by email or verbally, why not go with that for now? If six months later you find out they haven’t kept their promise, then you can move to contract language. But you should never reflexively decide up front that you have to have contract terms for every threat you’ve decided to mitigate, and for every vendor). But the point is to only do that for the threats that matter to you, not the ones that are in your contract just because someone at EEI thought it would be cool to throw some extra terms in their document.

And there’s another big potential cause of wasted legal effort in CIP-013 compliance: Inserting language that addresses threats that are irrelevant for CIP-013. Remember, CIP-013 deals with threats to the BES – which specifically means control systems (ICS). The number – and magnitude – of threats to control systems is dwarfed by the number of threats that target IT systems. And when you look at expected loss from an IT attack, the difference is even greater. There have only been two documented cyber attacks on control systems that caused loss of load – both for short periods of time. On the other hand, just one attack on IT systems, NotPetya, caused an estimated $10 billion in damage worldwide.

When you look at a document like NIST 800-53 or NIST 800-161, probably 80-90% of the threats described don’t apply to ICS. That’s because IT systems are valuable because they hold or use valuable information; control systems are valuable because they control important processes. Concerns like data privacy, protection of intellectual property, non-repudiation, separation of duties, etc. – none of these are at all relevant to ICS.

That’s why it was disheartening to see that EEI ­included a whole section of contract language about encryption (p. 13). Encryption has nothing to do with ICS security. Unless you’ve decided that your ICS are a wonderful place to store your Protected Health Information or your customer credit cards, the only information you need to protect on ICS is information on the configuration of the devices themselves. And God forbid you should encrypt it! That would be a wonderful way to make your BCS totally unreliable (“I’m sorry about the outage, ma’am. You see, our PKI had a glitch today and our control systems couldn’t verify any of the certs they needed to decrypt their configuration information. We’re pretty sure we’ll have you up and running by say next Thursday. What’s that you say, ma’am? Where do you want me to put my PKI?).

And besides the reliability impact of including encryption in your ICS, you’ll have the legal time impact: There are about 8 terms related to cryptography X 20 vendors X 10 hours per vendor is 1600 lawyer hours. Times $100/hr. makes a $160,000 cost just for dealing with the cryptography terms.

But the EEI document has an even bigger source of potential cost in it that frankly makes everything I’ve discussed so far look like small potatoes indeed. At various places in the document, they refer to NIST publications as kind of a shorthand for listing out a bunch of other contract terms (in other words, the vendor has to read the references, then decide for itself how they’ll actually comply with those terms. Of course, this is all in the contract, meaning they’ll in theory be hung out to dry legally if they don’t interpret the NIST references to your taste. Come to think of it, that sounds a lot like some other cybersecurity standards for critical infrastructure that I’ve heard about – I forget the name right now, or perhaps I’ve repressed it. What could possibly go wrong with this?).

For example, one term in the EEI document is:

(d) Contractor shall implement a vulnerability detection and remediation program consistent with NIST Special Publication 800-53 Rev. 4 RA-5, SA-11, and SI-2, as may be amended.

Let’s look at just one of these references, SI-2. Here is the control itself:

SI-2 FLAW REMEDIATION

Control: The organization:

a. Identifies, reports, and corrects information system flaws;

b. Tests software and firmware updates related to flaw remediation for effectiveness and potential side effects before installation;

c. Installs security-relevant software and firmware updates within [Assignment: organization-defined time period] of the release of the updates; and

d. Incorporates flaw remediation into the organizational configuration management process.

How many individual controls are included in this? I’d say about 10 or 12. So these are effectively 10 or 12 more terms for the contract. And this is just one reference, of say about 20-25 in the EEI document. If they all have 10 or 12 terms in them, then that’s between 200 and 300 more terms that are being added. And since the $1 million cost estimate above was just for 50 terms, we’re talking about $4-6 million in additional cost for including the NIST terms.

But there’s more. On page 15 of the EEI document, under Contractor Cybersecurity Policy, there’s this paragraph:

Contractor will provide to Company the Contractor’s cybersecurity policy, which shall be consistent with NIST Special Publication 800-53 (Rev. 4) as may be amended. Contractor will implement and comply with that cybersecurity policy. Any changes to Contractor’s cybersecurity policy as applied to products and services provided to Company under this Agreement and Company Information that are inconsistent with the security requirements of NIST Special Publication 800-53 (Rev. 4) as may be amended shall be subject to review and approval by Company prior to implementation by Contractor.

So now we’re saying that the vendor’s cybersecurity policy must be consistent with the whole of NIST 800-53, which runs 462 pages in the PDF that I have (and here we’re not just talking about the controls in Appendix F. We’re talking about the entire 800-53 program), and the slightest deviation from that policy needs to be reviewed and approved by your lawyers. And of course, this goes for all of your say 100 vendors subject to CIP-013 (and one entity estimated to me that they had in the hundreds of vendors of BCS hardware and software).

At this point, we start asking questions about what the liquidation value of the utility would be, since obviously it will have to stop wasting money generating, transmitting and distributing electric power and just focus all of its resources on what’s important: implementing and enforcing the EEI contract language!

I just looked at the EEI document to see if it was released on April 1. But no, it was released in March. Too bad. That would have explained a lot.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post – especially on compliance with CIP-013. My offer of a free webinar on CIP-013, specifically for your organization, remains open to NERC entities and vendors of hardware or software components for BES Cyber Systems. To discuss this, you can email me at the same address.