Today, NERC
put out a draft
Data Request. NERC entities will have 3 weeks to make comments
on it. The purpose of this DR is to help NERC staff determine “whether new
information supports modifying the standards to include low impact BES Cyber
Systems with External Routable Connectivity” (and by “the standards”, NERC
means the Supply Chain Standards, CIP-013-1, CIP-005-6 and CIP-010-3).
I will point
out to start that, if NERC is really just interested in whether Low impact
assets with External Routable Connectivity should be included in CIP-013, I can give them
a quick answer: There aren’t any Low impact assets with ERC, so it doesn’t
matter if they’re included or not. ERC (capitalized, since it’s a NERC Glossary
term) only applies to High and Medium impact assets. The definition refers to
the Electronic Security Perimeter, and only Highs and Mediums have ESPs.
But I
digress. This DR wasn’t a surprise to me, since I and a small subcommittee of
the NERC CIPC’s Supply Chain Working Group (which is itself a subcommittee of
the CIPC, making the small group a sub-sub-committee, for those keeping score
at home) have been drafting this with Howard Gugel, NERC Director of Standards
Development, for more than a month (Howard came to the group with a first
draft, but what came out in the end was completely different). I’ll say at the
outset that there were some very interesting discussions over that time, and
this version was approved over my strenuous objections (not that there was a
vote, but it seemed pretty clear that I was the only person still willing to
voice objections at the end of our discussions, which was two weeks ago. What
was sent out today seems to be close to what was approved then, but is somewhat
improved – at least in the first section).
To spoil the
ending for you, my objections to this draft are:
1) There are
two overall sections in the DR. The first starts on page 5 under the heading “BES
Cyber Systems”. I think this section is confusing and misuses terms, but it
could be saved with some important changes (it was worse two weeks ago, but it
was somewhat cleaned up after it went from the SCWG to NERC). As it stands now,
I doubt it will provide much useful data.
2) The
second section begins on page 7 under the heading “CIP-013 Cost of
Implementation”. This can’t be saved, and needs to be put to sleep with the
fishes, as they say in “The Godfather” (or, if you prefer, it needs to be “terminated
with extreme prejudice”, as the CIA agent says to Martin Sheen in Saigon at the
beginning of “Apocalypse Now”. I just realized that both of these great movies
are from Francis Ford Coppola! No wonder they’re the two that seem relevant
here).
3) But my
biggest objection isn’t to this draft, but the whole idea that there is a need
for a DR in the first place. I think this reflects very bad strategy on NERC’s
part. What’s driving this DR – judging from what I’ve heard from people at NERC
and elsewhere, and drawing a couple conclusions of my own from what I’ve heard –
is that people in Congress are itching to see much tougher cyber regulation of the
power industry, and they’ve keyed in on the idea that this should start with more
controls on Lows (and CIP-013 would probably be the logical place to start with
Lows).
The data
generated by the DR won’t satisfy Congress (as I’ll show later on), and the
fact that NERC will spend 4-7 months drafting the DR, getting comments,
gathering responses, analyzing them and then generating some sort of report
might very well be held against the indusry. This means that the hawks in Congress
are still likely to require that Lows be included in CIP-013 (and FERC will go
along with them), regardless of the DR results, meaning the DR is probably a big waste
of everybody’s time. Even more importantly, if this all comes to pass a big blow will have been delivered
to the idea that the electric power industry should be both developing and
enforcing its own cybersecurity standards in the first place. Need I point out that that isn't good for NERC or the industry?
I had
thought I could easily write about all three objections in one post, but when I
got to page 4 of the draft and I was still nowhere near finished with the first
objection, I decided to leave the other two objections for the second post. I expect I’ll
have that up on Sunday evening, so your otherwise dreary Monday-after-a-long-July
4th weekend will be greatly enlivened by Part II of this post – or maybe
not. In either case, I think I’ll have it up Sunday evening.
Now to my
first objection, which consists of about 7 or so sub-objections on the first
section of the DR. The first sub-objection is to the title of the section
itself, which is “BES Cyber Systems”. Those of you who survived the CIP v5 and
v6 rollouts probably have your antennae up when you hear that this is the title
of the section, since a DR regarding Low impact assets shouldn’t be talking
about Low BCS at all (reasons to follow).
To give you
some background on this section, the DR started off (in the first draft, that NERC
provided to the SCWG) requesting rough numbers of Low impact BES Cyber Systems
with and without ERC – and when a few of us pointed out that ERC doesn’t apply
to Lows, this was changed to “external routable connectivity”.
I and a few
others on the sub-sub-committee pointed out next that, while some of the
Regions have in fact requested lists of Low BCS from their member entities, since
CIP v5 came into effect (where the Low/Medium/High impact levels were
introduced) the standards have stated explicitly that an inventory of Low BCS
isn’t required; the fact that NERC was just requesting rough numbers in the initial draft of the DR really
didn’t change that at all – there would definitely be a lot of resistance to
this question among NERC entities.
That
objection didn’t seem to change things, but what finally did was when I pointed
out in a later meeting that I knew one large NERC entity that stated in a Regional
compliance meeting a few years ago that they had in the thousands of relays in
their Medium impact substations, but they were grouped into only 2 or 3 (I
believe) BES Cyber Systems (of course, this is perfectly legal). But I also
know one even larger utility that has declared every Medium or High impact BES
Cyber Asset to be its own BCS. So, if those two entities both have say 5,000
relays at Low assets, the first entity might report that it has say five Low
BCS, while the other will say it has 5,000 Low BCS. If hypothetically they were
the only two entities responding to the DR, NERC would dutifully say that the average
large NERC entity has between 2502 and 2503 Low BCS! Obviously, such data are
totally useless. That finally seemed to put the kibosh on requiring entities to
estimate Low impact BCS numbers in the DR.
We then agreed
that we should ask for numbers of Low assets with and without risk factors like
external routable connectivity, dial-up access, etc. I suggested to keep it
simple: Ask the entities to just estimate numbers of Low assets with erc,
dial-up, and maybe a couple other risk factors – as well as those without any
of these, of course (I also suggested breaking each of these down into Control
Centers, substations and generating stations). As you can see, the first section
of NERC’s draft DR goes beyond what I suggested, and breaks each type of asset
down by three “Location Risk Scores” (so you have three types of Low Control
Centers, three types of substations, etc. - and questions on the risk factors
for each of these types).
I have no
objection in principle to that change, but the problem is that it’s been worded
very poorly. Were these problems to be fixed, I think this would be a usable
question, but that’s only if the DR
goes out in the first place (and for that issue, see the second post). However,
I think the following problems first need to be fixed.
The first “question”
starts off on page 5 by saying “In order for NERC to understand the data they
receive from this Data Request, they need to understand the basis for each
entity’s answer. This means that how your entity categorized your BES Cyber
Systems can have a huge impact on these survey results.” OK, fair enough.
The rest of
that paragraph is “In order to have Data Request results that can be used and
compared, the common basis are the six locations called out in CIP‐002. This
Data Request is focused on those locations and not how entities have designed
their BES Cyber Systems.” OK, that also sounds reasonable. Let’s go to
CIP-002-5.1a to determine what NERC means by “locations”. Hmmm, it seems that
the word “locations” isn’t used in CIP-002-5.1a at all, except in a few of the
bright-line criteria in Attachment 1 (e.g. Criterion 2.1), where it has a very
specific use that has nothing to do with what we’re talking about here. So what
does NERC mean by this term?
Two
paragraphs below, NERC says “The term “location” refers to physical space
associated with an asset.” And there is a footnote that says in part “For the
purpose of this data request, the word “asset” is used in the same way as it is
used in CIP‐002‐5.1a Requirement R1.” Since what they’re referring to is the
six types of assets listed in R1.1, I’ll combine the three statements to yield
what I think is the meaning of what NERC’s asking: “This Data Request is
focused on the physical spaces associated with assets (i.e. the six asset types listed) in CIP-002-5.1a Requirement R1.” I assume “physical space” means the
land on which the asset is built and the air for a certain distance above the
land.
Isn’t this
kind of odd? You’re being asked how many physical spaces you have that house
Low impact assets; I guess that makes a little bit of sense. But look at the “Location
Risk Score Table” on page 5. Let’s take the row that starts with “3.3” (but any
row will do) and continues with “Generation resources”. It provides location
risk scores based on megawatts at the location. Do the dirt and air in which a generating
plant is built generate megawatts of power? No, the plant itself does, which is
an “asset” in R1. Why not just ask about the asset in the first place?
And while we’re
on page 5, move up to the table before this one. That asks for “Number of
assets containing BES Cyber Systems”, broken down in four ways. That comports
perfectly with R1, but it doesn’t with the rest of this section of the DR,
which of course is after “locations” not assets. And a final confusion
regarding “location” is found in the introduction to the table on page 7, which
says “You may group assets with the same answers into a single line item.”
However, of course, the table below it is asking about locations! The word “asset”
is nowhere to be found there. It seems whoever wrote that really thought he or
she was talking about assets anyway.
Again, why
is “location” being asked for at all in the DR? I asked this of the person on
the SCWG that drafted the version of the DR that’s the basis for what NERC sent
out today. He said it had to do with the fact that SPS/RAS (Special Protection
Systems/Remedial Action Schemes) can be at multiple locations, and it’s important
to protect each of those locations. Presumably, NERC wants an entity to count
each of the locations that contains part of an SPS or RAS as a separate
location for the purpose of this DR. But nowhere in the DR does it say this! So
why even ask for locations in the first place?
This is a
classic case of the tail wagging the dog. For every Low SPS/RAS, how many Low impact
Control Centers, substations and generating plants are there? My guess it’s in
the high hundreds, if not over a thousand. So NERC is confusing the whole first
question because of a need to get SPS/RAS right, when they are a very small
percentage of the total Low assets. This could be handled much better by replacing
all the references to “location” with “asset”, and simply putting a footnote on
the last line of the table on page 6, explaining that the entity should count
each location of a part of an SPS or RAS as if it were its own asset, or
something like that.
And while we’re
on the subject of misrepresenting CIP-002-5.1a R1, let’s go to footnote 6 on
page 6. There we read “If your entity has performed generation segmentation and
created multiple low impact BES Cyber Systems, please account for them as
individual low impact BESCS as per your CIP‐002.”
I don’t want
to go into detail on the reason this is a misrepresentation of what Attachment
1 Criterion 2.1 says (misunderstanding would be a better word); if you would
like to research this fascinating subject, I refer you to at least four or five
posts I wrote about this criterion alone in 2014 and 2015, when people were
trying – mostly unsuccessfully - to figure out CIP version 5 (but don’t ask me
which posts they were. Who wants to read all that stuff?).
I’ll just
point out that the footnote sounds as if the entity is being asked to enter
numbers of Low BCS (or BESCS) in this table, when of course the idea is that it’s
just Low assets – or locations or whatever. This may be explainable by the fact
that I think this table was originally drawn up when the DR
was actually asking about Low BCS; it
was probably never changed. I did point out to the person that drafted the
final SCWG version of the DR that the wording of this footnote wasn’t really
what he wanted to ask – and I suggested wording that I thought would work. But
this change wasn’t made, as well as all of the other changes I suggested at
that time (basically, everything that’s in these two posts was rejected by the sub-sub-committee. But
don’t worry, I can take rejection. In fact, the fact that nobody listens to
what I say
saved
my life earlier this year, as well as won me a prestigious Russian medal –
which I’m still waiting for, but the Russians assure me it must have been lost in the
mail and will reach me eventually).
And two
footnotes above footnote 6 (that’s footnote 4, for those who struggled with
math in second grade) we find this sentence: ‘“external routable connectivity”
refers to the requirements under CIP‐003‐7, Attachment 1, Section 3’. When I
and a couple others on the SCWG pointed out, early in the DR process, that the
capitalized ERC was just for Mediums and Highs, the person drafting this for
the SCWG obviously decided to find an official “definition” of erc (lower case).
If the term
Low impact External Routable Connectivity (LERC) were still in force, that
definition would have been perfect here. However, that definition was part of
the late, lamented CIP-003-6 R2 Attachment 1 Section 3, which never was implemented,
having been replaced – at FERC’s insistence - by CIP-003-7 R2 Attachment 1
Section 3, which will come into effect January 1, 2020. What has replaced it is
an operational “definition” which is effectively 16 pages long (occupying pages
32-48 of the Guidance and Technical Basis for CIP-003-7).
I think this
is actually a really great way of “defining” what external routable
connectivity means at a Low impact asset (since, of course, there is no ESP to
make it easy to write the definition). In fact, the
post
where I raved about the CIP Modifications SDT meeting that developed this
definition (written on the 4
th of July, 2016. Must have been a rainy
day in Chicago) became a sudden hit in the last couple of weeks, yet I have no
idea why - since I haven’t referred to it in a post for a long time. My guess/hope
is somebody realized it can be a guide to CIP-003-7 compliance, or more
specifically to
documentation of that
compliance. But I hope to write a post in the not-too-distant future to
actually spell out why I say this.
However,
great as this “definition” is, I really doubt there are 50 people in the NERC
community who really understand how that replaced LERC – and I doubt half of them
could clearly articulate this to you. So for NERC to suggest here that, if you
want to understand what external routable connectivity means, you just have to
understand CIP-003-7, Attachment 1 Section 3 is somewhat like saying that if
you want to understand why your car keeps rolling after you turn off the gas
but leave it in neutral, you just need to develop a thorough understanding of
Einstein’s theory of General Relativity, with maybe Quantum Mechanics and String
Theory thrown in for good measure.
So while it
may be technically correct to refer people to CIP-003-7 to understand erc, NERC
should instead provide an unofficial definition that people can understand without
undertaking a huge research project. I suggested a pretty simple explanation to
the SCWG team at one point, but that didn’t gain traction either. Maybe it will
now, although it doesn’t require me to write it. Anybody experienced in the
black arts of NERC CIP could write a simple explanation of erc in one sentence (that
doesn’t use the phrase Electronic Security Perimeter). Just try it.
I have just one
more issue with the first “question”, and then I’ll stop writing ‘til the
second post, where I’ll address my second and third objections (I heard loud
cheers from across North America when I said I’d stop writing. Don’t you know
that’s mean?). This has to do with the rows labeled f. and g. in the table on
page 7. These ask for “Number of locations with constant monitoring of remote
connectivity to a BES Cyber System” and “Number of locations participating in
industry programs” respectively (a footnote clarifies that “Industry programs
include CRISP, CYOTE and/or Neighborhood Keeper”).
What’s my
objection to these questions? I take you back to page iv of the DR, where (in
the second paragraph of the Background section) it states “…the Board directed
NERC to study the nature and complexity of cyber security supply chain risks, including those associated with
low impact assets..” (my emphasis). Returning to the table on page 7, I can
certainly understand how rows b. through e. are asking about potential sources
of risk. But f. and g. seem to me to be mitigations
of risk. Why is NERC asking about those here? It doesn’t seem to comport
with the purpose of issuing the DR in the first place.
I can think
of two possible answers to this question. The first is that NERC thinks both
constant monitoring and “industry programs” might be good things to require –
or at least strongly advise – NERC entities to deploy at BES assets (and
presumably not just at Lows). So they decided to piggyback this separate
question onto this DR. If so, I can understand that, but they should state this
up front, rather than slip the questions in and hope nobody will notice.
The other
possible answer is that NERC isn’t looking at just risks to the BES from Low
impact assets, but possible mitigations of those risks. So if they find that a
large number of Low assets participate in these programs, they can tell
Congress “I know we have a lot of Low assets and they all pose some risk, but
on the other hand these two mitigations are widely deployed, so a lot of that
risk is already mitigated.”
This is
again a perfectly legitimate objective, but my question is, why stop with these
two mitigations? There are a number of other mitigations which I’m pretty sure
are much more widely deployed at Lows, including regular patching (say every
two months), daily A/V signature updates, background checks, etc. Why not ask questions
about all of these as well?
Have a good
Fourth!
Any opinions expressed in this blog post are strictly mine
and are not necessarily shared by any of the clients of Tom Alrich LLC.
If you would like to comment on what you have read here, I
would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that
if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or
challenges like what is discussed in this post – especially on compliance with
CIP-013. To discuss this, you can email me at the same address.
