Judging by
the large number of page views, my recent post
on firmware – and the fact that firmware patches are subject to CIP-007-6 R2 –
has clearly struck a chord. The day after the post appeared, I received an
email from a well-known CIP auditor. As usual, this person had some very good
points to make. I reproduce his email in full, with my comments interspersed in
italics.
Auditor: First,
firmware is nothing more than software burned onto a chip. It might be
updatable (through a process of “flashing” the PROM), or it might require
replacement of the chip to implement the update. However, to emphasize
what you referenced in your blog, it is software and it is subject to CIP-007-6
until such time as an official, FERC-approved definition of Programmable
Electronic Device determines otherwise.
Tom: It hadn’t occurred to me that the
definition of “Programmable” – or rather the lack thereof – would have anything
to do with this. In any case, I wouldn’t count on a definition coming out any
time soon, and especially one that ruled out firmware updates being subject to
CIP-007-6 R2.
A: Probably
the biggest question is whether an update that provides for configuring and
entering a complex password constitutes a security patch. While I would
dearly like to assert that it is, I cannot. A security patch is an update
that corrects a vulnerability in the affected code. Modifying the code to
allow a better password is a feature update and does not address a
vulnerability. Therefore, the patch that allows for stronger passwords is
not subject to the CIP standards. Having said that, I am not convinced
that all Regional auditors share the same view, so the entity would be well
advised to have this discussion with their Region sooner rather than later.
T: This point was also made by Brandon
Workentin of EnergySec, in my long footnote quoting him. The bottom line is
that, if you want to be safe, you need to consider a firmware patch adding a
feature like stronger passwords to be a security update, even though it isn’t
one from a purely technical point of view. Of course, like all CIP questions,
you can always discuss this with your region. And like all questions since a few
weeks ago, you probably can’t count on your region’s answer (if you get
one) guiding your auditor, since your auditor may now be from FERC, not your
region. In general, it’s better to take the more “hawkish” interpretation,
until FERC comes out with guidance on these and a lot of other issues (and
there is no assurance now that they’ll come out with guidance on any
issue regarding CIP v5 interpretation).
A: And, to
confirm what others have stated, you have to assess security patches for
applicability. You do not have to implement them as long as you have a
mitigation plan that addresses the vulnerabilities covered by the patch.
I disagree that you have to test and determine with certainty that
implementation of the patch “breaks” the system before you can choose to not
implement the patch. It is well established that outside of the
traditional servers and workstations found in the Control Center, (1) updates
are often cumulative, meaning you cannot pick and choose what you are going to
implement, (2) vendors to this point have very frequently not identified
whether a patch or update includes security fixes, (3) vendors have been
reluctant to go back and try to provide information necessary to identify
security patches for updates released over the past many years, and (4) even if
the next released update does explicitly include a security patch, the fact
that updates are generally cumulative and the vast majority of the previously
released, unimplemented updates are feature releases still causes an entity to
appropriately be wary of installing the update. Extensive testing would
be required and even then there is no guarantee that some feature or function change
won’t be overlooked or misunderstood, introducing reliability risks that exceed
whatever the security patch may be addressing.
T: The second part of this paragraph brings
up something I’ve heard elsewhere: namely, that the patching process is very
different once you move away from the standard IT OS platforms and go to
devices like relays. As the auditor points out, the fact that the firmware
updates are often cumulative, and that security updates are often not
explicitly identified as such, means the entity is probably justified in being
very cautious about applying even updates that are identified as being for
security purposes. So having a way of mitigating the risk addressed by a
security update – without actually applying the update – is especially important
for non-standard OS’s.
A: The
bottom line is that the entities need to make sure they are on the top of their
game, that they have implemented as many defensive controls as they can
regardless of whether there is a pending security patch, and that they are
focused on cyber security rather than the absolute minimum necessary to
demonstrate compliance.
T: This
paragraph makes yet another good point: That it is important to have defensive
controls (like IDS or firewall rules) that go beyond immediate compliance
needs. You never know when a patch will come out that can’t be installed
without impacting reliability (or for another reason). This will require you to
have an alternative means of mitigating the risk, and those controls may turn
out to be absolutely necessary for compliance.
T: I
do have another point to make. I got an email from a compliance professional
that referred to the part near the end of the post where I mentioned that
firmware for video cards, etc. needs to be updated as well. This person said:
“In line with this question I feel like
there’s the potential for this to bring device drivers into scope…It could be
argued that on top of checking my source for firmware updates for my network
card, I also need to check my source for driver updates for the same NIC.”
I really
don’t see why this wouldn’t be the case. Device drivers would seem to have the
same status as firmware updates – they’re “software” updates that affect the
basic operation of the system.[i]
I can hear 1,000 hearts stop beating as I’ve just added yet another item to
your list of “Things I Need to Address in my CIP v5 Compliance Program.”
Believe
me, I’m not writing this post (and the previous one) because I’m trying to make
your job harder. But that is the way CIP v5 has been ever since its approval in
2013 (two years and two days before the day I’m writing this): as entities (and
auditors) get further into the implementation effort, they discover more and
more “implicit requirements”. These were in theory implied by CIP v5 all along,
but just haven’t been discovered until now. And guess what: These will
continue, probably long after April 1, 2016.
Here’s
a tip: I’m soon going to start writing about how I think CIP v5 needs to be
rewritten, so we can finally get off this hamster wheel of constantly-increasing
requirements, explicit or implicit. And I’ll be talking about it (although
“preaching” might be a better word) at Digital Bond’s S4X16
conference in Miami Beach in January.
The views and opinions expressed here are my own and don’t
necessarily represent the views or opinions of Deloitte Advisory.
[i]
I showed this comment to the auditor. He agreed with it, and had the following
additional comments: “The CIP standard is explicit in that the registered
entity is not expected to separately baseline drivers and other components that
come with the operating system (see the discussion found in the Guidelines and
Technical Basis section of CIP-010-1/2). But, video drivers and network
drivers are often provided by different vendors from the operating system
vendor and are separately installed from the operating system. In other
words, these drivers are not part of the installed operating system. So,
yes, you must baseline, track, and patch the nvidia display driver and the
Intel NIC driver software packages that were installed on the PC (and show up
as installed, versioned software under Programs and Features on a Windows
system). This auditor, at least, is applying the same concepts to
installed third-party software as apply to the operating system. You must
baseline and manage the package. You do not have to separately baseline
all of the drivers and other components comprising the package. Again,
registered entities should check with their Regions to make sure their auditors
share the same view.”
No comments:
Post a Comment