Monday, December 13, 2021

NIST’s hopefully-not-final ideas for an IoT device “labeling program”


I’ve been working with an interesting client since last spring: Red Alert Labs, a French company that specializes in helping IoT device manufacturers understand and comply with cybersecurity regulations and standards, as well as helping organizations that utilize IoT devices (of course, that’s just about every organization in the developed world) understand how they can do so securely.

In the course of that work, I’ve written several posts about what’s going on with IoT regulation in the US, the most recent of which is this one. In a nutshell, there are two current programs to regulate IoT in the US. One is the IoT Cybersecurity Improvement Act of 2020, while the other is the consumer IoT device labeling program, described in the Executive Order of this past May. Here is how the EO described the labeling program:

Within 270 days of the date of this order, the Secretary of Commerce acting through the Director of NIST, in coordination with the Chair of the Federal Trade Commission (FTC) and representatives of other agencies as the Director of NIST deems appropriate, shall identify IoT cybersecurity criteria for a consumer labeling program, and shall consider whether such a consumer labeling program may be operated in conjunction with or modeled after any similar existing government programs consistent with applicable law. The criteria shall reflect increasingly comprehensive levels of testing and assessment that a product may have undergone and shall use or be compatible with existing labeling schemes that manufacturers use to inform consumers about the security of their products. The Director of NIST shall examine all relevant information, labeling, and incentive programs and employ best practices. This review shall focus on ease of use for consumers and a determination of what measures can be taken to maximize manufacturer participation.

Does this give you a clear idea of what the program should be about? If you say yes, you either didn’t read this carefully or you’re a liar. This paragraph was deliberately made as vague as possible, in order not to constrain NIST in what they develop. That’s all well and good, but February 6 is the 270th day after the EO date, so by then, NIST has to have their final answers on both a) how the program will work, and b) what criteria will be used for the program.

NIST published draft criteria in August, and my guess is there wasn’t a lot of serious criticism of them (there was a comment period, and the comments – including the ones I helped prepare – seemed quite positive). They’re a subset of the criteria in NISTIRs 8259A and 8259B, which I think are excellent starting points for IoT device cybersecurity (NISTIR 8259A is about technical capabilities – or lack thereof – in the devices themselves, while 8259B is about the manufacturer’s practices and programs).

But besides publishing draft criteria, NIST was also required to draft how the consumer IoT device labeling program itself will work. That document appeared in early December, and was then discussed in a four-hour web meeting last Thursday (the recording should be available here by December 23). BTW, the webinar had 700 registrants from all over the world. I’ll bet the final number of viewers will be higher, after the recoding has been posted.

While the NIST presenters made a number of interesting individual points during the workshop, various issues were raised in the chat and as questions. Of course, the whole point of the workshop was to learn about what the issues were, and the staff seemed eager to hear about them. If you still want to comment, NIST will accept emailed comments – sent to labeling-eo@nist.gov - up until later this week (I believe the deadline is Thursday the 16th). Because NIST is under a very tight deadline for February 6 (the EO gave NIST a whole slew of deadlines for that date), NIST didn’t have a formal comment period this time.

While I have some minor issues with what NIST said, I have two major ones. These both have to do with what I see to be fundamental contradictions in what they’re proposing. If not addressed, I believe that either of these might sink the whole IoT device labeling program. I will address the first of those contradictions in this post, and the second in another post in the near future.

The first contradiction has to do with the type of label. This doesn’t mean whether it should be on white or yellow paper: NIST has already decided that there will be a small physical label attached to the IoT device (or perhaps included on a sheet of paper with the device), but that will have a URL or QR code that will take the consumer to a much more detailed discussion on the web.

NIST distinguishes between three different types of labels, in the answer to question 10 on pages 21 and 22 of the Discussion Draft: descriptive, graded, and binary. NIST defines them this way:

1.      A descriptive (or informational) label provides facts about properties or features of a product without any grading or evaluation.”

2.      “A tiered (or graded) label indicates the degree to which a product has satisfied a specific standard, sometimes based on attaining increasing levels of performance against specified criteria.”

3.      “A binary label (sometimes called a “seal of approval”) is a single label indicating a product has met a baseline standard. Examples include Energy Star [EPA], USDA Organic [USDA], and the government of Finland’s cybersecurity label [FINLAND].”

NIST indicates they have decided that a binary label is best, meaning the product meets the baseline criteria listed in the document (more on those in a moment); they set out their reasoning in the answer to question 11 on pages 22 and 23. While I’m not crazy about binary, up-or-down judgments in cybersecurity matters, there might be some cases where they would work – so for the moment, let’s just say I accept NIST’s decision.

However, this directly conflicts with another decision they have made: that the criteria should be “outcomes-based”, meaning that each criterion should list an outcome to be achieved, rather than a particular means of achieving an outcome (those of you who are veterans of the NERC CIP Version 5 Wars of the early part of the last decade – has it really been that long? – will remember lots of discussion of prescriptive vs. objectives-based requirements. Of course, objectives-based means just about the same as outcomes-based).

NIST didn’t just express a preference for outcomes-based criteria; they actually rewrote all of the criteria from the August paper as outcomes-based. For example, the August paper proposed this wording for the Logical Access to Interfaces criterion:

1. The ability to logically or physically disable any local and network interfaces that are not necessary for the core functionality of the product component.

2. The ability to logically restrict access to each network interface to only authorized persons or devices.

3. The ability of the product component to validate that the input received through its interfaces matches specified definitions of format and content.

4. The ability to authenticate individuals and other IoT product components using appropriate mechanism to technology, risk and use case. Authenticators could be biometrics, passwords, etc.

5. The ability to support secure use of authenticators (e.g., passwords) including:

a. if necessary, ability to locally manage authenticators

b. ability to ensure a strong, non-default authenticator is used (e.g., not delivering the product with any single default password or enforcing a change to a default password before the product component is deployed for use)

Of course, these sub-criteria are all quite measurable, but they don’t allow for a device to utilize other means to achieve the same goal of interface access control. Contrast the above to the wording for the same criterion in the Discussion Draft (although the criterion is now called Interface Access Control):

1. Each IoT product component controls access (to and from) all interfaces (e.g., local interfaces, network interfaces, protocols and services) so as to limit access to only authorized entities. At a minimum, the IoT product and its components shall:

a. Use and have access only to interfaces necessary for the IoT product’s operation. All other channels and access to channels are removed or locked down

b. For all interfaces necessary for the IoT product’s use, access control measures are in place (e.g., unique password-based multifactor authentication)

c. For all interfaces, access and modification privileges are limited for the interfaces and users of the interfaces

2. The IoT product, via some, but not necessarily all components, executes means to protect and maintain interface access control. At a minimum, the IoT product shall:

a. Validate data sent to other product components matches specified definitions of format and content

b. Prevent unauthorized transmissions or access to other product components

c. Maintain appropriate access control during initial connection (i.e., on-boarding) and when reestablishing connectivity after disconnection/outages

NIST has rewritten all of the other criteria from the August paper in the same way. Of course, I don’t object to outcomes-based criteria, and in fact I think that, in domains like cybersecurity and safety, they’re the only good option. However, it’s when you put outcomes-based criteria together with an up-or-down assessment program (in this case, a binary label) that you get a problem: The binary label essentially says, “This product[1] has met all of the baseline criteria established by NIST.” Someone  is going to have to make the decision whether or not this is true for a particular device, and that decision has to be up or down; it can’t be “Well, they did a good job in this area, but not such a great one in this other area.” How will the decision be made?

In the case of prescriptive criteria (like the criteria in the August paper), it should be fairly easy to make such an up-or-down decision. As I look at the five sub-criteria in the first example above (and the two sub-sub-criteria under sub-criterion 5), it seems to me that it shouldn’t be hard to come to an objective decision on whether or not a particular IoT product meets each of those criteria. For example, “The ability to logically restrict access to each network interface to only authorized persons or devices” could be assessed by a) identifying each network interface on the device (this would include both interfaces for network cabling, as well as wireless “interfaces” like a wi-fi radio), and b) verifying that the device allows access to each interface to be restricted only to authorized persons or devices.

Now let’s look at the first outcomes-based criterion from the Discussion Draft in the example above (the one starting “Each IoT product component…”), and specifically the three sub-criteria under it. Before I can determine whether a device satisfies those three sub-criteria, I need to answer a number of questions, including:

1.      How can any party but the device manufacturer itself determine which interfaces the device “uses and has access to”? That can’t be determined by looking at any physical characteristics of the interfaces. An assessor would need to examine the code that runs on the device. But even being able to look at the code won’t necessarily let an outsider determine whether a particular interface is used or not, unless they have experience writing device software. And let’s face it: examining all of that code would be a very expensive, time-consuming process. Plus this assumes that the developers of the software – usually not the device manufacturer – would be willing to provide the assessor with access to that code, in exactly the version that happens to be on the device. Oh…and don’t forget the developer of the real-time operating system (RTOS) that runs the device, like Microsoft or Wind River; they’ll need to fork over their code as well. Lots of luck getting that to happen.

2.      How can any party but the device manufacturer determine which interfaces are “necessary for the IoT product’s operation”? Even an examination of the code probably can’t answer this question. It really has to be the product’s designer, who presumably is an employee of the manufacturer. The designer might well be willing to answer this question for the assessor, but then the assessment becomes a self-attestation. There’s nothing wrong with self-attestations per se, but when an up-or-down binary label is on the line (and not receiving the label might have a serious negative impact on sales of the device), is it likely that the product’s designer will reveal anything that is going to jeopardize their being awarded the label? In other words, a binary label that depends on the self-attestation of the manufacturer isn’t worth the paper (or electrons) it’s printed on.

3.      In the third sub-sub-criterion, how does an assessor determine whether “access and modification privileges are limited”? Does “limited” mean nobody is allowed to make any change to the device at all? Or should some people be allowed to make some changes, and one person (the super user) can make any change they want? Or maybe anybody can make any change they want, except for say disabling the device altogether? These – and myriad other scenarios – could all be interpreted as “limited modification privileges”.

A binary label is a very powerful thing. Receiving it or not might in some cases mean the difference between commercial success and failure of a product. A manufacturer that doesn’t get the label is going to be quite unhappy and challenge how the assessor arrived at their judgment on each criterion. And that’s what this assessment will really come down to – judgment. The assessor is going to have to answer a huge number of questions like the three above. They won’t have any objective way to answer most of them (except in some cases by just appealing to the manufacturer for an answer, which then creates its own problems). They will have to use their judgment to answer those questions, so they can award (or not) the label.

What’s the bottom line on all this? If you want to assess based on very prescriptive criteria (like those from the August NIST document), then a binary label is achievable. But if you want outcomes-based (or objectives-based, and there are other terms as well) criteria, you are going to open up a can of worms if you want to assess those in a binary fashion. With outcomes-based criteria, I think there is no “objective” way to make an up-or-down decision on whether the device meets a particular criterion.

Frankly, for the device labeling program, I think an informative label is the only way to go. Yes, the majority of consumers will have no idea what all that information means for them. But IMHO, it’s much better to give at least a few consumers some good information that will let them make an informed decision, than it is to give them a thumbs up that in the end rests on subjective judgments, and will be subject to endless challenges, with no real resolution of those challenges.

If we were talking about clear, objective criteria, like whether the device is properly electrically grounded, there’s no question that binary is best; you don’t want to just give someone information and hope they make the right decision, when the device could electrocute them if they make the wrong decision. But cybersecurity ain’t that way, which is why there are very few cyber regulations that even call for up-or-down decisions. NIST would make a big mistake were they to try to implement a binary IoT security label.

P.S. In general, I think the only workable cybersecurity regulations are those that are risk-based – that is, the organization is required to implement a risk management program to cover a certain domain, such as supply chain security (as in NERC CIP-013) or software vulnerability management. The organization is judged on how well they’ve developed and implemented their program.

However, this was clearly not what the EO had in mind when it ordered labeling programs for consumer IoT devices and consumer software, and I agree it’s hard to see Joe and Mary Smith conducting a full cyber risk assessment before buying a baby monitor – although that would definitely be in order before say a military purchase of an IoT device.

But given that the EO did order a consumer IoT labeling program, I don’t see any good alternative to an informative label. Let’s save binary and graded labels for the commercial, industrial and government use cases, where they make some sense.

P.P.S. I have another important disagreement with NIST on the IoT device labeling program, which I will hopefully write about in the very near future.


Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. Nor are they shared by the CISA’s Software Component Transparency Initiative, for which I volunteer as co-leader of the Energy SBOM Proof of Concept. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.


[1] In NIST’s parlance, there’s a difference between an IoT device and an IoT product. The device is one component of a product, but another component is the software running somewhere in the cloud that interacts with the device, as is usually the case. A third component might be a hub through which the device connects to a network. The product consists of all three of these components.

No comments:

Post a Comment