The NERC CIP standards define required cybersecurity programs and controls for North American Bulk Electric System (BES) Cyber Systems. The standards are written in an abstract language that is technology-independent and network-independent. Interpreting the language for specific technologies and network designs can be tricky. In this article, we look at one of the tricky bits in the standard: mixed-trust Active Directory servers.
Why is this important? Ransomware criminal groups are increasingly causing production outages and other physical consequences by compromising IT networks in critical infrastructures. One of the ways that compromised IT networks cause production outages is dependencies. If OT or BES systems depend on IT systems, such as Active Directory servers, for correct and continuous operation, and those IT systems are compromised and impaired, then the BES systems are likely to fail.
Active Directory
A common example of OT dependencies on IT systems is joining industrial control systems to IT Active Directory servers and domains. NERC CIP does not forbid such dependencies, but the standard contains language that makes such dependencies very difficult. Finding this language can be problematic – the words “Active Directory” do not appear in the standard.
Worse, it is very tempting to join BES assets into IT-managed security domains. Active Directory or equivalent systems dramatically simplify compliance with certain NERC CIP directives. For example: CIP-004-R5.4 requires that owners and operators revoke access to CIP-critical information within one calendar day of the termination of any employee with such access. An Active Directory server lets enterprise security personnel delete all such permissions for terminated employees, company-wide, with a few mouse clicks.
NERC CIP Compliance – The Problem
Here’s the problem, though. In NERC CIP terminology, Active Directory servers that control passwords and permissions for BES Cyber Systems are Electronic Access Control or Monitoring Systems (EACMS). All EACMS systems are in scope for many requirements in the NERC CIP standard – requirements that IT teams generally find onerous. Consider for example CIP-004-R4.3 – for High Impact BES Cyber Systems and Medium Impact systems and their associated EACMS (ie: Active Directory servers):
R4.3: For electronic access, verify at least once every 15 calendar months that all user accounts, user account groups, or user role categories, and their specific, associated privileges are correct and are those that the Responsible Entity determines are necessary.
So consider the numbers – a large power utility might have 40,000 employees, ranging from power plant operators to sales people and janitors. Most of these employees have accounts on the IT network and in the IT Active Directory servers – so that the company can send them email for example. Most of these employees do not have accounts on or access to OT / BES systems.
004-R4.3 requires that, if even one of the company’s Medium Impact power plants or high voltage substations is managed by the IT Active Directory server, then all accounts on that AD server come into scope for R4.3. All 40,000 accounts must be reviewed every 15 months to ensure that each and every account, account group, role and privilege is correct. Not just the accounts relevant to the substations or power plants – all the company’s accounts. This is an enormous amount of work.
Separate Trusts
As a result, to my knowledge, no power utility in North America with Medium Impact or High Impact BES Cyber Systems manages any computers in those cyber systems with an IT-based Active Directory controller, nor do they set up an OT AD controller that trusts any IT-based controller. Computers in those BES / industrial networks are either not in a domain, or they are in an OT domain that uses Active Directory Servers that are completely distinct and separated from the IT domains and servers.
Now, in theory, NERC CIP allows some advanced functions of Active Directory servers to be shared with OT networks – services such as patch management and anti-virus policies. But if even one account or permission for any important BES Cyber System winds up managed by that IT AD, that dependency brings all the accounts on the IT AD server into scope for all CIP EACMS rules. In practice, nobody risks this.
Other Industries
Why is this important outside of the NA power grid? Well, many industries in many geographies take inspiration from the NERC CIP standards – the CIP standards have a very good reputation in much of the world. When I talk to practitioners in these other industries and geographies about eliminating OT dependencies on IT systems, Active Directory always comes up. The question I’m asked repeatedly is “why?” Why must we pay to set up two AD systems, one for OT and one for IT? Why set up two Anti-Virus systems? Two WSUS systems? Two SolarWinds Orion firewall management domains?
Two of anything costs close to twice what one costs – though there can be minor savings on the second instance because of marginally reduced purchasing and training costs. The answer is that two of all these things are going to cost much less than an unplanned outage due to ransomware taking over your IT network.
Modern attacks often target Active Directory or other important enterprise-wide management systems specifically. An adversary with control of our AD system can create accounts for themselves, accounts that let them log into our OT systems or any other systems they want to log into. Such an adversary can also disable any accounts they wish. If they disable accounts that OT systems rely on, that adversary has just crippled all our industrial operations at one stroke.
Bottom Line
Many industries in many geographies take inspiration from the NERC CIP standards, and so I wish the standards would say some things a little more precisely – in examples if necessary. To be fair, there is a document on the NERC website that talks about OT dependencies on IT Active Directory servers, but the document is not a standard. The document is guidance: Lessons Learned – CIP Version 5 Transition Program – Mixed Trust Authentication Environments.
The bottom line: IT networks are constantly exposed to attacks from the Internet. When the worst-case physical consequences of compromise of OT systems are unacceptable, no OT systems should depend on Active Directory systems or any other systems that are hosted on IT networks – not in the power grid and not in any other industry. The NERC CIP standards do not forbid such dependencies, but they make these kinds of dependencies prohibitively expensive – and they do so for a reason. These kinds of dependencies are dangerous.
NERC CIP provisions in a real sense strongly encourage separate IT and OT Active Directory servers and domains, but they do so in a way that is a little hard to find.
