Cloud-Rendezvous-Style OT Remote Access: Residual Risk
Cloud-based remote access systems for OT networks have become popular in recent years, but many vendors make misleading claims about their security. While these systems do offer some advantages, it's crucial to understand how they work and some of the residual risks they pose.
Andrew Ginter
Security product vendors sometimes make outrageous claims – in this article we look at cloud-rendezvous style remote access systems for OT networks and how they work. We debunk the most outrageous claims, we look at residual risk that we accept when deploying these systems, and we suggest circumstances where deploying these kinds of systems actually does make sense.
Learn more about OT Remote Access in our Upcoming Webinar on Oct 30th.
How They Work
A lot of these cloud-based remote access systems work vaguely like the now-ubiquitous Microsoft Teams work:
- Software is installed on a host on the OT network – the host that the remote access session will “take over” as if we were sitting in front of the host – seeing that host’s screen, moving its mouse, and entering keystrokes for the computer remotely.
- Software is also installed on a remote computer – say my laptop. With Teams, this is exactly the same software as is installed on the OT host, but products differ – sometimes the remote client is a different piece of software than is installed on the OT server.
- Now we create a “session” – with Teams it’s a “meeting”, with other remote access products it’s called something else – and you get an identifier for the session – often, but not always, a long, somewhat hard to guess string or number or web link.
- To connect my laptop to that session, I need to get that session identifier to the remote laptop. Session IDs are usually in email, or instant messaging or maybe verbally over the phone.
- And now to connect my laptop to the OT host, I give the software on my laptop the session ID, and maybe a password & some two-factor stuff, and I’m in. A real-time image of the screen of the OT host comes up on my laptop and I can start doing stuff to that host.
In that last step – what happened under the hood? When the session was created, it was created by the software on the OT host, but the session was created in the cloud. When I connect to the session on my laptop, my software reaches out to the same cloud: the remote access product’s cloud, which is the Microsoft Teams cloud in this example. My laptop’s software asks that cloud to connect to the session ID that I provided. The cloud checks permissions, and if everything aligns, the cloud tells the OT host to start sending screen images, and tells my computer to start sending keystrokes and mouse movements (KMM). Every screen image the cloud receives from the connection to the OT host, it now send down the connection to my laptop, and vice-versa with received KMMs. Said another way, each of the computers opened a connection to the cloud service, and the cloud provided a “rendezvous” of those two connections “in the head” of the cloud.
"No Firewall Changes"
One claim made by a lot of vendors of this kind of remote access software is that no firewall changes are needed to enable the software. They contrast their rendezvous approach with traditional remote access, where we need to set up a VPN server or jump host in our OT network that receives connections from the Internet or from the IT network. Most firewalls, both IT and OT, are configured to forbid incoming connections from the Internet by default, accepting only those connections or types of connections for which we’ve configured a firewall rule allowing the connections.
This claim is in fact true when the rendezvous tech is used to access most IT networks. Not all, but the vast majority of IT networks permit most or all outbound connections to the Internet. This is so IT users can access the Internet sites and services of their choice. Most, but not all, IT firewalls control very strictly only incoming connections.
On OT networks, however, this practices is strongly discouraged by pretty much all regulations, standards and guidance. The buzzword is “deny by default.” OT networks should be configured so that by default all connections through the firewall in either direction are denied. If you want to connect into the OT network through the firewall, you need to add a rule for the kind of connection you are enabling. And the same holds true on the outbound side – if you want to connect out of an OT network – for example out to the remote access rendezvous-ing cloud, you must also add a rule. The “no firewall rules need to change” claim is true for most IT networks, but had better not be true for OT networks, not if your IT/OT firewall is following best practice.
"Completely Secure"
Many vendors also claim their rendezvous-style OT remote access system is “completely secure.” Why? Because you do not need to create an “allow inbound connection” rule in the IT/OT firewall to enable this remote access. The assumption is that TCP connections created from the Internet back into OT is the only way to put OT at risk. This is of course nonsense.
When looking at these kinds of remote access systems, whether we create an outbound firewall rule or not, we have to ask about other kinds of attack scenarios:
- How hard is it to guess the session ID – if I connect my laptop to the cloud and start entering random IDs, will I wind up connected to someone else’s OT network “by accident,” where I can work my will upon it? Or is the session ID really easy to guess because it’s my email, or my name, or some other piece of information about me that’s really easy to find?
- How hard is it to steal the session ID? People send these things in email, in instant messaging and so on. If I can shoulder surf these communications, or am copied on the communications, or the sender, receiver or someone else is sloppy about these communications, can I just connect to the session? Or is there a second level of authentication beyond the session ID – can the session be configured to be open only to a specific user / password / two factor dongle?
- And even if there is secondary authentication, how hard is it to steal / spoof that password or other authentication with phishing attacks or by other means?
- And what about vulnerabilities? These rendezvous systems are software systems where there is always the risk of vulnerabilities. What happens when a vulnerability is announced in the remote access system I have installed and the bad guys exploit that vulnerability – in my laptop client, in the rendezvous cloud, or in the OT host, before I have a chance to patch it all?
There are other attack paths that are a bit more complex, stuff like session hijacking, or interfering with encryption systems and the like. Nothing is ever “completely secure” – and too often, most things are really, embarrassingly far from “completely secure.”
Nothing is ever “completely secure” – and too often, most things are really, embarrassingly far from “completely secure.”
"Read Only" Sessions
Another claim I’ve heard some vendors make is that their remote access systems are read only. A couple vendors for example, basically reproduce HMI’s in the cloud and let remote users connect to those copies to visualize real-time data from the OT process. In this kind of system, most users are read only – you can press what buttons you like on the copy of the HMI, but no commands can be sent back to the OT systems. Unless you buy the remote control option. Or an enemy uses stolen credentials to log into the remote access system, puts down a stolen credit card, and enables remote access.
Who Should Use These Systems?
Despite all of my negativity thus far in this article, there is a role for this kind of remote access system in a large number of OT facilities. As defenders, we must not accept “no firewall changes,” “completely secure,” and other nonsense claims. We need to dig into the tech and understand what is the residual risk – what kinds of attack paths are possible to get through the defenses – because there are always attack paths.
Given these attack paths, however easy or difficult they are, the key question that we as defenders need to answer is whether the residual risks of those attacks are acceptable. A key criterion for making this determination is consequence – what is the worst that can happen if the remote access system is compromised, and the OT host is used either to mis-operate the physical process or to attack other hosts in the OT network in order to use them to mis-operate the process?
If the worst that can happen is that things blow up and people die, well, then we need to look really hard at the remote access system and how robust it is in the face of the kinds of attacks I’ve sketched above, and others. If the worst that can happen is that we suffer a few ten thousand dollars damage and our insurer will pay us back for that, this is probably an acceptable loss. Talk to the insurer – if they are OK with us using this kind of remote access, or are OK with the system provided we have specific security measures deployed, then we can probably use the remote access system safely.
If we go forward with the system, though, it’s important that, one, we aren’t fooling ourselves about worst-case consequences. It is easy to miss consequences – for a comprehensive set of worst-case consequences we generally need many kinds of engineers to rendezvous with many kinds of IT / cybersecurity specialists to really understand what’s possible. And two, we need to be sure that we consistently execute on whatever assurances we make to the insurance company who is taking on our risk. Too many people say “yes yes yes” to cyber insurance questionnaires in the hopes of minimizing the cost of the policies. That’s a bad goal. Our goal with insurance should be to assure with a high degree of confidence that if we suffer a loss, the insurer will pay us out. Which means doing everything we’ve assured / promised the insurer we are doing to manage their risk.
And of course, if we decide the consequences and residual risks are unacceptable, then we need to re-evaluate the use of this kind of remote access.
If this was useful to you, and you’d like a similar analysis on a lot of other kinds of remote access systems, please join my webinar Oct 30 – you can register here.
About the author
Andrew Ginter
Share
Trending posts
Cloud-Rendezvous-Style OT Remote Access: Residual Risk
The 2024 Threat Report: Prioritizing Cyber Security Spending
Stay up to date
Subscribe to our blog and receive insights straight to your inbox