Please note: This transcript was auto-generated and then edited by a person. In the case of any inconsistencies, please refer to the recording as the source.

Nathaniel Nelson
Welcome everyone to the Industrial Security Podcast. My name is Nate Nelson. I’m here with Andrew Ginter, the Vice President of Industrial Security at Waterfall Security Solutions, who’s going to introduce the subject and guest of our show today. Andrew, how’s going?

Andrew Ginter
I’m very well, thank you, Nate. Our guest today is Christina Kiefer. She is an Attorney at Law and a Senior Associate in the Digital Business Department of reuschlaw. And she’s going to be talking to us about cybersecurity regulation in the European Union. As we all know, NIST 2 is coming and there’s other stuff coming too.

Nathaniel Nelson
Then without further ado, here’s your conversation with Christina.

Andrew Ginter
Hello, Christina, and welcome to the podcast. ah Before we get started, can i ask you to say a few words, introduce yourself and your background, and tell us a bit about the good work that you’re doing at Reuschlaw.

Christina Kiefer
Yes, of course. So first of all, thank you very much for the invitation. I’m very happy to be in your podcast today. So, yeah, to me, my name is Christina Kiefer. I’m an attorney at law working as a senior associate at our digital business unit in the law firm reuschlaw.

Christina Kiefer
We are based in Germany and reuschlaw is one of Europe’s leading commercial law firms specialized in product law. And for more than 20 years, our team of approximately 30 experts has been advising companies in dynamic industries, both nationally but also internationally.

Christina Kiefer
And for me myself, in my daily work, I advise companies and also public institutions on yeah complex issues in the areas of data protection, cybersecurity, but also IT and contract law.

And one focus of my work is on supporting clients in introduction of digital products in the EU market. And also looking at the field of cybersecurity and IT law. Since my studies, I have already focused on IT law and cybersecurity. And yes, I have been involved in the legal development since since then in this area.

Andrew Ginter
Thank you for that. And our topic is, you know, the law in Europe for cybersecurity, its regulation. The big news in Europe is, of course, NIS2. And it’s not a law, it’s a directive to the the nation states to produce laws, to produce regulations. So every country is going to have its own laws. Can I ask you for an update? How’s that going? who’s Who’s got the law? I thought there was a deadline. do the do the Do the nations of Europe have this covered or or is it still coming?

Christina Kiefer
Yes, so it’s the last point, so it’s still coming. Some countries have already transposed NS2 Directive into national law, but also a lot of countries are still in the developing and the transposition yeah period.

And that that’s why we are yeah confusing because NIS2 Directive it’s already or has already been enforced since January 2023. and and also the deadline for the EU member states to impose the NIS2 directive international law was October 2024.

So because of that, because of a lot of member states haven’t transposed the NIS2 directive international law, the EU Commission has launched an infringement proceeding against 23 member states last fall in 2024. And this has led to some movements in some EU member states. So as of now, 10 countries have fully transposed this to international law.

So for example, Belgium, Finland, Greece or Italy. And then another 14 countries have published at least some draft legislation so far. And there you can call ah Bulgaria, Denmark and also Germany. And then there are also two countries, it’s Sweden and Austria, and those two EU member states, they have not published neither a draft or also a final national law. So there we have no public information available on their implementation status yet.

Andrew Ginter
And, you know, someone watching this from the outside with, you know, a command of English and of very limited command of German, is there sort of a standard place that a person like me looking at this from the outside could go to find all this stuff? Or is it on every country’s national website in a different language in a different location? Is is there any central repository of these rules?

Christina Kiefer
No, not yet at least. Maybe there will be some private websites where you can find all the different implementation information. But until until now, when you are a company, either you within the EU or also the EU, when you are providing your services into the EU market, you have to fulfill with the NIS2 directive. And this means you have to fulfill with the national laws in each EU member states.

And this is yeah a big challenge for all international companies because they have to check each national law of each EU member states and they have to check if they fall under the scope of application. And what is also very important that the different national laws have different obligations. So the NIS2 directive has a minimum standard which all national legislators have to fulfill But on top of this, some EU member states have imposed more obligations or ah portal for registration or new reporting obligations.

So you have to check for each EU member state. But here we can also help because we see in our daily work that this is a very, very hard yeah challenge for companies to check all the laws and also understand all the national laws. We offer a NIS2 implementation guide where you can get regularly updates on and an overview of how the different EU member states have transposed NIS2.

And yes, in addition to this, we also have a NIS2 reporting and obligation guide, especially looking at the reporting and registration obligations to see where you have to register in each EU member state, but guide So you can book our full guide, but we also post yeah some overviews on LinkedIn and our newsletter.

Andrew Ginter
So thanks for that. You touched on the yeah the the goal of NIS2 was to increase consistency among the nation states of Europe in terms of their cyber regulations, and in my understanding, to increase the strength of those regulations across the board. How’s that coming? Are the regulations that are coming out stronger than we saw with NIS2? And are they consistent?

Christina Kiefer
Well, it’s… correct that the idea behind NIS2 or the NIS2 directive was to create ah stronger and also more consistent cybersecurity framework across the whole EU and the EU market. And also the NIS2 directive should also cover a broad set of sectors for regulated companies. So there should be some consistency within the EU. but it’s an EU directive and not an EU regulation. So this means the NIS2 directive sets only a minimum standard to all EU member states that they can then transpose into national law. And that’s why EU member states are allowed also to go beyond if they want to. And some of the EU member states have already done this. this So what we’re seeing right now, looking at the national laws which have already been enacted and also looking at the draft of some national laws, we see quite a mixed picture. So we don’t see a whole consistency what a lot of companies were hoping for. We see more like a mixed picture with some countries like Belgium again, for example.

They have pretty much stuck to the core of the directive and haven’t added much on top. So there you are also for you as a company, you can ensure when you’re looking at this two directive or when you have already looked at this two directive, you can be yeah positive that you also fulfill the requirements of the law of Belgium. But on the other hand, looking for example, on Italy, they have expanded the the scope of application. So Italy has, for example, included a cultural sector as an additional regulated area. So the sector of culture hasn’t been mentioned in NIS2 directive at all. But Italy ah had the idea, well, we can regulate also the cultural sector. So that’s why they have also sort in yeah included it into their national law.

And also in France, you can see that they have imposed more obligations and also have broadened the scope of application of their national law. because here they have also widened up the regulated sectors and here they have added educational institutions, for example. We have a minimum set of standards set out in the NIS2 directive, but across the EU, looking at the national laws, we have a lot of national differences. And that’s why it’s very hard for companies to comply with the NIS2 directive or with the national laws within the EU market.

Nathaniel Nelson
One of the more interesting things that Christina mentioned there, Andrew, was Italy treating its cultural sector as like critical infrastructure, which sounds a little bit, it sounds very Italian, frankly.

Andrew Ginter
Well, I don’t know. It’s not just the Italians. The original, you know, this was back in the, I don’t know, the the late noughts. One of the original directives that came out of the American administration was… a list of critical infrastructures. And at the time it included something like national monuments as a critical infrastructure sector. And the justification was, you know, any monument or, you know, cultural institution that was that was seen as essential to national identity, national cohesions,

And then it disappeared in the 2013 update of what were ah critical national infrastructure. So it’s no longer on CISA’s list of critical infrastructures, but it used to be. And, you know, in terms of Italy, oh I don’t, you know, I don’t have a lot of information about Italy, but again, you might imagine that national monuments and certain cultural institutions are vital to sort of national identity. Think the Roman Colosseum. Should that be regarded as critical infrastructure? It’s certainly critical to tourism, that’s for sure. So that’s that’s what little I know about it.

Andrew Ginter
And in my recollection of NIS2, one of the changes was increased incident disclosure rules. Now, i’ve I’ve argued or I’ve speculated. we We did a threat report at Waterfall. We actually saw numbers sort of plateau in terms of incidents. I wonder, I speculate whether increased incident disclosure rules are in fact reducing disclosures because lawyers see that disclosing too much information can result in lawsuits. For instance, SolarWinds was sued for incorrect disclosures. And so they they i’m I’m guessing that that they… they yeah conclude that minimum disclosure is least risk. And if they get partway into an incident and say, this is not material, we don’t need to disclose it we’re not going to disclose it, we actually see fewer disclosures.

Can you talk about what’s happening with the the disclosure rules? are they How consistent are they? Multinational businesses, how many different ways do they have to file? And are we seeing greater disclosure or in your estimation, fewer disclosures because of these rules?

Christina Kiefer
Yeah, that’s a really good question and honestly it’s something we get also asked all the time right now because once we hear again all over if we operate in several and several EU countries do I need to report a security incident in one you member states or via one portal and then I’m fine or do I really have to report a security incident to each EU member states which is kind of affected with the with regard to the security incident.

And yeah, unfortunately, the answer right now is yes, you have to report your security incident to each EU member state or to each national authority of the EU member state, which you fall under the scope of the national law. Because the NIS2 directive does not really require one portal or one obligation registration and also a reporting portal for all EU member states. So it’s up to the national authorities and also up to the EU member states to regulate this field law. And you can see that many national authorities have already recognized this issue and they are also looking at ways to simplify the process of registration but also of reporting security incidents and there you can see some member states try to yeah at least include or to to set up a portal a national-wide portal where you can yeah report your security incident.

Some other national authorities go even further. They say they implement a yeah scheme or structure where you only have to report to them and then they will yeah transfer the report to the other relevant EU authorities. But again, this is each and in e in each EU member state national law, so then you also have to check again all the other national laws within the EU. Yes, but also the authorities of the EU member states have already, well, at least indicated that they are talking to each other. So maybe in the future we will get one portal to report everything. But as I said before, it’s not regulated in the NIS2 directive and is also not foreseen for now.

Yes, and to the other part of your question. You could think that when you’re obliged to report everything and each security incident that the reporting would decrease But you also have to look at a yeah at the at the risk of non-compliance and the risks are very high because the NIS2 directive is imposing high sanctions and also a lot of yeah authority measures, authority market measures. And that’s why in the daily consulting work, it’s better to say, please report an incident because also the national authorities communicate this to the companies. They say, please report something because then we can work together. So the focus of the national authorities, at least in Germany, we see right now is they want to cooperate together.

They want to ensure a cyber secure en environment and a cyber secure market. So the focus is to report something that they can yeah work on together and that’s why it would be better to report and I would say maybe we get also an increase of reporting.

Andrew Ginter
So I’m a little confused by your answer. the The rules that I’m a little bit familiar with are the American ah Securities and Exchange Commission rules. And those rules mandate that any material incident must be reported to the public, any incident that might cause a reasonable investor to either buy or sell or assign a value to shares in in a company.

Which means non-material incidents can be kept quiet. And the SEC disclosures are public. Everyone can see them because reasonable people need information to buy and sell shares. The NIS2 system, is it requiring all incidents to be reported? And are those reports public?

Christina Kiefer
That’s a good point. To your first part of your question, the NIS2 directive and also the reporting obligation is kind of the same as the regulation you mentioned before, because you have to report only severe security incidents. As a regulated company, you are obliged to check if there is a security incident in the first step and then the second step you have to check if there a severe security incident.

And only this security incident you are obliged to report to the national authorities. So that’s kind of the same structure or mechanism. And to the second part of your question, the report will not be published for everyone. So first of all, if you report it to national authorities, only the national authorities have the information. It can happen because we have in some Member States some laws where yeah people from the public can access or can get access to information, to public information. It can happen that some information will be publicly available. But the the first step is that you will only report it to the national authority and that the report will not be available for the public as such.

But next to the reporting obligation to the national authorities, you also have information obligations in the NIS2 directive. So it can happen that you are also obliged to inform the consumers of your services.

Andrew Ginter
So thanks for that. The other big news that I’m aware of in Europe is the CRA, which confuses me because I thought NIS2 was the big deal, yet there’s this other thing that sort of came at me out of the blue a year ago, and I’m going, what’s what’s going on? Can you introduce for us what is the CRA, and how’s it different from NIS2?

21:30.66
Christina Kiefer
Yeah, sure. So, as you mentioned before, the CRA is like the sister or brother and the second major piece. of the new European cybersecurity framework alongside the NIS2 Directive.

Christina Kiefer
It’s the Cyber Resilience Act, or for short CRA. And while the NIS2 Directive focuses on the cybersecurity requirements for businesses or entities in critical sectors, the CRA takes a different angle and the CRA introduces EU-wide cybersecurity rules for products.

So NIS2 is focusing on cybersecurity of entities and the CRA is focusing on cybersecurity for products with digital elements. And also the other difference is also that NIS2 directive, we have an EU directive, so it needs to be transposed into national law by each EU member state and the Cyber Resilience Act is an EU regulation So when the Resilience Act comes into force, it will apply directly in each EU member state.

Andrew Ginter
Okay, so that’s how the CRA fits into NIS2. What is the CRA? What are what are these rules? is it Can you give us a high-level summary?

Christina Kiefer
Yeah, sure. So the CRA is the EU-wide first horizontal regulation, which imposes cybersecurity rules for products with digital elements. So regulated are products with digital elements and this definition is very broad. It covers software and also hardware and also software and hardware components if they are yeah brought to the EU market separately. And products with digital elements are kind of like connected devices and as I said, software and hardware that can potentially pose a security risk. Also, what is very important, the CRA imposes obligations not only to manufacturers, but also to importers, distributors, and also to those companies which are not resident in the EU, because the main point for the geographical scope of application is that you place a product in the EU market, whether you are placed in the EU or not.

Christina Kiefer
So this means also that the Cyber Resilience Act, such as data and such as the General Data Protection Regulation, has a global impact impact for anyone selling tech products in Europe.

Andrew Ginter
So let me jump in real quick here, and Nate. What Christina‘s described here, oh you the CRA, the scope applies to all digital products sold in Europe. To me, this the CRA is, in my estimation, and she’s going to explain more in ah in a few minutes, it’s probably the strictest cybersecurity regulation for products generally in the whole world. it It sounds to me like this might become just like GDPR. This was ah a European regulation that came through a few years ago. It had to do with marketing and the use of private information, in particular my email and sending it. Basically, so it was like an anti-spam act. It’s the strictest in the world. And everybody who has any kind of worldwide customer base, which is almost everybody in the digital world that that’s sending out marketing emails, is now following the GDPR pretty much worldwide because it’s just too hard to apply one law in one country and one law in the other. So what you do is you pick the strictest that you have to comply with worldwide, which is the gp GDPR, and you do that. worldwide instead of trying to figure out what’s what. It sounds to me like the CRA could very well turn into that kind of thing. It might be the thing that all manufacturers that embed a CPU in their product have to follow worldwide because it’s just too hard to to change what they do in one country versus another.

Andrew Ginter
Okay, so can you dig a little deeper? I mean, an automobile, you buy a a ah new automobile from the from the dealership. My understanding is that it has 250, 300, maybe 325 CPUs in it, all of them running software. It would seem to me that ah a new automobile is covered by the CRA. what What are the obligations of the manufacturer? What should customers like me expect in automobiles that that might be different because of the CRA?

Christina Kiefer
Thank you. First of all, looking at your example, automobiles are not covered by the CRA, because the CRA some exemptions. And the CRA says, we are not regulating digital products with the digital elements, which are already regular regulated by specific product safety laws. And here, looking at the automotive sector, we have for sure in the EU very strong and very specialized regulation for product safety of cars and so on. So just for your example, but looking at other products with the chill elements, for example, wearables or headphones, smartphones, for example, you can say that there are kind of five core obligations for manufacturers in the CRA. So the first obligation is compliance with Annex 1, which means you have to fulfill a list of cybersecurity requirements. And you don’t only have to fulfill those cybersecurity requirements, but you also have declare and show compliance with Annex 1 of the CRA. So it’s a conformity assessment you have to undergo.

Christina Kiefer
The other application, number two, is cyber risk assessment. If you are a manufacturer of a product with digital digital elements, you are obliged to assess cyber risks and not only during the development and the construction of your product and also not only during the placing of your product to the EU market, but throughout the whole product life circle. So if you have a product and you have it already placed on the market, you are obliged to undergo cyber risk assessments. Then looking at the third obligation, it’s free security updates.

Christina Kiefer
So manufacturers have to provide free security updates throughout the expected product life cycle. We have also mandatory incident reporting. So we have here also reporting and registration obligations, such as we already talked about looking at the NISS2 directive. And also like in each product safety law in the EU, we also have the obligation for technical documentation. So this is of those are the five core obligations, compliance, cyber risk assessment, free security update, reporting and documentation.

Andrew Ginter
And you mentioned distributors. What are distributors and importers obliged to do?

Christina Kiefer
yeah there We have some graduated obligations. So they they are not such strict obligations such for manufacturers, but importers and distributors are obliged to assess if the product, what they are importing and distributing to the EU market are compliant with the whole set of cybersecurity requirements of the CRA. So they have to check if the manufacturer and the product is compliant and if not, They have to inform and yeah cooperate with the manufacturer to ensure cybersecurity compliance. But also importers are also obliged to yeah impose their own measures to to fu fulfill with the CRA.

Andrew Ginter
Okay, and you said there were five obligations. You spun through them quickly. Some of them make sense on their own. Do a risk assessment, do it from time to time, see if the risks have changed. That kind of makes sense. The first one, though, comply with Annex 1. That’s like an appendix to the CRA. What’s in there? what What are the obligations?

Christina Kiefer
Yes, sure. Annex 1 is, yeah the you can also say, Appendix 1 to the CRA. and And there are you can see there is a list of certain cybersecurity requirements which manufacturers have to fulfill. And the list is divided into two different main areas. And one area is cybersecurity requirements. So it focuses on no known vulnerabilit vulnerabilities at the time of the market placement, secure default configurations, protection against unauthorized access, ensuring confidentiality, integrity and availability, and also secure deletion and export of user data. So kind of all of cyber security requirements such as them which I have mentioned. And the other area is vulnerability management. So manufacturers have to ensure that they have a structured vulnerability management process and they have to yeah install a software bill of materials.

They have to provide free security updates. They have to undergo cybersecurity testing and assessments. there needs to be a process to publish information on resolved vulnerabilities. And again, here we also need a clear reporting channel for known vulnerabilities.

Andrew Ginter
So it sounds like you said that a manufacturer is not allowed to ship a product with known vulnerabilities. Practically speaking, how does that work? I mean, a lot of manufacturers in the industrial space use Linux under the hood. Linux is a million lines of code of kernel. And, you know, the, these devices don’t necessarily do a full desktop style Linux, but they still have a lot of code that they’re pulling from an open source distribution. And in these millions of lines of code, From time to time, people discover vulnerabilities and they get announced. And so it’s it’s almost a random process. Do I have to suspend shipments the day that a vulnerability a Linux vulnerability comes to light until I can get the thing patched and then three days later ah start shipments again? Practically speaking, how does this zero known vulnerabilities requirement work?

Christina Kiefer
Basically, it is like, as you said, because the Cyber Resilience Act focuses on known ah no known vulnerabilities not only in your product but also in the whole supply chain. So the Cyber Resilience Act focuses not only on products with digital elements but also focusing on the cybersecurity of the whole supply chain. So this means looking at Annex 1 and the cybersecurity requirements Products with digital elements may only be placed on the EU market if they don’t contain any known exploitable vulnerabilities. So it’s not any vulnerability, but it’s any known exploitable vulnerability. That is a clear requirement under Annex 1. And also when you’re looking at making a product available on a market, that doesn’t just mean selling it.

Christina Kiefer
It includes any kind of commercial activity. And also what is also a very good question also in our daily work, looking at making a product available on the market. A lot of companies say, well, I have a ah batch of products. So, and if I have placed this batch of products on the EU market, I have already placed product on the market. So I can also place the other products of this batch also in the future. But it is not correct, because looking at EU product safety law, the regulation is focusing on each product. So looking at these requirements, you can say, first of all you really have to check your own product, your own components, but also the products and the components you are using from the supply chain. And you have to check if there are any known exp exploitable vulnerabilities. So you have to yeah impose a process to check the known vulnerabilities and also to ah impose mechanisms to fix those vulnerabilities.

Christina Kiefer
And if you have products already on the market, you don’t have to recall them because first of all, it’s okay if you have a vulnerability management which is working and where you can fix those vulnerabilities. And when you have products already in the shipment process, there it’s up to each company to assess if they have to yeah recall products in the and the shipment process or if they say, okay, we leave it in the shipment process because we know we can fix the vulnerability within two or three days. So in the end, it’s kind of a risk-based approach and each company has to assess what measurements are yeah applicable and also necessary.

Andrew Ginter
So that that makes a little more sense. I mean, the Linux kernel and sort of core functions in my, but I don’t have the numbers, but I’m guessing that you’re going to see a vulnerability every week or two in that large set of software. And if that’s part of a router that you’re shipping or part of a firewall that you’re shipping or part of any kind of product that you’re shipping, Does it make sense that, you know, you discover the exploitable vulnerability on Thursday and you have to suspend shipment until, ah you know, three weeks out when you have incorporated the vulnerability in your build and you’ve repeated all of your product testing, which can be extensive.

Andrew Ginter
And by the time you’re ready to ship that fix, two other problems have been developed and now you have to, you can’t ship until, you know, it, It sounds like it’s not quite that strict. it’s not that That scenario sounds like nonsense to me. It just it would never work. You’re saying that there is some flexibility to do reasonable things to keep bringing product to market as long as you’re managing the vulnerabilities over time. Is is that fair?

Christina Kiefer
Yes, yes, that’s right. Because in the CRA we have a risk-based approach and also you have to… No, the basis for each measure you have to to impose under the CRA is your cyber risk assessment. So you have to check what kind of product am I using or am i manufacturing? Which kind of product am I right now placing on the EU market? What are the cybersecurity risks right now? And also what what are the specific cybersecurity risks of this known vulnerability?

Christina Kiefer
And then you have to check, have i do I have a process? Do I have a process imposing appropriate measures to to fix those vulnerabilities? And if I have appropriate measures, to fix the vulnerabilities in a timely manner, then it’s not the know you are not obliged to recall the product itself. But at the end, looking at a risk-based approach, it’s up to the decision of each company.

Andrew Ginter
So this is a lot of a lot of change in in for a lot of product vendors. Can I ask you, how’s it going? Is it working? Are are the vendors confused? can you Do you have any sort of insight in into how it’s going?

Christina Kiefer
Yeah, sure. So what we’re seeing right now, a lot of companies, both manufacturers, but also suppliers, are getting ahead of the curve when it comes to the Cyber Resilience Act, because they see that there is a change and there there will be new strict obligations, not only on manufacturers, but also in the whole supply chain. So suppliers, distributors, importers are also coming to us and asking if they are under the scope of the CRA. So this is the first point. If you’re a distributor or an importer, you already have to check if you and your company itself falls under the scope of the CIA. And if it is like this, then you are already obliged to ensure all the obligations of the CRA. But it can also happen that suppliers are under the scope of the CRA in an indirect manner.

Because ensuring all those new cybersecurity requirements from a manufacturer point of view, you have to ensure it within the whole supply chain. And the main instrument to ensure this was already in a future in a and the past and will also be in the future is contract management. So you have to impose or transpose all those new obligations to the suppliers via contract management. And there we see different reactions, but there’s definitely a growing awareness that cybersecurity needs to be addressed contractually, especially in relation to the CRA obligations. And yeah looking at contract negotiations, of course, we have some negotiations with the suppliers And one of the main points which is negotiated is the regulation of enforcement.

Christina Kiefer
Because when you have contractual management looking at cybersecurity requirements, you can not only yeah transpose those obligations to the suppliers, but you also have rules on enforcing those new contractual obligations. For example, contractual penalties. And there we see that contractual penalties often sparks some debate during negotiations. But to sum up, in practice, we’ve always been able to find a balanced solution that works for all parties involved.

Nathaniel Nelson
I suppose I could think about any number of potentially trivial electronics products, Andrew, but let’s say that I or my neighbor has ah a smart fridge, a fridge with a computer it. We generally assume that those devices don’t even really have security in mind at all. And a security update is like so far from the universe of how anyone would interact. with such a device and now we’re saying that that kind of thing is going to be regulated in these ways.

Andrew Ginter
I think the short answer is yes. You might ask, what good does this regulation do for a fridge? And, you know, I think about this sometimes. I think the answer is it depends. If, you know, a lot of the larger home appliances nowadays have touchscreens. There’s a CPU inside. There’s software inside. These are cyber devices. You might ask, well, when was the last time I updated the firmware in my fridge? How many times am I going to update the firmware in my fridge? Those are good questions. Most people never think about something like that. But the law might… you know, very reasonably apply to the fridge if the fridge is connected to the Internet so that I can see, for example, how much power my fridge is using on my cell phone app.

Isn’t that clever? But now I’ve connected the fridge to the Internet. We all know what what happened to, what was it, the Mirai botnet took over hundreds of thousands of Internet of Things devices and and used them as attack tools for denial of service attacks. If you’ve got an internet connected fridge, you risk that if you haven’t updated the software. Worse, if someone gets into your fridge, takes over the CPU, you could change the set point on the temperature and cause all your food to spoil. This is a safety risk.

Andrew Ginter
Again, how many consumers are going to update the software in their fridge? Realistically, I don’t think… You the majority of consumers will, even if there is a safety threat. To me, you know, the risk, this this is part of the risk assessment. If there’s a safety threat because of these vulnerabilities, you might well need to… I don’t know, auto-update the firmware. That might be part of your risk assessment so that the consumer doesn’t have to do it. Or better yet, design the fridge so that safety threats because of a compromised CPU are impossible, physically impossible. Make the the temperature setting manual or something. But this is this is a bigger problem than I think one regulation, the the the question of safety critical devices connected to the cloud.

Nathaniel Nelson
Yeah, admittedly, the the notion of a smart refrigerator safety threat isn’t totally resonating with me. And then we haven’t even discussed the matter of like, OK, let’s say that my refrigerator gets automatic updates or I just have to click a button in an app when it notifies me to do so to update my firmware. At some point, you know, fridges sit in houses for long periods of time. I can’t recall the last time that my fridge has been replaced. In that time, any manufacturer could go out of business. And then how do you get those updates, right?

Andrew Ginter
Exactly. So, you know, to me, but this is outside the scope of the CRA, but, you know, to answer your question, to me, the solution you know, two or threefold, we we need to design safety-critical consumer appliances in such a way that the unsafe conditions cannot be brought about by a cyber attack. I mean, we talk about, you know, fixing known vulnerabilities. That’s only one kind of vulnerability. What about zero days? There is, there’s there’s logically no way that someone can, you solve all zero days. It it It’s a nonsensical proposition. So there’s always going to be zero days. What if one is exploited and, you know, a million fridges set to a ah set point that that’s unsafe?

Andrew Ginter
To me, we’ve got to design the fridges differently, but that’s that’s sort of a different conversation. In fact, that’s the topic of my next book, but which is why I care so much about it. but but it’s These are important questions, and I think the CRA is a ah step in the direction of answering them, but I don’t know that it has all the answers.

Andrew Ginter
So work with me. you know, what, what you described there makes sense for, you know, manufacturers like, uh, IBM who can, you know, produce high volumes of, or, you know, Sony or the, the big fish. But, you know, if I’m a small manufacturer, I produce a thousand devices a year. I buy components for these devices. I buy software for these devices from big names like Sony and Microsoft and Oracle. And, you know, I go to Oracle and say, you must meet my contract requirements or I won’t buy my thousand products from you at a cost of $89 a product. Oracle is going to say, take a flying leap. We’re not signing your contract. Is this realistic?

Christina Kiefer
Yes, and we see this also in practice because we are not only consulting the big manufacturers but are also the smaller companies in the supply chain. And there you can have different approaches because when you are buying products from the big companies, First of all, you have to know that they are or they might be obliged also under the CRA. So they are fulfilling all those new cybersecurity requirements. And you also have to take it though there you also have to check their contracts because there you can see already they have a lot of new regulations looking at cybersecurity, either if it’s implemented into the the general contractual documents or implemented into one cybersecurity appendix.

So you see all the companies are looking at the Cyber Resilience Act and then they are taking measures and also looking at their contract management. So if you are lucky enough, you can see, okay, they have a contract which is already regulating all the obligations under the CIA. And then if it’s not like this, We take the approach that we establish a cybersecurity appendix. So when you’re already a contractual relationship with the big players, you don’t have to negotiate the whole contract from the beginning. You can only show them your appendix and then on on basis of this appendix, you can discuss the cybersecurity requirements. So this is kind of a approach which has helped also smaller companies in the market.

Andrew Ginter
So you gave the example of of headphones and smartphones. For the record, does this apply to industrial products as well? I mean, our our listeners care about programmable logic controllers and steam turbines that have embedded computer components, or is it strictly a ah consumer goods rule? Now, and this is a very important point to highlight, the Cyber Resilience Act explicitly applies not only to consumer products but also to products in the B2B sector. so this means that all software and all hardware products along with any related remote data processing solutions fall under the scope of the CRA, either in B2C or also in B2B relationships.

Andrew Ginter
Well, Christina, thank you so much for joining us. Before we let you go, can I ask you, can you sum up for our listeners? What are the the key messages to take away to understand about what’s happening with cyber regulations, both NISU and CRA in Europe, and and what we should be doing about them as both consumers and manufacturers?

Christina Kiefer
Yeah, sure, of course. So let me give you a quick recap. So first of all, you see the EU legislature is tightening the cybersecurity requirements significantly with both the NIS2 directive and also the Cyber Resilience Act. And the new requirements affect any company that offers products or services to the EU market, no matter where they are based. So it is it has a very broad scope of application. Looking at the NIS2 directive, it’s very important to know that the NIS2 directive is already enforced, but it has to be transposed into national law, which has not been fulfilled by all EU member states, and that the national implementation across the EU is still quite varied.

Looking at the Cyber Resilience Act, the CRA brings new security obligations to products with digital elements, so for all software, for all hardware products. And it also is focusing not only on cybersecurity on products, but also in the whole supply chain. So both frameworks require companies to take proactive steps right now, looking at risk assessment, risk management, reporting, and also contract management, particularly when it comes to managing their supply chain. So looking at the short implementation deadlines ahead, both from the NIS2 Directive and also the CIA, it’s very important for companies to act now. And the first step we consult to do is to identify the relevant laws, because we have a lot of new regulations looking at digital products and digital services. So, yeah first of all, check the relevant laws and the relevant obligations which are applicable to your business.

And here we offer a free NIS2 quick check and also a free CRA quick check where you can just click through the different questions to see if you are under the scope of NIS2 and CRA. And then after all, when you clarified that you are affected on the one or both of the new regulations, the company needs to review and adopt their cybersecurity processes, both technically and also organizationally. So it’s very crucial to continuously monitor and ensure compliance with the ongoing legal requirements, especially also looking at contract management and focusing on the supply chain. And yeah, there we can help national but also international companies with kind of a 360 degree approach to cybersecurity compliance because we enter ensure solutions with the range from product development and marketing to reporting and market measures. So, yeah, we we give companies ah practical and also actionable guidance in ah in an every step way.

So looking at the first step to to act and yeah to identify the relevant laws and obligations to your business, companies can yeah visit our free NIS2 QuickCheck and our free CRA QuickCheck, which is available under nist2-check.com and also And yeah, if you have any further question, you are free and invited to write to me via email via LinkedIn. Yeah, I’m happy to connect. And thank you very much for the invitation.

Nathaniel Nelson
Andrew, that just about concludes your interview with Christina Kiefer. And maybe for a last word today, we could just talk about what all of these rules mean practically for businesses out there because, you know, it’s one thing to mention this rule and that rule in a podcast, but sounds like kind of stuff we’re talking about here is going to mean a lot of work for a lot of people in the future.

Andrew Ginter
I agree completely. It sounds like a lot of new work and a lot of new risk, both for the critical infrastructure entities that are covered by NIST or by the local laws, especially for for businesses, the larger businesses that are active in multiple jurisdictions, and certainly for any manufacturer who wants to sell anything remotely CPU-like into the the the European market. It sounds like a lot of work, but I have some hope that it’s also, because it’s such a lot of work, it’s also a business opportunity. And we’re going to see entrepreneurs and service providers and even technology providers out there providing services and tools that will automate more and more of this stuff so that not every manufacturer and every critical infrastructure provider can. in the European Union or in the world selling to the European Union. Not every one of them has to invent all of this the the answers to these these new rules by themselves.

Nathaniel Nelson
Well, thank you to Christina for elucidating all of this for us. And Andrew, as always, thank you for speaking with me.

Andrew Ginter
It’s always a pleasure. Thank you, Nate.

Nathaniel Nelson
This has been the Industrial Security Podcast from Waterfall. Thanks to everyone out there listening.

Please note: This transcript was auto-generated and then edited by a person. In the case of any inconsistencies, please refer to the recording as the source.

Nathaniel Nelson
Welcome listeners to the Industrial Security Podcast. My name is Nate Nelson. I’m here as usual with Andrew Ginter, the Vice President of Industrial Security at Waterfall Security Solutions.

He is going to introduce for all of us the subject and guest of our show today. So Andrew, how are you?

Andrew Ginter
I’m well, thank you, Nate. Our guest today is Tom Sego. He is the CEO and co-founder of BlastWave. And he’s going to be talking about distributed asset protection, which is a fancy name for a very common problem in the industrial space. We have – Stuff – devices, computers, assets, cyber assets all over the place, might be distant in pumping and substations might be local. The stuff was bought, on the cheap. It was the the lowest bidder.

It’s old. It’s ancient. And we have no budget to rip in place. So what do we do about cybersecurity? And this is something he’ll he’ll be walking us through.

Nathaniel Nelson
Then let’s get right into it.

Andrew Ginter
Hello, Tom, and thank you for joining us. Before we get started, can I ask you to say a few words of introduction? Tell us a bit about your background and about the good work that you’re doing at BlastWave.

Tom Sego
Sure, Andrew. Thanks for having me. So my background is I started my career as a chemical engineer at Caterpillar. I also spent eight years at Eli Lilly designing and building processing facilities to make medicine.

I was also a certified safety professional during that period and managed a 24-7 liquid incineration operation, which burned a 30,000 gallons of liquid waste per day.

So a shit ton. And then I went to Emerson, got did business development, corporate strategy there. Then I did product management at AltaVista. Then I went on to do sales support at Apple, where I was at Apple for almost 10 years.

And then that’s when I started my entrepreneurial career. I started a mobile telephony company, started a solar storage company, started a wine importing business, then played professional poker for a few years, and then eventually started this cybersecurity business called BlastWave.

I co-founded that in 2017. And our mission then is the same as it is today, which is to protect critical infrastructure from cyber threats.

And we wanted to kind of come at this with a very different approach than other cybersecurity companies in that We kind of started from first principles thinking about what are the three highest kind of classes of threat and categories of threats, and can we actually eliminate those?

The biggest category is probably no surprise to anybody here, but it’s phishing, credential theft, et cetera. I’m like, well, let’s just get rid of usernames and passwords altogether. And come up with a different model for for MFA that can actually apply to industrial settings.

So we did that. The second category of threats was really CVEs and and vulnerabilities. And could we make those unexploitable? And we came up with a concept called network cloaking, which I’m sure we’ll discuss, which kind of addresses that issue. And then the last one is human error, which is impossible to get rid of.

But if you can make human beings make fewer decisions, they can also make fewer mistakes. So we also incorporated that into a lot of our UI and UX.

Andrew Ginter
That’s, wow, that’s that’s a history like none other I’ve ever heard, Tom. Makes like I’m thinking it makes my own, what I thought, storied background look completely mundane.

You’ve been in lots of different industries. Now, I understand that a lot of what BlastWave does right now is upstream and midstream. And we’ve never had someone on the show explaining how that works. I mean, I think we’ve had one person on talking about an offshore platform at some point.

But when you’re looking at the industry, can we start with the industry? what’s What’s the physical process? Physically, what’s this stuff look like? What’s it do? How does it work?

Tom Sego
Yeah, it’s really interesting because I can talk about the physical process and it’s also evolved quite a bit in the last 20 years. So first of all, just stepping back, looking at the industry, the overall oil and gas market globally generates $2 trillion dollars of revenue per year, and it generates $1 trillion in profit.

So there’s a lot of money in this business. And that also means that there’s a lot of gallons of oil and lot of cubic feet of gas that are being extracted and transmitted and sent everywhere around the world.

And the other thing that’s interesting is that in spite of how old this industry is, there’s between 15 and 20 thousand new oil wells created per year and in fact, half of those were done in the Permian Basin. So about 8,000 wells were created last year in the Permian Basin.

Tom Sego
I don’t think people realize the magnitude of which the oil and gas companies are continuing to create wells and extract oil. The other thing that’s interesting about it is 20 years ago, we had a traditional vertical drilling approach to oil and gas.

And in that to last two decades, we’ve noticed that there are capabilities to actually now drill horizontally. And what’s pretty interesting is you can actually, as you start drilling a well today, you create the initial bore, which is, usually a foot or more in diameter.

And then you can send these kind of devices and drill bits down a relatively sloping curve that over the course of maybe 100 or 200 meters, you’ve now done 90 degree angle.

And then you can start drilling horizontally, which allows you to have higher probabilities of not hitting a dry well. It gives you more capabilities for lower cost extraction.

And so it’s been a great boon for the industry. Hydraulic fracturing, which is another technique that’s been exploited to to get much higher yields out of these wells, also contributed to the the recent boom in oil and gas.

So There are many, many things that have to be considered when you start doing this process. You’ve got to go through site selection, permitting. You’ve got to do all this site prep. And one thing people may not realize is site prep means building roads.

You have to build an entire infrastructure to get to and from these wells. And then once you start building. Actually drilling the well, it’s much like a CNC machine if you’ve been in a factory like Caterpillar or something where there’s a fluid, heat transfer fluid that allows you to cut the metal.

In this case, they use a mud that both stabilizes the wellbore and it also helps you manage pressure. And that that mud flows down through the the drill pipe and then it comes out around in kind of an annulus, almost like a donut that comes back up the outside of that drill pipe to be then cleaned, having the the rock kind of cuttings removed from it using a screening and operation.

And then you kind of reuse the mud and so forth. So there’s a lot to it. And And increasingly, much of this is being automated.

And you’re having connectivity that is absolutely essential to be your eyes and ears in these wells. Because once you start producing oil and gas, these things are hours and hours away from each other.

They’re very remote, very rural areas. And so that connectivity is absolutely critical. And you may have, we have one customer who has 700 sites that they’re trying to manage.

And so they have to have the ability to do this in an automated fashion, which requires not just connectivity, but secure connectivity.

Andrew Ginter
Cool. I mean, it’s a piece of the of the the industry I’d never dug into. So thank you for that. Can I ask you, you’ve said in the modern world,

you know it increasingly everything is automated. I mean, that makes perfect sense. The The example I often use is you buy an automobile, it’s got 300 CPUs in it. It Everything, every every device, but every non-trivial device you you you buy nowadays has a CPU in it.

Can you talk about the automation in these these drilling systems, in these these upstream systems? what does, what’s that automation look like? Is it like built into the device like an automobile? Is it a programmable logic controller? I mean, I’m familiar with, power plants vaguely. I mean, bluntly, I don’t get out much. I’m i’m a software guy more than a hardware guy, but but I’ve had a few tours. I know what a PLC looks like. If if i If I visited one of these well sites, would I recognize the automation? What’s it look like?

Tom Sego
Yeah, you would definitely recognize the automation. So what you see is your classic kind of SCADA tech stack, if you will. So you’ll have remote terminal units. You’re going to have PLCs.

You’re going to have these things mounted on a DIN rail in a cabinet. And there can be various size cabinets at some well locations.

You’re going to have just a few number of devices. And then at some other well sites, again, I go back to the horizontal drilling, you’re going to have a much bigger operation there. You’re also going to have those well sites connected to what are called tank batteries.

so that you can essentially manage the flow of oil and gas into these storage facilities. So there’s there’s a lot of automation that’s necessary using kind of PID control loops to maintain equilibrium within these systems.

And there can also be Oftentimes, challenges that happen, shocks to the system, where let’s say in the case of oil and gas, the price starts dropping.

But when the price starts dropping, the motivation of the business unit is not to just keep cranking production at maximum capacity. And so you actually want to have dynamically, you want to manage your your operation dynamically based on economic conditions that can change over time.

And I’ll tell you something else, Andrew, about what’s happening today. There’s a lot more uncertainty in the business world today than there was four months ago. And I think that is going to affect oil and gas.

It’s going to affect the price of oil and gas. It’s going to affect the supply of oil oil and gas. It’s going to affect the transmission across borders. So these kinds of things can affect the the automation.

I’ll call it like Uber automation. Okay. Not just between the actual plant operations and facilities, but also between different entities in the upstream, downstream and midstream ecosystem.

So there’s a lot of very interesting factors that affect that. And I’ll tell you one other thing that’s kind of interesting. That’s how everybody’s talking about ai and there are some of the larger oil and gas companies that are trying to figure out how to apply AI to optimize their operation.

And everybody knows that there’s there’s automation that’s used to help identify ways to to to deliver predictive maintenance to rotating machines.

But there’s also uses of AI in oil and gas to to prevent things like spills. And one of the big challenges is it’s easy. If you go talk to someone at BP or Shell or Chevron and you say, can I get data to the cloud? They’re going to go, well, heck yeah.

There’s all kinds of great things that can allow you to get data out of your process. And in fact, I think you’re associated with a company that does a really good job of doing that kind of one-way transmission of data.

And the other thing is, but once you have that data, and you’re using it to build AI models, then how do you get, deliver those set points and control variables back to the process?

It scares the crap out of these people. The idea of connecting their control network to a much less secure cloud network or corporate network.

Because as we all know, security is a continuum. It’s not Boolean secure insecure. So I think there’s a lot of interesting things that are happening with that. And I think just to to kind of close the story on that, one company, for example, is pulling that data, they’re analyzing it actually in AWS, and then they are taking some of those control variables and they’re using a human in the loop process so that they’ll say, this is the recommended set point for this this process.

And then the human in the loop then implements that through their control HMI. So there’s a lot of very interesting traditional ways in which automation is applied to oil and gas.

But there’s also some very interesting evolving mechanisms that involve machine learning.

Andrew Ginter
So, Nate, let me jump in and and give sort of a bit of context here. Yeah, AI and cloud-based systems, in my opinion, these are the future of industrial automation in pretty much… Everything.

The question is not if, the question is when, because different kinds of cloud systems are going to be used in different kinds of industries at different times, with different intensities. So, I care enormously about this topic because I am writing my fourth book. The the working subtitle of the book, possibly the title of the book is CIE for a Safety Critical Cloud.

You know, when you have cloud systems controlling, you potentially dangerous physical processes. How do you do that? There are designs that work. I… I’m keen to to to listen to the rest of the episode here. I’m keen to, but when I had Tom on, I was keen to learn from him. When I write these books, I try not to make up solutions myself.

I tend to get them wrong when I do that. I try to learn from experts like Tom and, gather up the best knowledge in the industry and try and trying package it up in a digestible format.

So, yeah, that the cloud is the future and I’m, yeah when When we recorded this, I was keen to to learn from Tom about what the future looks like.

Nathaniel Nelson
And I know we’re about to get right back into the interview. And what I’m about to say actually kind of has nothing to do with what you just said. But before we go, a few times now, it feels like you guys have mentioned the terms upstream, downstream, midstream. And I just want to make sure I’m clear on this before we continue.

Andrew Ginter
Sure. This is This is standard oil and gas terminology. People say, oh, oil and gas, as if it were one industry. It’s not. Really, there’s three industries involved, and each of these these sort of sub-industries have a lot of different kinds of facilities. So the stream is generally considered to be the pipeline.

So we’re talking upstream is producing stuff to feed into midstream, the pipeline. And downstream is taking stuff out of the pipeline to for for refining and such. So, sort of next level of detail, what’s involved in upstream? Exploration is considered part of upstream.

Initial drilling is part of upstream. Offshore platforms are part of upstream. The, onshore pump jacks are part of upstream.

The whole infrastructure, building roads is part of the upstream process. Midstream is pipelines and tank farms. And, in in the natural gas space, you need to do sort of an initial separation and, discard waste from the the product. You might even need this in liquids to take if you can do an initial filter and take water out of the oil and pump it back down, the dirty water back down into the well, sort of waste, or carbon dioxide out of the natural gas, there’s initial processing facilities that are sort of pre-sending stuff into the pipeline. There’s tank farms where the pipelines store stuff sort of intermediate. There’s liquid natural gas ports. There’s oil oil ports. There’s oil tankers. This is all part of midstream, the process of moving stuff and you’re from from place to place and to a degree storing it while you’re moving it.

And then downstream is sort of everything you do after it comes out of the pipeline. So there’s refining, turning it into diesel fuel and and jet fuel. There’s the the the finished processing on on natural gas, taking out all of the the natural gas liquids, making it basically pure methane with not much else.

There’s even stuff like trucking. Gasoline from the pipeline to the gas stations is considered part of downstream.  Midstream kind of rears its head again because, you you might have the concept of a gasoline pipeline. So you’ve got the oil pipeline bringing the crude oil to the refinery. Then you’ve got the, you sort of hit midstream again, taking the finished product, gasoline, and sending it to consumers. Then you’ve got the trucks, you’ve got the gas stations.

Each of these sort of upstream, midstream, and downstream sub-industries has sort of many components. I I’ve lost it now, but I saw a list once of, here’s all the different kinds of things that can be in midstream.

And it was like, I counted, it was 27 kinds of things. So it’s a complicated industry, but very loosely, upstream produces, midstream transports, and downstream consumes, in a sense, refines and produces the goods that we actually consume.

Andrew Ginter
So that’s interesting. I mean, human in the loop, I’ve heard that described as open loop, in power plants, which I’m more familiar with. You you monitor the turbines.

13:42.13
Andrew Ginter
The AI in the cloud comes back and sends you a text message and says, you should really service, the turbine in generating unit number three sometime in the next four weeks. And it goes into my eyes, goes into my brain. I go and double check with my fingers. I type on things. I say, i think they’re right.

And I schedule the service. That’s open loop. And yeah, it it gets scary when you start doing closed loop.

Yeah. Yeah. And And I would say that one of the key things, if you look at some analogous systems where they have actually gone from open loop, human in loop, if you will, to closed loop, you can you I’ll give two examples. One would be autopilot on planes and another would be self-driving cars.

And in both of those cases, you don’t just switch from open loop to closed loop. No, you do an extensive amount of testing and validation.

And you also, in many cases, build redundant systems that allow an an additional level of supervisory control on top of your normal process control loops.

And so like an example that I had heard about was a company that was looking at having, tank level measurements and looking at an AI model that would actually analyze the input feeds to that tank model. So, and and it would pull data from third parties that would look at the truck routes for the tankers that were pulling oil from that tank.

And so you could actually synthesize that data. Now you would have to put in place a lot of, I’ll call it ancillary systems and ancillary testing to make that safe enough to be like an autopilot on a car.

Because theoretically now with all that supporting testing, autopilot on a car is is supposed to be safer than humans.

And with people on their phones, like I see them these days, I think that’s become an increasingly low bar.

Andrew Ginter
Fascinating stuff. The The future of automation, I’m convinced. But if we could come back to the to the mundane, you talked about phishing, you talked about CVEs, exploiting vulnerabilities.

We’re talking about protecting these assets in the the the upstream and midstream oil and gas. Can you Can you bring us back to cybersecurity? How does how does this big picture fit with with what you folks do and and what you’re focused on cybersecurity-wise?

Tom Sego
Absolutely. So one of the things that’s interesting is, I love talking to customers and I try to spend at least 50% of my time and actually listening more than talking to customers and understanding what their challenges are and how we can solve those.

And in the case of oil and gas, there were three customers that came to us and told us the identical story and they became our largest customers.

And this the story they were telling us was that they had these highly distributed assets all over these these very wide geographic areas And they had spotty cellular and they had backup satellite to enable that connectivity that they need. They need the eyes and the ears in the field because it would be cost prohibitive for them to get in a truck and and drive out there to monitor that every few hours.

So the challenge they brought to us was the security team didn’t like the operations team having this insecure connectivity to these remote areas.

And so the security team said, you need to do something about that. And that’s where BlastWave came in. And we said, we can actually use our software-defined networking solution to cloak those assets so they’re undiscoverable to adversaries.

but also segment them so that if there were malware that were to get introduced in one area, it would not spread to others. And then finally, you would have the ability to get secure remote access.

And one of the coolest parts about this is this is not a bump in the wire kind of solution. This is a solution that allows routing and switching between groups of devices and users.

So it cuts across firewalls as if they don’t exist. It doesn’t route traffic based on source and destination. It routes it based on identity.

And this is something I think is very unique to us. And it’s something that I think customers absolutely love. And this has enabled us to address a benefit that we hadn’t even thought about, which was when oil and gas companies acquire other oil and gas companies that one of the first things they face are the need to maybe re-IP this architecture.

Because oftentimes the IP space, there’s overlapping addresses. And the that can be problematic. It can take a lot of time.

It can take a lot of money. And that’s another solution that we’ve been able to deliver calm almost by accident. We had one company, an oil and gas company, that acquired a $30 billion dollars acquisition target.

That’s a big company that you’re acquiring. And they were able to protect that with Blast Shield in three weeks of acquiring them. And they didn’t have to re-IP anything.

Again, that’s just because of the way we do this network overlay. So there’s a lot of cool things that that that use cases that we’ve discovered through the process of listening and talking to customers.

Andrew Ginter
Cool. So, so, you’ve said the the phrase SD-WAN, software defined wide area network. I have never figured out what is an SD-WAN. I mean, I’ve worked with firewalls for 20 years.

I did a lot of different kinds of networking, not not hugely. I mean, and I never worked for a telco, but but can you work with me? What is an SD-WAN? What is your SD-WAN? How does one of these things actually work? What does it do?

Tom Sego
Yeah. Well, first of all, I said SDN, not SD-WAN. So I said software-defined networking, which is a principle, not SD-WAN, which is an architecture.

What I guess the best way for me to think about this, and keep in mind, I’m a chemical engineer, not a software engineer. So I That means i’ll yeah if it takes me it may take me longer to understand these concepts, but when I finally do, I can probably explain them to people.

So the the the way I’ve learned this is that we essentially establish, we abstract the policy from the network infrastructure so that such that you can have a group of devices or a device itself that essentially associates with an IP address that’s an overlay address, much like you get network address translation.

All right, so you have a an original IP address, you have and a translated IP address, and the software-defined network then uses the overlay address to both communicate with each other, to establish the most efficient route,

because performance is very important in OT environments, unlike IT environments. And this allows us to optimize the path for any given packet, which is also very cool. So that’s one of the elements that I think is important in software-defined networking.

um The other thing is, is that it creates this illusion that it is a point-to-point between two different devices or two different groups.

And so that’s part of the abstraction. So if you don’t have to like set the path, which is what firewalls do, path, looking at the routing, how you go from this firewall to that firewall, from this port to that port, when you just abstract that to, I wanna go from this centrifuge to that control room,

It doesn’t matter if the infrastructure changes. And this is a very powerful yeah benefit of software-defined networking. Because if you’re just looking at the device you want to protect and the user who wants to connect to that protected device, as the environment evolves and it absolutely will, you don’t get put in the penalty box like you would in a firewall situation where you could get firewall rule conflict.

And if one thing to think about, Andrew, is when you think about the breaches that occur, about 100 percent of those breaches already have firewalls.

And so that means that the firewall didn’t work properly, which is usually a result of a firewall rule problem or the the environment has evolved in such a way that it’s no longer protected. There’s a hole.

And of course, we all know that adversaries just need to be right once. Whereas us defenders, we’ve got to be right all the time, which is very tough unless you’re my wife.

Andrew Ginter
There you go.

Andrew Ginter
so So Nate, let me jump in here. I’ve, the as I told Tom, I’ve wondered about this space of software-defined networking, wide area networking for some time, and i’m I’m beginning to wrap my head around it.

um he gave the example of, you you might imagine that we’ve got oh the internet, local area networks, wide area networks were designed so that devices have internet protocol addresses and they talk to each other and, routers move messages from one network to another. So they get from the source to the destination.

Why is any of this complicated? Why do we need any more than that? One example that that Tom gave was acquisitions. If company A, i mean, there’s there’s internet addresses, the 10-dot series, two to the 24th addresses are private addresses.

Private businesses can assign them to their, ad written to to assets on their private networks and never show those those ad addresses to the public, to the the public internet. That’s fine.

There’s another set, 192.168 is a 16-bit address range that everyone uses. So you might say, so so what? Company A uses, let’s say 10.0.1 through 10.0.20.

They’ve got a lot of assets. They use up a bunch of the address space. And then they buy company B that’s used the same addresses because they’re private addresses. You don’t have to register that you’re using them in public.

And now all of the equipment has the same IP addresses. For For each IP address, there’s two pieces of equipment in the network. How do you route messages from from these subnetworks, from these assets to each other?

um This is the problem of renumbering when you acquire a business. Often you have to renumber it’s it’s a pain in the butt on on IT t networks.

It can shut you down until you’re done and tested the renumbering on OT networks and nobody wants to shut down. So you if if there’s a piece of technology, i mean, the the the textbook technology is network address translation, part of most firewalls.

It lets you hide some private addresses and assign a different address to sort of that set of of private addresses. You’ve got to set up a whole bunch of firewall rules You can do that sort of manually painfully, but it gets worse than that.

I mean, I was talking to Tom after the recording. He gave me an example that I didn’t capture on on the recording, but he said, Andrew, they’re they’re working with an airport and the airport’s building a new wing.

I mean, this is common. Airports expand. And in every, let’s say there’s 27 gates in the new wing. Every gate has got one of those machines, those those ramps the that sort of snuggle up to the aircraft and the door opens and people come out and step onto this device that has, I forget what the name of it is, moved up to the aircraft and then they they walk into the into the airport building.

Every one of these devices has automation, has computers.

Every one of these devices, when you buy it from the manufacturer, the manufacturer assigns the same private addresses to every one of their products. So now you’ve got 27 of these ramps in the new wing, and every batch of 20 computers or devices that are built into the ramp have the same IP addresses.

How do you route this stuff? Again, you can put firewalls in place. You can do So now you need a firewall in every ramp. You need you need technology. And it gets it gets more complicated than that.

Andrew Ginter
For example, many years ago, I worked with a bunch of pipelines. I remember one pipeline, thousand kilometers long, pumping stations, compressor stations, all the way down the pipeline. Communication was important.

You have to communicate with these these stations or you have to shut down the pipeline. It’s illegal to operate a pipeline in in that jurisdiction unless there’s human supervision.

And so you had, there there was a fiber laid along the right of way for the pipeline. And from time to time, some fool would run a backhold through it.

So you’d need backup communications. I kid you not, this pipeline had something like seven layers of backup communication. There was satellites, there was DSL modems to the local internet service provider.

There was cable modems when there were a local internet service provider. There was… I don’t think I think this was before the era of of cell phones.

there were There were analog modems. We’re talking 56 kilobit, 100 kilobit per second modems that you can route in an emergency internet protocol down very slowly.

And they had built their own by hand. They had rolled their own, what today I think would be called a software-defined wide area network, where the task of that component was to say, I need to send an internet protocol message from the SCADA system to device 500 kilometers away

what infrastructure is up, what infrastructure is dead. If a piece of the infrastructure, the communications but infrastructure has failed, then activate another piece of the, one of the backups and change all the routes, change all the firewall rules so that

All of the messages that have to get from a to B can get from a to B. It was it was it seemed to me ridiculously complicated, but in hindsight, it it sounds like the same kind of need that modern software-defined wide-area networks address.

They address security needs as well as just the basics of getting the messages from one place to another when the underlying infrastructure changes from moment to moment.

Andrew Ginter
um So so that that kind of makes sense. You’re I think of wide area network, I think of routing. So there’s a routing element. You’ve got multiple paths. The system sort of auto-heals and figures out the best paths or presumably the cheapest paths.

But you’ve also talked about users and and security. How does How does this routing concept work with security?

How is security part of this? You’ve also mentioned firewalls. Can you can you can you dig a little deeper?

Tom Sego
Yeah. Well, I think I think we in a way are disrupting firewalls that are used for industrial, lots of industrial applications.

There are great uses of firewalls. They’re a fantastic tool, but it’s it’s kind of been used like the if you have a hammer, all the world looks like a nail. And, especially again, I’ll talk about these remote oil and gas locations where you may only have five or 10 devices.

And so the idea of having a firewall to segment that is ridiculous. The expense would be prohibitive. So that’s one of the other reasons why it’s so cool about the way we can scale dramatically from protecting five devices at a very remote well site to 2000 devices with a single gateway.

So there’s a lot of flexibility that we have that, that firewalls can’t deliver. And when you look at a comparison of a project that involves a firewall as a solution versus blast shield, we are, we take one 10th the time, cost one fourth as much.

We can deliver this with half the administrative lift. It’s much easier to deploy as well. And it actually works. So there’s a lot of benefits that we bring over a firewall kind of solution.

Andrew Ginter
Okay, so so I understand these are these are powerful benefits, but can we come back to the technology? Can you tell us what does this stuff look like? I mean, you said it’s not a bump in the wire.

Physically, what does it look like? Is it a DIN rail box at each of these sites? Is it a DIN rail box on on a central tower? is it what Is it something in the cloud? Can you talk about what is it that that is solving these problems?

Tom Sego
Sure. So there are basically five components that we have to our platform. The first two create the authentication handshake. One is a client that runs late locally on on your HMI or on your machine.

And then you also typically have either a mobile application that provides the and MFA without passwords. And that was patterned after Apple Pay.

So again, I spent a decade at Apple. And so the idea was, let’s try to use some of that technology to provide stronger authentication. The other thing that we have is we have a gateway.

And the gateway is a software appliance. And it can be deployed on x86 bare metal. It can be deployed… On containers. It can be deployed on Kubernetes clusters.

It can be deployed in the cloud, AWS, GCP, Azure. It’s very flexible and it can be operated both in passive mode and active mode. So in the pat traffic path or outside the traffic path.

We also have an agent that can run locally on a machine, which most people know what agents are. And then finally, there’s an orchestrator that is used to drag and drop devices and people into groups and then establish policies between those groups.

So that’s a little bit about the way that the but technology is set up. And one of the things that that we found is that you can have people who are, I’ll say, less sophisticated than many CCNA trained professionals.

So they don’t even need to know how to use command line to deploy our solution. So it’s relatively simple. We have an example where one person is managing 22,000 devices.

So again, that provides a benefit to them in terms of OPEX reduction ongoing. So that’s a little bit about the way technology work and these the and the way these components fit together. Does that answer your question, Andrew?

28:55.44
Andrew Ginter
ah That’s close. I mean, what what you’ve described is sort of the the pieces of the puzzle. But, I’m still a little weak on on on how they work together. I mean, you again, we’ve we’ve used the word routing a couple of times.

29:09.02
Andrew Ginter
um To me, there’s there’s two ways to do routing. You can either take the message messages into one of your components, I’m not sure which one, and figure out where they belong and send them on the way yourself. You can be a router.

29:24.15
Andrew Ginter
Or, and I understand sometimes some software WANs can do this, they reach out to routers like firewalls and just routers and who knows what else that can route messages.

29:38.01
Andrew Ginter
And they send commands to those devices when things need to be routed differently. Is one of these models what what you use? how How do you guys do the routing?

Yeah, so let me talk about how these pieces all fit together. So the software appliance that is the gateway sits upstream of the switch and usually downstream of the firewall.

And what it often will do is it will provide what we call layer two isolation. And so what that is, if you think about, we can essentially turn a 48 port switch into 48 VLANs so that each one of those is its own encrypted unit that can’t see their neighbors and can’t talk to their neighbors in unless the policy allows that to happen.

And so that level of very granular control is something we can deliver because of the way the gateway controls and manages the routing that you’re discussing.

Now, there’s two other components I didn’t really talk that much about. One was the authenticator, and the second was the client. And the client is different than the agent. And so the what the client does essentially is a challenge response between either the SSO, the FIDO2 compliant key, or the mobile authenticator.

And so what it’ll do is essentially produce a QR code that the mobile application would scan and then apply your face ID, and then you would be into the system, but not authorized or permitted to see anything unless the policy had already been allowed.

So that’s the way we manage both the authentication and the authorization. And that’s also the way we manage routing of traffic between devices, gateways, and the groups that that those devices are in kind of encapsulated in.

Nathaniel Nelson
So in his answer there, Tom was was trying to describe things, but admittedly I was getting a little bit mixed up because there were certain things that were upstream from other things and downstream from other things and layer two and switches. And be like Can you, Andrew, just help simplify everything we’re talking about here?

Andrew Ginter
Yeah, sure. So in my understanding, they have a few different kinds of components. And And I might have got this wrong. But, what I got out of it was, imagine… Um

You know, firewalls can do network address translation. They can say, I’ve got a bunch of addresses here. I’m going to show you a different address to the world. But, managing them in sort of scale, at scale with tens of thousands of devices can be a real challenge, especially if each firewall is only managing a handful of devices. That’s a ridiculous number of firewalls to manage.

So what Thomas got, I believe, is a, I think he called it a gateway device. It’s something that sort of sits between, let’s say, a small network of five to 10 devices and the infrastructure.

And you can assign whatever IP address you need to to that gateway. Oh It might, in fact, have two addresses, one on sort of the infrastructure side and one on the device side.

So it has a device address that is compatible with whatever stupid little network of five local, always reused, ramp IP addresses, the, the, the airport ramp addresses, it’s, it’s compatible with that bit of address space.

It talks to those five devices. And when those devices send it messages, it forwards those messages into the infrastructure and it figures out the addressing. It figures out the, it does encryption.

If you’ve got sort of more conventional, um, Windows or Linux communications, you can put his software on those devices. They that That software will do the crypto, the software will connect sort of natively into the infrastructure and and sort it all out.

And then, the the thing of beauty is, okay, those pieces kind of make sense. The thing of beauty is what I heard was they’ve got a management system, which says, okay, you have 20,000 devices.

um half of them have exactly the same IP address. That doesn’t matter. This device over here in this building in this country can talk to that device over there.

It’s allowed. But when that device wants to talk to Andrew’s laptop, because I’m a a maintenance technician, Andrew has to provide two-factor authentication.

So you can, you basically, you you you stop caring what IP addresses these devices have you don’t have. You’re not configuring routing rules. You’re configuring permissions in a sort of a high-level user-friendly permission manager.

And all of the routing nonsense and the encryption nonsense is figured out for you under the hood. So you can you can think about… Your your big picture of devices that need to talk to each other, who should be allowed to talk to each other, instead of how do I route this when the IP address is conflict? You don’t have to ask that question anymore.

Andrew Ginter
Cool. So that that starts to make sense. I mean, can you talk a little bit about, you’ve been doing this for, 2017, this eight years. Can you talk about, can you give us some examples to to to help us understand, how this stuff works?

Tom Sego
Well, I think the, having run this for almost eight years now, the the journey was not a straight line. We went through, we originally started out, believe not, Andrew, as a hardware company.

And the the thesis was to build an unhackable stack. So this sounds naive, and it was. We were going to start with a chip, a new chip, that we had a partner developing that would have an onboard neural net.

It would create 17 key pairs and it would encrypt the bootloader in the factory and burn a fuse so it couldn’t be reset. And that was the foundation of our product. And then we were gonna write our own kernel, write our operating system. And this was from someone who helped write the OS 10 kernel.

We were gonna write that in such a way that it used byte codes and would not be exposed to buffer overflows and other issues. So it could, we were going to use formal methods to even prove the kernel.

And then we’d have our networking layer, which is what our company is now. And then we’d have our own SDK to manage applications that would also use formal methods. And then finally, we would have the authentication layer that we also have today. So we went from a five,

very ambitious levels of of tech stack to two. And then we have other people doing some of those other things. I think the market really wasn’t ready for something that complex, maybe that secure from a, on the higher end of the security spectrum, if you will.

um the market just really wasn’t willing to pay that. And so we simplified, we pivoted. And then by the way, once we did come out with our hardware product in February of 2020, there was another global issue that hit everyone that caused us to then pivot to a software as a service model, which then required some more development and everything else. So we didn’t really launch our product until late in 2021 and started getting our first customers very shortly thereafter.

And since then, we’ve grown very rapidly to the point where this most recent year, we quadrupled our our revenue and tripled our customer count.

So it’s been an exciting ride.

So let me give you an example. The one one customer, again, an oil and gas customer who was, again, trying to, they were faced with a challenge where they were going have to build their own cell towers, essentially become their own wireless ISP. And this is not unique to this oil and gas customer.

There are many that are facing that. And I don’t know if you or your audience knows, but it’s about a quarter million dollars to build a cell tower. And you have to have many of them. So in in in a relative sense, we are not just delivering security to this customer, we’re also so helping save them a ton of money.

So instead of 10 to $20 million, dollars they’re spending a fraction of that, which is also very interesting. One of the When they did this acquisition, there was another company that did an acquisition.

They wanted to sell off certain components too. So they wanted to sell off the saltwater rejuvenation or… It I don’t know exactly what the right word is, but they wanted to offload this asset.

And one of the things that they were able to do very quickly, because all of our segmentation, all of our granularity and access is done in software.

We can essentially just take that new entity. Put their users in a group, put the devices that they control into another group, and they would have complete control of just their newly acquired saltwater assets and no visibility, no access at all to the oil and gas parent company.

So that was another great example of using this in a creative way.

Andrew Ginter
So you’ve mentioned acquisitions a few times. I mean, I live in Calgary. This is oil country. I hear about these acquisitions all the time. Is this Is this sort of part of the the the the genesis of your organization? is is this How often do these things happen? How complicated are these sort of mergers and acquisitions technology-wise that happen all the time?

Tom Sego
Well, they happen very frequently, especially, again, in oil and gas. In the In the case of oil and gas, because one customer sorry one asset owner has a certain tech stack that can only profitably make money up to a point.

And then they can sell that asset to someone else who has a richer skillset that can extract more profit, more money, more revenue from that same resource.

and And I would say an example that we’ve also seen where people are pleasantly surprised about Blast Shield is when there yeah there’s one one oil and gas customer that acquired a company.

And their biggest fear was they were going to have to do an IP space assessment and figure out whether there were overlapping IP addresses. And so instead of having to do that, which they didn’t have to do at all, they just deployed our software overlay and immediately were able to segment using software each one of these devices, even regardless of whether the underlay IP address was the same.

That saved a lot of money in truck rolls. That saved a lot of money and hassle and headaches in managing that that IP space, which which they were very happy about. And the way they described it, actually, they described it two ways to me.

One way was, my God, this is like a Swiss Army knife. And the other guy said, this is like duct tape. It’s like networking duct tape. It has It provides lots of different purposes and is very versatile to basically deliver the network they want with the network they have.

Andrew Ginter
So let me just sort of emphasize, Tom has said, you talked about changing IP addresses a few times. I talked about it a few times. I’ve actually, from time to time had to change IP addresses on stuff, not so much in an industrial setting, just, just internet protocol networks, just, business infrastructure.

And here’s the tricky bit. It’s very hard to do that remotely.

You know, Imagine that you you want to remote into a remote substation. There’s nobody there, but there’s 100 devices. And you have to log into each device with, I don’t know, SSH or remote desktop.

And you’ve got to change the IP address on the device. And at some point, you’ve got to tell the firewall that it’s talking to a different network of IP addresses.

And if you do that in the wrong order, if you, let’s say, hit the firewall first, now you can’t send messages to any of the devices because the firewall doesn’t know how to route to those devices anymore. They have different IP addresses. So you have to undo that. Now you go into the device and you give the SSH command a Linux box. You give the that that command line command to change the IP address, and it stops talking to you because you’re connected to the old IP address. You’ve got to try and connect to the new IP address.

Only the firewall won’t connect you to the new IP address because it its IP address hasn’t been updated. So now you have to sort of blindly change all these addresses. Then you change the firewall, and then you see if you can still talk to these devices, and three of them have gone missing.

Why? Did I fumble finger the IP address? Is there some other problem? It’s just really hard to do this remotely. And so, again, if you have 700 sites, you’ve got to put people in trucks and drive out to these wretched sites to make these changes.

If there’s a way to avoid that, you can save a lot of money. So, yeah, I kind of get that it’s really useful to avoid doing that.

Andrew Ginter
so So this is starting to come together for me. I mean, you can do the network address management in your, what did you call them?

The gateways.

Tom Sego
Gateway, yeah.

Andrew Ginter
And that gives you an enormous amount of flexibility. But And it’s it’s the the client that does the the crypto. Or maybe it’s the agent.

39:22.07
Andrew Ginter
I’ve i’ve i’ve lost track.

Tom Sego
The client is used to authenticate.

Andrew Ginter
Right.

Tom Sego
The agent runs on typically a server in the cloud, those kinds of maybe a historian type of use case. The gateway is the workhorse because so much of OT infrastructure cannot run an agent.

And so because it can’t run an agent, you need to have a gateway that can do the encryption and decryption of traffic. Now, when you think about the way a lot of these processes are controlled, they use PLCs.

And the PLCs, we don’t encrypt the traffic below the switch.

We don’t interfere with that. However, with the traffic that is upstream of the switch, all of that’s encrypted wherever it may go.

So I think that’s that’s the way it’s done.

Andrew Ginter
One other technical question, you mentioned CVEs and exploits and vulnerabilities earlier.

I mean, i’m I’m familiar with, let’s say firewalls that that say they do stuff like virtual patching, meaning if there’s a vulnerability in a PLC, the firewall, if it sees an exploit for that vulnerability come through, will drop the exploit and will protect the, the prevent the exploit from reaching the the the device. Is Is that the kind of thing you do when you talk about about protecting from exploits or are you doing something else?

We’re definitely doing something else. And I think the the approach that we take is we use this networking cloaking concept where you have to authenticate first before you can see anything.

There’s no management portal. So there are zero exposed web services. If you run a network scan on a factory, that’s protected by blast shield, you’re going to come up with nothing.

And what that means is if there are CVEs, and I guarantee you there will be, there will also be zero-day viruses, okay which may not be on anyone’s list.

And so in those both of those cases, as well as ancient devices that are never going to be patched, you’ve got a way to deal with these unpatchable systems because they’re unaddressable. And so it’s going to be very difficult to exploit those.

Andrew Ginter
Cool. So, I understand you’re you’re you’re heavy into oil and gas with all of the examples we’ve been talking about oil and gas, but I’m guessing you you are active in other industries as well. Given your personal background, are you active in other industries? what Can you give me some examples of what’s going on there?

Tom Sego
Yeah, absolutely. I think manufacturing is a fantastic kind of industry for us. They oftentimes have our little bit early adopters with with as it pertains to machine learning, predictive maintenance, those kinds of things, advanced analytics.

And we had one a manufacturing customer, in fact, who was hacked and many manufacturers do get hacked from time to time. They were hacked and the board asked the CISO to have an assessment to figure out what their risk posture was.

And before they could complete that assessment, they were hacked again. And so this really lit a fire under the entire kind of security team.

And they basically came up with a list of findings. And with those findings, they started implementing those findings. And they were testing various kinds of solutions.

And in one facility, they had 10 different lines, manufacturing lines. And they had deployed blast shield on one of those manufacturing lines.

They got hacked a third time. Now, this time, though, nine of the 10 lines shut down, whereas the line that was protected by Blastshield continued to run.

And what was really interesting about that is how quickly the organization responded. The CFO of this company responded and elevated that to the parent private equity company.

And now that’s leading to us becoming the default standard for not just that one company and all of its 17 plants, but also the parent private equity company and all the other manufacturing facilities that they’re trying to manage. Okay.

Andrew Ginter
Cool. I’m I’m delighted to hear it. The world needs more cybersecurity. Um

I mean, I’ve learned a lot. Thank you so much for joining us. Before we let you go, can we ask you to sum up? What what are the key concepts we should be taking away from from our conversation here?

Sure. So I think the company, as it was founded, was trying to establish protecting critical infrastructure based on first principles. And the first principle was to try to eliminate entire classes of threats if possible.

And so our solution then tries to eliminate phishing credential theft. So we we have an MFA passwordless feature. We also allow you to segment using software.

We cloak your network so it’s undiscoverable. 35% of all CVEs discovered last year are what are called forever day vulnerabilities. And so that network cloaking capability means that they’re not exploitable.

And then finally, we also have a secure mode access component in there. So we’re trying to deliver a lot of value to our oil and gas manufacturing customers so that they when you couple this with a continuous monitoring and visibility tool like a nozomi dragos dark trace armis SCADAFense industrial defender the group clarity so when you combine those two you get a ton of protection at a very low price

Nathaniel Nelson
So that just about does it, Andrew, for your interview with Tom. Do you have any final words to take this episode out with?

Andrew Ginter
Yeah, I mean, I really like Tom, the the the customer that gave the duct tape analogy. You have lots of little networks, sometimes thousands of devices.

Half of them have literally the same IP address or half of these, tiny little subnetworks of of five devices on on airport runways or on, on webbages.

ah networks that you’ve acquired with, acquiring an oil field, they all have the same IP address. They all have the same IP address range. None of it’s encrypted. It’s just a mess.

And, this is something that lets you patch it all together. You need crypto, you need authentication, you passwordless is good. Use certificates instead. They’re harder to phish. You need to hide all of these repeated subnets with the same IP addresses.

You need a permissions manager, saying A can talk to B.

You need infrastructure underneath the permissions manager to make the messages from a go to B. You need to to have some synthetic IP addresses so that when you set everything up, your SCADA system can talk to an address and a port, I don’t know, probably on the gateway or or some piece of the infrastructure rather than the real address that’s repeated a hundred times in your infrastructure.

This just makes… A lot of sense. I It seems to me there’s there’s a a bright future for this kind of, of again, duct tape or just patch it all together and make it work and throw some security on top of it. Crypto authentication, this is all good. I’m i’m i’m impressed.

Nathaniel Nelson
Thank you to Tom Sego for speaking with you about all that, Andrew. And i always, gotta say that again. Well, thank you to Tom Sego for speaking with you about all of that, Andrew. And Andrew, as always, thank you for speaking with me.

Andrew Ginter
It’s always a pleasure. Thank you, Dave.

Nathaniel Nelson
This has been the Industrial Security Podcast from Waterfall. Thank you to everybody out there that’s listening.

Please note: This transcript was auto-generated and then edited by a person. In the case of any inconsistencies, please refer to the recording as the source.

Nathaniel Nelson
Welcome everyone to the Industrial Security Podcast. My name is Nate Nelson. I’m here with Andrew Ginter, the Vice President of Industrial Security at Waterfall Security Solutions, who’s going to introduce the subject and guest of our show today. Andrew, how are you

Andrew Ginter
I’m very well, thank you, Nate. Our guest today is Kenneth Tittelstad. He is the Chief Commercial Officer at Omni, and he’s also the Chair of the Norwegian International Electrotechnical Committee of Subgroup working on 62443. So this is the Norwegian delegation to the IEC that produces the widely used IEC 62443 standard.

We’re going to be talking about credible threats. What should we be planning for security wise? And by the way, I happened… I had opportunity to be in Norway and I visited Kenneth at the Omni head office where they have a lovely recording studio. So we recorded this face to face in their in their studio in their head office.

Then let’s get right into your conversation with Kenneth.

Andrew Ginter
Hello, Kenneth, and welcome to the podcast. Before we get started, can you tell our listeners, give us a bit of information about your background, about what you know, what you’ve been up to and and the good work that you’re doing here at Omny Security

Kenneth Titlestad
Thank you so much, Andrew, and welcome to Norway and our office. It’s, I’m so glad to have you visiting us. So my name is Kenneth Titlestad and I’m working as a Chief Commercial officer in Omny and I’ve just started as a commercial officer here in Omny. I went over from Southwest area where where I was heading up OT cyber security for. I’ve been doing that for six years.

Before that I was working in Ecuador also working on OT cybersecurity, so I’ve been working in the field now for almost 15 years and also for the last five or six years I’ve been chairman for the Norwegian Electrotechnical Committee, the the group that is handling IEC 62443. I’ve been diving deep into the cybersecurity now for quite many years.

And at Omny, we are developing a software platform for for handling cyber security and security for critical infrastructure. It contains security, knowledge graph and. AI that provides actionable insights into security for critical infrastructure. So it’s about it out and physical infrastructure.

Andrew Ginter
OK. Thank you for that. Our topic today is credibility. Now this is talking about risk. You know a lot of people think risk is boring. OK, a lot of people when they enter the industrial security space, they they want to know about attacks. They want to know about the technical bits and bytes. You tell me that you got interested in risk. Very long time ago. Can you talk about that? Where? Where did that come from?

Kenneth Titlestad
Absolutely. I’m I’m not sure if I when I when I considered it as as a as a risk or as a as a field of expertise. So when I was just a small boy, actually my dad, he worked as a control room technician offshore in Conoco Phillips or back then it was called Phillips. So when I was only two years or three years old in 1977. He was working at the Palau offshore oil and gas. Before and I don’t remember this of course back then. But it it, uh, it was always a topic around the dinner table at my my home where he talked about how it was working in the oil and gas business. So in 1977 he was on his way out to the platform when the big horrible blowout happened. He was not actually. He hadn’t arrived at the platform, but he was on his way out there. So it it really was a big topic around the dinner table all the time about safety risks involved in oil and gas.

So I was always listening with my my small ears back then being a bit fascinated about this world, I didn’t see the real danger in it, but I I was trying to picture it in my mind what it was to actually work on in these kind of environments.

So it I was kind of primed back when I was just a small, small boy and later on when I moved into the I I was more into computers. So I did a lot of gaming and programming on Commodore 64 and I started to work in Ecuador on the IT side. But I was still fascinated, fascinated, about the core business being oil and gas and production and exploration. So when I actually got my first trip offshore. I kind of felt that the the circle was closed and I saw the big world, the industrial world that my dad was had been talking about for several years and the kind of the risk perspectives also kicked in. The first thing you meet when you step on board, such a platform is the HSE focus a lot of focus on HS.

OK. And it’s for a reason and I fully got to understand that first, when I actually came on board such a facility, I understood why it’s so important, because it’s it can be really dangerous if you don’t have control over what you’re doing. So that’s when I actually saw the big scale of risk as a perspective.

Andrew Ginter
Yeah. Offshore platforms are intense. I’ve never set foot on one myself, but I’ve I’ve heard the stories quite the environment. And this is I mean, we’re talking about industrial cybersecurity, so you know offshore platforms are intense in terms of physical risk. Can you talk about cyber?

Kenneth Titlestad
It’s it’s an emerging topic. So when I was working in, in Statoil when it was called stator, now it’s equinor, we started to look into that area Around 2010 two 1011 I I still remember the day when people came charging into the meeting room and they started talking about the news of Stuxnet. So that was I I think we got to hear about it in 2010. I was working on the IT side and I I was responsible for large part parts of our Windows infrastructure in the company and we started to I I started to look into what what this SCADA things, what what is it I didn’t know about. PLCS I had never seen a PLC. I didn’t know that there was actually other kind of digital equipment operating critical infrastructure. So so with Stuxnet I started to to dive into the landscape of cyber security.

Kenneth Titlestad
And also as a company, we started a big uh journey back then on on really making uh OT much more cybersecurity. And Stuxnet was kind of a kickstart for it.

Nathaniel Nelson
Andrew, it feels like maybe there are certain kinds of seminola cyber security incidents in the O2 world. We talk, we reference off in the 2007 Aurora test. Maybe, you know, Triton and destroyer. But Stuxnet is that foundational thing that, you know, set the timeline for everybody, right?

Andrew Ginter
Indeed. And you know I was active in the space. I mean, I was leading the the team at Industrial Defender building the world’s first industrial SIM at the time. So Stuxnet was big news. I did a lot of work on Stuxnet. I had a blog at the time, you know, every time I learned something new about it because somebody had published a report, somebody had published another blog.

I’ve done a little research on my own. I published a paper on how Stuxnet spread because, you know, analysis had been done of the artifact. You know, the malware. But it had been done by IT. People at Symantec at I think he said, a bunch of people had analyzed the malware and you know, that’s work I couldn’t do. I’m not a I’m not a a reverse analyst.

But I sat down with Joel Langill. I sat down with Eric Byers and we investigated the impact that Stuxnet would have in a network. What would what would happen if you let this thing loose in a network? Given our understanding of the the Siemens systems, Joel was nexpert on the Siemens systems. You know, Eric and I were sort of more expert more generally, firewalls and industrial systems. So we all contributed to this paper and said here’s what happens if you let loose Stuxnet into an industrial network.

And in hindsight, I have to wonder if we didn’t do more damage than than good, because a lot of people learned stuff about Stuxnet, but there was only one outfit that benefited. And that was Iran’s nuclear weapons program. That was the only, site in the world that was physically impacted.

Why? I regret some of the stuff that I published about about Stuxnet.

Nathaniel Nelson
Do you recall if that research got traction, whether it might have gotten over there or is there no way to tell?

Andrew Ginter
I have no way to tell. I do recall a conversation, sometime later, because I’m a Canadian, I I work with the the Canadian authorities. I remember a conversation with Canadian intelligence services. And I remember, asking them. I’ve stopped, but at one point when I figured out that there’s only one place in the world that’s physically benefiting from my research, I stopped publishing anything about Stuxnet. And I remember some time after that talking to Canadian intelligence saying, I’ve stopped publishing anything about Stuxnet. You don’t have to tell me nothing. In the future, if you ever see me putting out information that’s helping our enemy, tap me on the shoulder, would you? And tell me. Shut up, Ginter. You’re doing more harm than good, and I will shut up. So, yeah, I, I look back on Stuxnet with with mixed emotions. It it was a wake up call for the industry. a lot of people learned about cyber security because of Stuxnet, but who benefited because of all that research?

OK. So that’s Stuxnet is. A lot of people got started in the OT space it was the big news years ago.

Andrew Ginter
Can I ask you, let’s let’s talk about industrial security and the work you, the work you’ve been doing. Stuxnet is where it got started. Where have you wound up? What are you up to today?

Kenneth Titlestad
Yeah, it’s. It’s as you say, it’s 15 years and it’s been, it’s for me. I think it’s been a very interesting journey. So but back in 2010 when when Stuxnet hit the news, I wasn’t immediately immediately diving into OT cybersecurity full time. I was working on the IT side, trying to secure Windows environment in a large oil and gas company.

But uh short, uh, after a while I move more and more over to outsider security, and I had my first trip offshore to oil and gas platform. I think that first trip was in 2013, so actually three years after the Stuxnet. But then I was going out just to to do some troubleshooting on a firewall. So, but more and more, I was moving into OT cybersecurity, and at the end I was. I moved over to Super Steria and I think it was in 2017. And at the end I was really working hard on finding really proper solutions for OT cybersecurity when when potential nation states are targeting you, what do you then do? If you must sort of have their mindset of assume breach and these kind of systems with the PLCS and all they are really, really vulnerable. What do you do when you are being targeted so then then I started to look into. I heard rumors that could there could be something that was non hacky.

So I started investigating into unidirectional data. Diodes was exposed to to waterfall. That was one of the first examples of of where I heard about non hackable stuff. And also I got to to to hear about the, the the Crown Jewel analysis, Cyber informed engineering. Back then it was consequence driven, cyber informed. Hearing. But those kind of topics really, really sparked an extra interest for me because then then I saw on some attack vectors on some of the risks I saw actually a solution that could remove the risk instead of just mitigating it.

Andrew Ginter
So your first sort of foray, everyone was interested in Stuxnet, but you started working on the problem you said with a firewall and to a degree that makes sense. I mean the the firewall, the Itot firewall is often the boundary between the engineering discipline on the platform in the industrial process and the IT discipline, where information is the asset that needs to be protected. And so that boundary is something that both the engineers and the IT folk care about, so that that kind of makes sense. I’m, I’m curious, you got out to the platform you were tasked with the firewall. What did you find?

Kenneth Titlestad
There. Yeah, it was actually kind of a long, long lasting ticket we had in our system, there was a firewall between it and OT that was noisy, so it was causing creating a lot of events and alerts on traffic that it shouldn’t have so I was tasked to go out there and try to troubleshoot this. We we absolutely didn’t think that it was a cyber cyber attack or kind of evil intent, but it was incorrectly configured firewall rule. But when I got out there I could see that it was. It was just incorrectly configured firewall.

There’s nothing, not, not anything dangerous or cyber attack involved, but I also got to to think of of a scenario where if it had actually been a cyber attack and one that created so much noise as well on a security boundary, a security component. Sitting on the outskirts of OT, shouldn’t the OT environment do something to sort of shut down or go into a more fail safe situation? So I got kind of interested in in actually the instrumentation behind your security components on the outskirts of OT. So that’s a topic I continued to explore for for several years, having in the back of my mind cyber informed engineering, non hackable approaches unidirectional systems and on on S4 last year I talked about the the safety instrumented system because safety has always been a particular interest of mine. So I talked about the cyber informed safety instrument. The system shouldn’t the safety instrumented system. At some point, when you’re under an attack, shouldn’t the the the sort of the big brain? Uh, in the room? Shouldn’t that actually take an action? An instrumented automated action and going into not necessarily. A fail safe only, but a more fail failover to a more safe and secure situation.

Andrew Ginter
So that makes sense in theory. I mean if the firewall was saying help help. I’m under attack over and over again. Should some action not have taken place on the OT side. But let me ask you this. It was a false positive. It would have shut down the platform. a very expensive that form unnecessarily, can we detect cyberattacks reliably enough to prevent this kind of unnecessary shutdown, and have if if we do shut down whenever there’s a bunch of alarms? Is that not a new sort of denial of service vulnerability? The bad guys don’t even need to get into OT. They just need to launch a few packets. That firewall generates some alarms in the shuts down without them even bothering to break in the OT. Is that really the right way forward?

Kenneth Titlestad
No, I totally agree. It’s not a good approach going forward. But at the same time I think to shut down one too many times, is is better than not actually doing it, so we should be kind of overreacting and and going into fail safe situation and it could cause unnecessary down time and it could. It’s vulnerability on the production side, but I think it’s much more dangerous with the false negatives where we actually don’t see any attacks and but it’s it’s actually happening. So false positive we need to reduce them, but it’s much more important to actually reduce the false negatives.

Andrew Ginter
So just listening to the recording here. I mean, this is not something I discussed with Kenneth, but we were talking about automatic action when we discovered that an attack might be in progress, for example, because there’s a lot of alarms coming out of the firewall, you know. He agreed with me that shutting down the platform was probably an overreaction because that introduces a new attack vector. The bad guys just need to send a few packets against the firewall, generate a few lines and the whole platform shut down, I agreed with him that something should be done, but we didn’t really figure out what. Here’s an idea in hindsight, a number of jurisdictions are introducing what they call islanding rules, meaning if IT is compromised, you need to, basically, I don’t know, power off the IT firewall, nothing gets through into OT anymore.

For the duration of the emergency, you have the ability to shut off all communications into OT. This is part of, the regulation says you must be able to island. So now you have that capability. I wonder if it isn’t reasonable to trigger islanding when you automatically discover a whole bunch of alarms coming out of anything, because the modern attack pattern, most of them of of modern day attacks, are not like Stuxnet, where you let it loose and it does its thing most of modern day attacks have remote control from the Internet, and if you island, if you break the connection between it and OT.

If there was an attack in the OT network, the bad guys can no longer control it. They can no longer send commands. So and this is not, this is not new. The the term islanding is a little bit new. The concept of sort of an automatic shut off is has been bandied about for for many years. But again, given that the regulators are demanding an islanding capability. maybe engaging it automatically from time to time is not the worst thing that can happen. It increases our security and the impact on operations is is minimal because you’ve you’ve deployed the ability to island already.

You’ve developed the capability of running your OT system independently, and so interrupting that communication for a period of hours at a time while you track things down and say, oh, that was a false alarm. I’m guessing is, minimal cost. So there’s an idea.

Andrew Ginter
OK. Well, let’s come back to our our topic here. The topic is credibility. we’re talking about the risk equation, the typical risk equation is consequence times likelihood. generally we do it qualitatively, but we we wind up with a number coming out of that to compare different different kinds of risks, high frequency versus versus high impact risks. can you talk about that? Where does credibility fit in that equation?

Kenneth Titlestad
I think it fits very well into that equation because when we we, especially when we talk about the likelihood or the probability part of it, the left left side of the equation it it’s always a very, very difficult conversation to have when you try to identify the risk or the the risk levels we are talking about or you try to identify the consequence levels involved. It’s sad to see that a lot of the conversations they go astray due to not being able to put the number on the probability or the likelihood, and I think it it the the conversation gets to be much more fruitful if we can get rid of that challenge on trying to figure out the number on the probability or the likelihood.

Credibility gives us tools in our language to actually be able to talk about the left part of the. So it’s something that is a bit more analog and analog value where we can move more towards the consequence approach, the consequence driven where the the right side of the equation is is more important to talk about as long as you get, if you consider it being credible.

Andrew Ginter
Well, I have to agree. Uh, I’ve argued in my previous in my last book that that likelihood is flawed, that at the high end of cyber attacks, not the low end, the low end likelihood actually works. The high end. The outcomes of cyberattacks are not random. If the same ransomware hits a factory twice and we’ve all we’ve done is restore from backup, it took them down the first time we restore from backup. We make no changes. It hits. Again, they’re going to go down the same way. It’s not random.

I argue that on the high end nation state, targeting is not random either. it’s not that they they they try for a while and if they if they don’t succeed they, go try somewhere else. Nation state threat actors keep targeting the same target until they achieve their mission objective. It’s not random. Once they’ve targeted you, it’s not random. Randomness to me doesn’t work at the high end. Credibility makes more sense. We know is is the threat credible? Is the consequence credible? If this threat comes after us, is this attack comes after us? Is it reasonable to believe credibility is what’s reasonable to believe, not who what’s reasonable to believe? Is it reasonable to believe that the consequence will be realized?

I think it makes a lot of sense, but it’s it’s new. I don’t see the word credibility in a lot of of standards. where does this sit? What what you know. Is this? Is this something people are talking about?

Kenneth Titlestad
Yeah, absolutely. In my work with the clients, I’ve been working with and also the professionals I’ve been working with, we have discussed for some years now that the, the OR we have discussed the big challenge of the the likelihood or the probability part of the equation. And we’ve we’ve without actually having having without following standards or best practices, we’ve seen that we need to skip the discussion on the probability or the likelihood and and talk about the consequent side of it first and then we revisit the likelihood and probability afterwards. But I also see in IRC 6243, especially with the 3-2, it actually talks about consequence, only cyber cyber risk analysis.

So that’s giving a opportunity to actually move away from the discussions on on probability and also of course with the consequence driven approach with cyber informed engineering, we start to see more focus on the far right side with the. The consequence consequence side but leaving out what to do with the likelihood, and I think with credibility we we get some some language based tools to actually play. Is it where we talk about it in a qualitative manner? Instead of having to force it into a number?

Andrew Ginter
So that makes sense to me. I mean, I have the sense that over time in the course of time, cyber attacks become more sophisticated, more sophisticated attacks become credible attacks that were dismissed A decade ago as theoretical have actually happened. Do you see that? what? What do you see coming at us in terms of sophisticated attacks in in the near?

Kenneth Titlestad
I think that’s a really challenging question looking far into the future or or far into the into the history to try to extrapolate what could we expect from the future we see with with the Stokes net, the against Ukraine. Triton, Colonial Pipeline. We see incidents that have had a really high impact, but there’s not very many of.

So, but we see it’s those kind of capabilities are being explored and are being put into different tools, so they can be used by not only nation states but also criminal groups. So with with that kind of analysis we can expect more and more sophisticated attacks and also by more and more non sophisticated groups. So we should expect increase in high impact incident.

Andrew Ginter
OK, so if we’re not talking likelihood, we’re not talking probability, we’re talking credible. How do we decide what’s credible? How do we decide what’s reasonable to believe?

Kenneth Titlestad
Yeah, that’s a that’s a good question. So we need to have some grasp of of what is credible and what is not credible. I’m also of the opinion that that the credibility part of the equation. It’s a qualitative thing. It’s not a zero or one, it’s something that is attached to a kind of a a slippery slope not easily defined. But what we could say if we are trying to to see credibility as a zero or one, what is credible things that have happened actually have happened once or twice or three times. They are credible, so the twice on incident or a safety only type of cybersecurity. That’s now a credible attack because it has happened.

And also near misses. That’s something that Triton was kind of a near miss. They didn’t actually cause it this this destructive attack, but it could have happened. And so we also have other near misses, incidents that we should be considering.

Andrew Ginter
So that makes a lot of sense to me. Credibility versus likelihood. How do we decide though credibility sounds like a judgment call. How do we decide? What’s?

Kenneth Titlestad
That’s a that’s a good question. I I I think there’s a good recommendations in 62443, for instance the 3-2 it it talks about the like I said, the consequence only as an example on how how you can approach the risk equation but it also talks about the need for focusing on worst case consequences. So it talks about essential functions, which basically could be the safety functions. For instance, you need to investigate the consequence if those are actually attacked and compromised. What could be the worst case consequence? So you begin there and then once you identify the worst case consequences, then you move over to the probability or likelihood dimension.

And then you need to consider all the factors. So what are the vulnerabilities involved? What are the safeguards and or what the the standard is talking about? You’re compensating countermeasures. You consider that you consider the function or the asset as well, that if there’s. If there’s no actual interest in the assets, then the vulnerability could be also non interesting to address or analyze. But you start with the consequence side, then you start to look at the likelihood and probability and then you are informed by the the consequence approach.

Andrew Ginter
OK, so let me challenge you on that. I’ve read the CI implementation guide. It says start with the worst case consequences. It says those words. I’ve not seen those words in three Dash 2. Are you sure that that you’re you’re not reading into 3-2?

Kenneth Titlestad
No, I’ve been searching for for that specific part of three dash too many times because because I’ve, I’ve heard others say that the same and it’s actually there. It’s really gold Nuggets in 3-2 talking about essential functions, specifically saying the worst case consequence and also specifically saying that you can choose to do a consequence only risk assessment, so that’s really important. Single words or single sentences in three after. So worth highlighting in the three Dash 2.

Andrew Ginter
OK. So that that makes sense in the abstract. Can you give me some examples what applying these principles? What what should we regard as credible?

Kenneth Titlestad
Yeah, interesting question. I think that the things that come to mind first is for instance the, the, the Triton incident. Before 2017, where when it actually happened, we didn’t think it was credible that someone would actually target a safety only system or cause a safety incident with a cyber attack with with Triton it we actually saw the first first of its kind and the threat became obviously credible. And then SolarWinds as well. It’s a very interesting study where the way they actually compromised the solar winds update mechanism, suddenly massive, massive deployment of kind of malware within critical and non critical infrastructure became really credible threat as well and also near misses. Of course we should be informed by things happening out there and coming on the news that are near misses that can talk about talk to us about what is a credible threat.

Another kind of near miss that I think or is not a near miss, but it’s scenarios or incidents that could talk about credibility is is where we actually have a safety incident. For instance, we we had have had lots of them in Norwegian oil and gas and in oil and gas gas. In general, is safety incidents where we, which is not cyber related at all, but where we see that it it could be able to be replicated by a cyber attack. So that’s something that we should be considering as a credible threat going forward where we actually could replicate the cyber or the incident with the cyber cause.

On credibility, I also think that we need to have in the back of our mind or in the analysis we have to have focus on on the technology evolution, the development and sharing of new technology. So we I see it as a graph where where we are exposed to more and more heavy machinery or heavy software that can be used on the adversary side.

Kenneth Titlestad
So with Kali Linux Metasploit now there’s also AI. So what is being about becoming a credible threat threat is more and more sophisticated stuff due to development of technology. So AI now is on on both sides of the table, or both as an attacker as a tool that makes more more attacks credible, but also on the on the defensive side where we actually need to use it to protect against more and more sophisticated attacks.

Andrew Ginter
So Nate, I was, let me go just a little bit deeper into into Kenneth’s last example. I remember talking to him about this two days before I recorded the session with Kenneth. I was at another event. I had 1/2 hour speaking slot. I was, listening politely to the other speakers. I remember. And one of the speakers was a a penetration tester. I remember asking the pen tester a question about AI and his answer alarmingly.

And, I discussed it with Kenneth. I discussed it with with others. Since the future is is difficult, I asked the AI the the pen-tester so you, you touched on AI. What should we look for from AI going forward? And I asked, should we worry about about AI crafting phishing attacks because I’ve I’ve heard of that happening. Should we worry about Ai helping the bad guys write malware to write more sophisticated malware because I’ve heard of that happening.

And I paused and his answer was Andrew, you’re not thinking hard enough about this problem, you know? Yeah, that stuff’s happening. But what you need to worry about is somebody taking a Kali Linux ISO image. This is the Linux disk image that everybody uses. All the pen testers use. Lots of attack tools, he says. Taking that GB of ISO image. coupling and adding it together with two gigabytes of AI model and the model has not been trained on natural language and creating phishing attacks. The model has been trained by watching professional pen testers attack OT systems, mostly in test beds. I mean, this is what pen testers do. They take a test bed that is a a copy of a system that they’re supposed to be, doing the pen test on no one that does the pen test on a live system. They do it on a test bed.

They use the Kali Linux tools. They attack the system and demonstrate how you can get into the system and cause it to bring about simulated physical consequences. So you’ve taught this AI model how to use the Kali Linux tools to attack OCF OT systems to brick stuff and bring about physical consequences. You take that training model, couple it with the image.

Wrap it up in enough code to run the image as a sort of kind of embedded virtual machine to run the the AI model the million by million matrix of numbers that is a neural network run the neural networ. Run the the the Kelly Linux image and have the AI operate the tools to attack a real OT system. Drop that three, 3 1/2 gigabytes of attack code on an OT asset, start it and walk away and it will figure out what’s there? It will figure out how to attack it. It will figure out how to bring about physical consequences.

I heard that and I thought crap. That’s nasty. back in the day, Stuxnet was autonomous. It did its thing, but it was a massive investment to to produce an an asset, a piece of malware that did its thing without human intervention. This strikes me as again something that will do its thing without human intervention, and it will figure out as it goes. It’s one investment you can leverage across hundreds of different kinds of targets.

I was alarmed. This is something I’m I’m thinking about going forward, it’s to me this is a credible threat. This is something we all need to worry about. I don’t know that the this thing exists yet. But I’m pretty sure it will in five years.

Andrew Ginter
OK. So that’s that’s a lot to worry about. Can I ask you know? Is everything credible? What? What in your mind is not a credible threat at this point.

Kenneth Titlestad
I would think that large scale destructive attacks on big machinery is not something that I would consider a credible attack, but it also goes back to the motivation of the threat sector, for instance, if you have a small municipality, I would lee that really heavy, sophisticated cyber attacks, a lot of them wouldn’t be actually credible due to the target not being interesting for such a threat actor. So large scale destructive attacks is something that in a lot of scenarios wouldn’t be a credible attack.

And then we have for, for instance, large large scale blackouts is quite an interesting story nowadays because a couple of weeks ago, I would think that it wasn’t actually a credible attack. Once we now see that it can happen, for instance, with Spain, it was probably not a cyber attack, but it was something that happened on the consequence side. If we can show that or or identify that it actually can be caused by a cyber attack, then that suddenly nowadays within the last week has become a credible attack.

And also swarm kind of attacks we I hear the discussions on that from time to time where where they see talk about whether it’s a credible thing where you attack millions of cars. As of now, I don’t see that as a credible attack, but things can change.

Nathaniel Nelson
You know, that’s an interesting statement he made there. That large scale attacks on heavy machinery. It isn’t credible. when I think about what we’re talking about on this podcast, the purpose of OT security presumably is that there are significant risks to really important machines. Large scale, but maybe at this point we’ve covered that.

That’s a good point. I think one of the the lessons here is that determining what is and is not credible is a judgment call. OK? Different experts are going to disagree. I’ve, few years ago I saw research published. Saying, look, here’s let’s take for the sake of argument, the possibility of attacking a I don’t know, a chemical plant and causing a toxic discharge. And the researchers concluded that it was theoretically possible, but it was such an enormous amount of effort on the on the part of the adversary, all of which would have to go on undetected by the sight, they said. in the end, I just don’t know that this is reasonable to believe that this will ever happen. So, that was one site.

But again, there are the experts, experts disagree. This is the the what I learned on the very first book I wrote. I got wildly different feedback from different internationally recognized experts. Here’s here’s an insight. To me, this means that when we make judgments about credibility, we probably have to be we have to make if we’re going to make a mistake, make a mistake on the side of caution, err on the side of caution because different experts have different opinions. We might be wrong. every expert has to be honest enough to admit that we might be wrong and build a margin for error into their judgment of what’s credible.

So even if we don’t believe that an attack that I don’t know destroys a turbine is credible, we might want to take some reasonable defences to against, such a not terribly credible attack in our opinion, but we might want to to deploy defences anyway.

Just because we might be wrong and this, this is something that that is also being discussed. It’s how big a margin for error do we need to build into our our planning. I mean I talked to a gentleman who produces who who designs pedestrian bridges. I said how do you how do you calculate the maximum mode? He says that’s easy. Andrew you you you build a barrier to either side of the bridge so vehicles can’t get on the bridge.

Most people are less than two meters tall. Most people are mostly water. You model 2 meters of water. The width of the bridge, the length of the bridge. That’s your maximum load. And then he says. And then he says, you multiply that by 8 and you build the bridge to carry the multiplied load. Because these are people we’re talking about, it is unacceptable for the bridge to fail under load. And so this is the margin for error that engineers routinely built into their safety calculations. I believe, we as as experts in cybersecurity need to build a margin for error into our security planning as well.

Andrew Ginter
So this all makes sense. One of the things that appeals to me very much about the credibility concept is using the concept to communicate with non-technical decision makers like boards of directors. You do this, you have experience with this. Can you talk about your experience?

Kenneth Titlestad
Yeah, I think it’s interesting. When we talk to board members and the the CXOS in different companies, they they don’t necessarily go into details about risk, but they know that they have a special accountability.

So so when we talk about credibility for for those kind of people, they are getting more on board with the discussions, they know they have a special accountability, they draw the line in the sand. For instance if if if the potential consequence is that somewhat somebody to die then that’s a non acceptable risk and they they take on that kind of position due to their accountability as as board members or or heads of of the company.

And they also are being accountable for from from the the government and from the for the society. So the some, some risks when it comes to the consequence side if if we talk about people dying then that’s absolutely and not acceptable risk for this?

Society and the representatives for for that kind of approach is is elected persons in the government and they put the the heads of the company or the Board of Directors as accountable for that on top of the company.

Andrew Ginter
So that makes sense. Boards care about consequences that the business or the society is going to find unacceptable. You didn’t use the word credible. How does credibility fit into acceptability when you’re communicating with?

Kenneth Titlestad
Yeah, we don’t have to defend against all possible cyber attacks. What we do have to protect against is the credible ones. So when we bring credibility in as a concept, then it’s something that communicates, communicates much better for the the Board of Directors and the heads of the companies.

Andrew Ginter
This has been good, but it’s it’s a field big enough that I fear we’ve missed something. let me ask you an open question. What? What should I have asked you here?

Kenneth Titlestad
We’ve been talking about credibility. Credibility is what is reasonable to believe. But it’s not enough to talk about reasonable attacks. We also need to be talking about reasonable defence. So what is a reasonable defence? We then need to be considering or or taking all the tools.

We need to use all the tools at our disposal for a reasonable defence, and nowadays that also obviously includes AI on the defensive side, not only on the offensive side.

This is also a very important part of me, of the reason for me joining Omny. So Omny is is built on our security knowledge graph, so it’s a data model where we can put all information we need about our assets on the vulnerabilities on the network, topologies, on the threats, the threat actors. So it becomes a digital representation or a digital twin of our asset. Combining that with AI which we have built in from the beginning, we get a very strong assistance on security where it matters most.

Andrew Ginter
Cool. Well, this has been great. Thank you, Kenneth, for joining us. Before I let you go, can I ask you to sum up for our listeners, what should we take away from this episode?

Kenneth Titlestad
Thank you, Andrew, for having me and and thank you so much for being here in Norway and and visiting us at our office. So we’ve we’ve had a good conversation about consequence, the focus on on the worst case consequences we’re we moved over to talking about credibility, replacing the the likely good concept with credibility, especially for high impact stuff where we don’t have the probability or the data to talk about it. We also talked about reasonable attacks and reasonable defences. So what is a reasonable defence against increasingly credible, sophisticated attacks with high consequences. So it’s been a really good discussion about all of these topics.

Kenneth Titlestad
If people want to know more about these topics or they want to discuss them, please connect with me on LinkedIn and message me there. I’m more than happy to discuss these topics and please visit our webpage Omnysecurity.com. Our platform addresses most of these topics we talked about today.

Nathaniel Nelson
Andrew, that just about does it for your conversation with Kenneth Title. Scott, do you have any final words you would like to take out our episode with today?

Andrew Ginter
Yeah, I mean we’ve we’ve talked about about credibility and this is a concept that is is relevant to sort of the high end of sophisticated attacks, the high end of of consequence. But I’m not sure let me.

Let me try and give a very simple example. I mean I was I was raised in Brooks, Alberta, little town, 10,000 people in the middle of nowhere. Literally an hours drive from any larger population centre. In terms of cyber threats, do let pick. Let’s pick on, I don’t know, the Russian military, does the Russian military have the money to buy three absolute cyber gurus, train them up on water systems, plant them as a sleeper cell in the workforce of the town of Brooks water treatment system. Have them sit on their hands for three years and after three years.

Using the passwords they’ve gained, the trust they’ve gained and the expertise that they have. Have them launch a crippling cyber attack that that damages equipment that takes the water treatment system down for 45 days is that a credible threat? Well, the Russians have the money to do that. It’s, they have the capability to do that.

But you have to ask, why would they bother? I mean, this is a little agricultural community. There’s a little bit of oil and gas, activity. Why would they bother? That does not seem to be it. It. It does not seem to be reasonable to launch that kind of attack against the town of Brooks. It just makes no sense. I don’t see that as a credible threat.

Is that a credible threat for the water treatment system in the city of Washington, DC, home of the Pentagon? I do think that’s a credible threat. So the question of what’s credible is an important question that I see more and more people asking in risk analysis going forward. we have to figure out what’s credible for us, what are what, what, what capabilities do our adversaries have? What kind of assets are we protecting? What kind of defences we have deployed what makes sense, what’s reasonable to believe in terms of the bad guys coming after us. This is an important question going forward and I see lots of people discussing it. I’m I’m, grateful for the the the chance to explore the concept here with with Kenneth.

Nathaniel Nelson
Well, thanks to Kenneth for exploring this with us. And Andrew, as always, thank you for speaking with me.

Andrew Ginter
It’s always a pleasure. Thank you, Nate.

Nathaniel Neson
This has been the Industrial Security Podcast from Waterfall. Thanks to everyone out there listening.

Please note: This transcript was auto-generated and then edited by a person. In the case of any inconsistencies, please refer to the recording as the source.

Nathaniel Nelson
Welcome listeners to the Industrial Security Podcast. My name is Nate Nelson. I’m here with Andrew Ginter, the Vice President of Industrial Security at Waterfall Security Solutions.

He’s going to introduce the subject and guest of our show today. Andrew, how’s going?

Andrew Ginter
I’m very well, thank you, Nate. Our guest today is Brian Derrico. He is the founder of Trident Cyber Partners, and he’s going to be talking about using asset inventory tools. I mean, we’ve had a lot of people on vendors mostly talking about what’s available, how it works.

He’s going to look at the problem from the point of view of the user using these tools and why using these tools turns out to be a little harder than you might expect.

Nathaniel Nelson
Then without further ado, here’s your conversation with Brian Derrico.

Andrew Ginter
Hello Brian, and welcome to the podcast. Before we get started, can I ask you to, say a few words of introduction, tell us a little bit about yourself and about the good work that you’re doing at Trident Cyber Partners.

Brian Derrico
Good morning, Andrew. I’m Brian Derrico. I’ve been in the critical infrastructure sector for about 15 years. i Spent my entire career at a large utility solely focused on the cybersecurity requirements for nuclear power plants.

And my last role there was actually, it was the program manager responsible for the entire cyber program across the fleet. Again, all really dealing with OT type stuff and regulatory requirements.

I left in October, started my own business, tried in cyber partners, and mainly aimed to help other critical infrastructure sectors with their cyber problems.

Andrew Ginter
Thanks for that. And our topic eventually is going to be asset inventory. But, let me ask you, you’ve spent a lot of time working at nuclear.

You’ve worked, in in very old plants in, you’ve done some work recently with a very modern plants. Can you talk about, in terms of automation, what’s the difference between sort of very old automation and very new automation that that you’ve been exposed to?

Brian Derrico
So there’s there’s a lot of similarities, right? At the end of the day, whether it’s a new plant or an old plant, it is still a nuclear power plant. So there is a nuclear reaction that is heating some water. That water is heating some other water in a secondary loop that is flashing to steam, spinning a turbine, making electricity.

So that is nuclear power 101. It doesn’t matter how new or old the plant is. They’ve all generally worked that way for a long, long period of time. To your point, what you do see is the amount of digital assets in those plants is drastically different from new to old.

So in my previous role, I had done some industry benchmarks to try and figure out what is sort of the average number of digital devices that are in a plant. And it came in around 1700 or 1800 per unit.

These new plants that they’re building, they’re an order of magnitude larger than that. There are potentially 10,000 devices on a single unit because everything is digital.

I don’t know how many people have had an opportunity to tour a nuclear plant. I would certainly advise if you have that opportunity is a really, really cool thing to see. And most plants are all analog. There is a lot of analog equipment, a lot of analog indication.

And the new plants, that’s not that case anymore. So trying to keep track of all of your digital devices becomes a very important and critical problem.

For example, in some of the older plants that we worked in, as you’re going through getting asset inventory, you open up the cabinet, you kind of look for what is digital, what are the blinky lights, and you go through and that is generally a manual way that we did a lot of asset inventory.

These newer plants, you open the racks and everything inside is digital. Everything inside could be considered an attack pathway. And there were some discussions and and there’s some thought process out there that essentially calling locations critical.

Is going to be an easier way to do it because saying this entire rack, no matter what’s in it, is going to be a critical digital component is an easier way than trying to label an inventory all 50 or 60 devices. So that was a thought process that was considered.

But again, at the end of the day, every device was considered on a case by case basis. But it kind of gives you an idea of just the scale of how much digital equipment there are in newer plants nowadays.

Nathaniel Nelson
Andrew, I’m glad we’re getting the opportunity to talk about nuclear because it seems like a pretty relevant and highly important field.

And yet it never seems like we get a guest on who wants to talk about it. So where does nuclear stand in the panoply that is industrial security for you?

Andrew Ginter
Well, we’re we’re going to be talking mostly about asset inventory, but let’s talk about nuclear for a while. I mean, Brian said a few words. in a sense, he’s lived a lot of this stuff, without even knowing how unusual it is.

Nuclear is an extreme. When we talk about worst case consequences of of compromise, what’s the worst case, the worst thing that can happen in a a coal-fired power plant? A boiler blows up, people die.

What’s the worst thing that can happen in a nuke? The nuclear core explodes, Chernobyl, and hundreds of square kilometers become unlivable for centuries.

Oh, that’s very bad. So the consequences drive the intensity of your security program, and and nukes are an extreme. I mean, the only thing I can imagine that’s possibly more sensitive than nukes is, I don’t know, nuclear weapons, targeting systems, launch launch protocols. It’s just it’s that extreme.

What does that mean for cybersecurity? Well, let’s start with physical security. In different parts of the world, there’s different rules. In a lot of the world, you need a security clearance to visit the site.

So In North America, you can get tours of the site. But in a lot of places, you you a lot of stuff is classified. I don’t have a security clearance. I’ve never seen network diagrams for a nuclear site. I’m guessing a bunch of this stuff is classified. it It’s national secrets. It’s it’s it’s that intense.

On the cybersecurity side, again, I talk to people, uh, we, we serve nuclear customers at waterfall, And they do things that, seem again, seem extreme.

They might have all of their OT systems in one room, in one building, and all of their IT systems, all their IT servers, email servers and whatnot, they do have IT networks in in nuclear plants. you need to You need to schedule work crews. got to pay your people.

So they have IT and OT networks. And all of the IT servers are in a different room in a different building. Why? Because they cannot afford someone any time, someday to make a mistake and plug a cable from an IT network into an OT asset. That’s completely unacceptable cybersecurity wise.

And so they physically separate it so that as much as possible, they make these kinds of errors impossible. You can’t do it. You can’t plug the wrong cable and it’s in a different building.

Another example, you might imagine that there would be multiple security levels. You might imagine that the technology that controls the core, the control rods into the core that keeps the core from exploding is more sensitive than the the OT systems that control the steam turbines. I mean, a coalpower a coal-fired power plant has steam turbines.

Steam turbines have steam turbines, you imagine. In fact, again, when I talk to these people, a lot of nuclear sites, in my understanding, have only two security levels. Absolutely highest critical and business and nothing in between.

Again, why? Why would the steam turbines be protected to the same degree as the core control system? In part, it’s because, the physics of these systems, the steam, there are… distant physical connections. the liquid from the core heats up the liquid in the steam. And so, there’s theoretically a risk that something happening to the steam turbines could leak back into the core.

But more fundamentally, these people just say we cannot afford to make mistakes with security. And so we’re going to dumb it down. We’re not going to have seven or eight or 13 security levels. And you have to remember which is which and apply the right policies to the right equipment.

It’s going to be absolutely critical, end of story. And which room you’re in. That’s the policy you apply. Again, as much as possible, they eliminate human error.

Regulations. I’m most familiar with the the North American regulations. You might imagine, I mean, NERC SIP handles the power grid. if you If you fail to live up to your obligations under NERC SIP, what happens? You can be fined as much as a million dollars a day.

It’s never been levied, but you get fined. With the nukes, if they fail to live up to their regulations, they’re shut down. They lose their license to operate. that’s it’s It’s that simple. If you cannot operate safely, you cannot operate. Bang, you’re down. So again, intense attention is paid to the detail of cybersecurity and cybersecurity regulations.

Another example. I’m not aware of any nuclear generator. now I might I don’t know all the generators in the world. I’m not aware of any nuclear generator that has any kind of OT remote access, period.

Nothing remotely gets into OT. You want to touch OT, you walk over to the server room. So again, intense. In a sense, though, what I what I what I see of of the nukes is that they are leaders in the cybersecurity field.

They they do things extremely intensely. And as other parts of the field, other power plants, other refineries, other high-consequence sites, as the threat environment continues worsening, as cyberattacks keep getting more sophisticated, they look over at what is nuclear doing, and they pull one after another technique out of the nuclear arsenal, and start applying it in in their in their circumstance. So even if you’re not required to follow the nuclear rules, I would encourage people to read NEI, the Nuclear Energy Institute 08-09 standard, or the NRC Nuclear Regulatory Commission 5.71,

I’d actually recommend NEI 08-09. It’s more readable. It’s got more examples. The NRC 5.71 is sort of more terse and saying, here’s the regulation, follow it. But they are leaders in the space. And over time, I see people drawing on their expertise and and the way they do things.

Andrew Ginter
And our topic is asset inventory. And so, we’re talking about how much automation there is. We’re talking about how hard it is to count. Can we back up a minute?

In principle, the truism is you cannot defend what you don’t know you have.

And so that’s why we do inventory. Is that it or is there more to it? Why are we doing these inventories? What good is an asset inventory?

Brian Derrico
So it’s a great question and I’m going to give two answers, right? So one is on the nuclear space. The first answer is we have to, right? And sometimes that is, it’s an an answer. I don’t think it’s a good one, but it is answer. So we do have regulatory compliance around an asset inventory because to your point, it does sort of fuel other aspects of your cyber programs, such as supply chain, vulnerability management, configuration management, et cetera.

The flip side is it’s just, it’s a smart thing to do, right? You can’t build a vulnerability management program if you don’t know what software is out there that you’re potentially vulnerable to.

So trying to build a vulnerability management program when you don’t know what’s out there is it’s it’s a fool’s errand because you’re never going to be able to understand your total risk.

And that’s really the key is understanding your assets gives you the ability to understand your attack surface. And once you understand your attack surface, you can then figure out what are my vulnerabilities? What do I need to mitigate? What is a possible threat vector an adversary could use to attack this device or this process?

And you can’t do any of that without having the asset inventory first.

This brings us back to our topic. We’re talking about asset inventory. We’re talking about tools. There’s tools out there to do asset inventory. We don’t have to do a manual walk down and count the blinky lights in the cabinets.

Do the tools not solve the problem? is Is there still a problem when you’ve deployed one of these tools?

Brian Derrico
So there are a number of tools that do this and some are better than others right nature of the beast, but they do a great job of asset inventory. So I currently do professional services for a software company and a lot of their deployments in the OT space are generally for people that want to use the tool as their asset inventory.

Now, the issue is sort of becomes a couple of pieces uh that comes up can come up often and I i saw this in nuclear all the time is a lot of those tools that we’re talking about they depend on network traffic right so they’re looking at source and destination and they’re passively trying to piece together these are their assets on your network and this is what they do and how they do it so one problem is going to be you have assets that are not networked so If you have safety critical devices, they may be isolated. So you’re not going to be able to deploy a tool to do that. So you are going to have to manually enter those in and manually keep track of those in some way, shape or form.

And then the second piece is a lot of these tools that we talked about, they can’t just be deployed instantly. You can’t just throw a box in a rack and call it macaroni. There are architectural changes that have to happen to your network. You have to get traffic from switches. You have to open span ports. You have to deploy sensors.

And that’s where things can get a little difficult on the OT side of the house.

Andrew Ginter
So work with me. modern switches, any kind of managed switch has got a span port or a mirror port.

You log into the switch, you turn on mirroring and and off you go. You can start seeing the traffic and a lot of these these asset inventory tools can start figuring out what are the assets based on their traffic.

I get that some systems are are not on the network, the safety systems, that makes sense. But is it is it more complicated than that? I mean, I imagine you’re working with some older systems, older switches, or do any of these plants use non-managed switches?

Brian Derrico
So I’m sure there are some non-managed switches out there. I would not be surprised if there are some hubs that are still out there and kicking.

While in theory, yes, opening up a span port is is a simplistic idea. Where that turns into and where it becomes difficult is a lot of these OT vendors and and even environments that you’re in, nobody wants to change the system without vendors’ involvement, because everybody’s scared about what are the consequences. Because again, this isn’t an IT system, this is an OT system. There could be some huge process changes and huge impacts and risk if whatever you wanna do doesn’t go according to plan.

And that’s where I have seen the most amount of struggle come from is, you wanna get some a span port, you reach out to the vendor, you say, hey, this is what we’re looking to do. We just wanna span this traffic and the vendors don’t wanna budge.

The vendor hasn’t deployed that. They don’t know what that’s going to look like. They tell you that, hey, we’re going to have to refat the entire system after making this change. now Now, meanwhile, is is there going to be an impact?

No. we We can look at switch utilization and see, hey, even if we double, we’ll double the switch utilization. you’re not gonna see a huge impact to that because your switch is only at five or 10% utilization.

But it’s just, it’s there isn’t an understanding on the vendor side. So for some of these big control system vendors, it becomes difficult for them to bless as it were making these changes. And that’s where we have seen the most amount of struggle.

And we even had projects where we had to provide a lot of the testing and we provided, this is what needs to happen because the vendor just didn’t have the knowledge.

And think as time goes on for those control system vendors that are out there, I think that’s gonna be more and more of an issue because more and more of their deployments are gonna have a requirement for some form of higher detection capability, but We can’t just say, these things are they’re in an ot environment they’re safe uh that this’s just this is not the case right there there needs to be higher level of detection and the vendors need to be more willing to work and as time goes on I think it’ll be easier but retrofitting this sort of technology in existing systems becomes increasingly difficult because nobody wants to touch the system that isn’t broke

Andrew Ginter
So A couple of quick points there. Brian used a couple of of acronyms people might not recognize. He said you might have to refat the entire system. What’s that? Fat is factory acceptance test.

It’s set everything up and test every function of the system. Emergency recovery, every function of the system and make sure that it meets the requirements that were laid out when you you issued the contract to get the system built.

Typically takes days. You have to shut the plant down to do it. So nobody wants to refat anything. So that’s that’s what the vendors are threatening, saying, well, if you make a change that we haven’t tested, we have to retest it, don’t we?

Another point he made was about, uh, bandwidth and, for anyone who, who’s not real familiar with how mirror or span ports work, you got a switch with, I don’t know, 24 ports on it, 48 ports.

It has to be a managed switch. You log into the switch with a username and password and you can configure the switch. And one of the things you can configure is it’s called a mirror port or a span port. Um,

It’s a port or, multiple ports where you send copies of stuff. So typically, if you’re going to do an asset inventory, you configure one port and say every message that anybody sends to anybody else on the system, send a copy of the message out this port.

And now… The asset inventory system can look at the messages and say, oh, there’s IP addresses in use. I wonder what kind of machine this is. It’s using this TCP port number, and it figures out what kind of stuff is on the network based on the network traffic. And the mirror port gives you that traffic.

And the throughput consideration is, I thought, and now I’m not an expert on switches, I assume that modern switches, you would put, they they have ports, 24 ports out the front, and every message that comes in goes onto to a backplane. It’s a very high-speed backplane.

And I thought that the message went to every one of the other ports, and the ports decided, do I send this out or not? And so it would go to the mirror port as well. That’s what I assumed. And so, turning on the mirror port would not, in fact, increase the, you the amount of traffic on the backplane because every message is visible to every port.

But what I didn’t get clarification from from Brian, but what it sounds like is at least some of the switches he’s dealing with, if you enable the mirror port, then the source. if If port A is sending a message to port B, it first puts on on on the backplane address to port B, and a second time puts the same message on the backplane address to the mirror port, because it’s been configured to send everything to the mirror port. And that would tend to double the amount of traffic on the backplane.

But these backplanes are massively high speed because they have to support all of the 24 ports simultaneously. So he’s saying, look, your average backplane is barely loaded and doubling the load is immaterial.

What he did not say was that configuring the switch causes the switch to malfunction. I would imagine ancient switches that were connected were around sort of at the beginning of the concept of mirror ports and and span ports might have defects in their software that if you turn on the mirror port, it might malfunction. But, he didn’t say that. I forgot to ask him. And the fact that he didn’t say it says to me he’s never run into it or, he would have mentioned it. So that’s I’m putting words in his mouth there, but I’m guessing that’s not so much a concern. The concern is throughput. The concern is testing. That’s just, people worry about

Things working the way they’re supposed to if you make a change that has not been anticipated. This is the essence of the engineering change control discipline that is, again, used intensely at at nuclear sites and used, but maybe just a little less intensely at at other critical infrastructure sites. Pause. Pause.

Andrew Ginter
So work with me. In the modern day, you’re saying, the control system vendors don’t get asset inventory. I mean, span ports, mirror ports, they’re also used for intrusion detection systems.

This is what Dragos uses. This is what Nozomi uses. the six pillars of the cybersecurity framework, the NIST framework, include detect, respond, recover. You’ve got to be able to look at what’s happening on the hosts. You’ve got to be able to look at what’s happening on the networks.

Really, the the vendors in the modern day don’t get this.

Brian Derrico
And I credit where it’s due, some do get it better than others.

However, there have been some vendors we’ve worked with that did not want to make any changes because they just wanted to give us the same system that they gave us 20 years ago. with one version, higher than than what we deployed, again, decades in the past.

And, when pressed, while the people on the vendor side are experts in what they are doing, they are experts in safety design, they are experts in PLCs and how all of these things talk together.

They’re not IT people. So when you start talking, hey, I want to open up a span port, it’s different. They don’t understand. They think it’s going to cause an impact to the system. Meanwhile, as people with an IT t background, we can see that, hey, you’re using managed switches. you can enable a span port.

The inputs are 100 meg. And, even if if all of your PLCs are, completely maxing that throughput the back plane of the switch is going to be nowhere near utilization and even doubling that you’re not going to see a decrease and it just it takes a long time to get the vendors on board and again we even offered to to do some testing and show what the utilization changes were

And, we have seen that again with some vendors are better than others. But, I feel like at the end of the day, it’s we just want to give you the same system that you’ve already had. And making changes to that is scary.

And, we’re an isolated system. So, we don’t need to deploy a lot of that technology because we’re just going to stay isolated and and not connected to anything. And the reality is that isn’t as effective either because you While you lose the sort of network attack path, you still have several others, such as physical supply chain and portable media.

So having detection capability is actually, in my opinion, it’s worth the risk of plugging that thing in as long as you have a sound architecture. And that’s where some of the struggles begin with changing sort of that mindset from on the vendor side.

So for example, some of the control system vendors that there’s workstations and stuff there, they understand that, yes, there are detection pieces. You’re going to deploy some level of network intrusion detection.

You’re going to deploy some level of SIEM agent, right?

So I need to send Syslog and we’ve had good luck, and again, with particular vendors there. Some vendors will actually included with their control system, they will also include a security suite.

So they will have their own HIDs, their NIDS, their SIEM, and that’s all included. They have a patching server that distributes Microsoft Quick Fixes and all that stuff. It’s great.

However, when you get to that lower level of your PLC type stuff where, again, we were working with a PLC vendor and they would not budge. They did not want to change their design.

They thought that the switch, there would be a loss in time of communication, which would affect the safety related aspect of the design, and they did not want to budge.

And it took two years for us to to work with them for them to understand that we have requirements and when the programs were implemented specifically across nuclear it was understood that you’re not going to go in and bolt this stuff onto existing systems but when you’re starting fresh when you’re building a system from the ground up it has to have all of these components there is no longer an excuse to say, oh, it’s and <unk> already working. we’re not going to go play around with it. It’s going to that obviously cause issues.

Everything has to be baked in from the ground up. The cybersecurity piece has to be foundational. And again, with the PLC vendors, we found it to be, again, one particular vendor, very difficult.

For us to get that through and it took a number of people, trying to work their, the PLC engineers through why this is, we promise here, here’s some data to back it up.

And they finally did agree to to use the architecture that that we were, we had kind of specified from a design perspective.

Andrew Ginter
So we we sweat blood, we fight with the vendors, we get our asset inventory system deployed, we augment it with with manual inventory for the air-gapped or the isolated networks, and we use it for managing patches and vulnerabilities.

Is there anything else we use it for?

Brian Derrico
Absolutely. To your point, Vulnerability management’s a big one, right? Because I think at the end of the day, your asset inventory is going to give you what your what your risk profile is, what your attack surface is.

Vulnerabilities is one part of that. There is another piece of it that is supply chain, right? So we talked about that a little earlier, being able to understand what are the important devices that I am going to produce procure and procure those with certain sets of requirements. That’s also critical.

Another thing that we would use it for is configuration management. So understanding what is your configuration. You can build tools, you can use tools. That tell you this is the configuration on the device.

And some of those tools out there, some of those network intrusion systems that are OT-centric can also give you alerts and understandings on what is when changes happen. You have a code download to a PLC.

Is that expected? And then also, this is the running code of that PLC, and this is what changed, and you would have visibility into all of that. And again, all based on your asset inventory and having as much information as you can about those assets.

Andrew Ginter
And if we could sort of bring it into the modern world, the, the latest automation systems have a lot of devices and asset inventory counts them. This is great.

But there’s a lot more we need to do with the information. So you’ve talked about patching. There’s a lot of We’ve had people on the show talking about SBOM, software bill of materials, keeping track of sort of embedded software when vulnerabilities are announced.

Is there automation for tracking SBOMs and vulnerabilities and doing the mechanics of patching and patching? Arguably, counting the asset is is the easiest part of managing the inventory.

Is there more in sort of that we can expect of modern tools?

Brian Derrico
I think there is. And, vulnerability management is always going to be one of the most difficult things to conquer because if you don’t have an updated software inventory, you’re never going to know what’s out there. You can do all the Windows patches in the world, but, there are obviously tens and tens of thousands of non-Windows vulnerabilities where if you’re running again, insert whatever software product, right? There are huge vulnerabilities around a lot of those. So can you automate it?

I think it comes down to you can automate the visibility. Right So you can at least understand and have up-to-date dashboards of this these are the devices that you need to worry about. Right This particular device has five critical vulnerabilities. And then that gives your your internal cyber engineers something to go after to mitigate to overall reduce that risk.

I also think it’s important from a business perspective to understand what are we going to do, right? On the IT t side, there’s a lot of patching processes and there’s, SLAs associated with is your, is the vulnerability critical, high, medium, low, et cetera.

On the OT side in general, OT is very adverse to patching and mitigation. And I agree with that in some senses, and I don’t agree with that in other senses. And I think as a business, you guys like you need to understand what is your tolerance for that risk? What are you willing to accept?

And are there areas where, yes, we we’re comfortable, we’re not patching because we have all these controls in place. And in order to get to the device, there’s guns, gates, and guards in the middle of it.

But, but hey, maybe if something really, really, really big comes out, we are going to take care of it. And We do have to come up. So I I don’t think there is a way to fully automate it, but you can at least automate the visibility.

So you don’t have people, just manually searching NVD with a software list that they don’t even know is accurate. You can get that part out of the way. There are tools out there that will help you. And then becomes a business decision and sort of a business process around, with all that information, here is your overall risk profile. What are you going to do about it?

And that that becomes the deeper discussion, again, around what specifically the business is, how much risk tolerance you do have, how much risk avoidance you want to have, and kind of go from there.

Andrew Ginter
Well, Brian, thank you so much for joining us today. Before I let you go, can I ask you, can you sum up for our listeners? What should we take away in in terms of what we’re doing with asset inventory?

Brian Derrico
Absolutely. I would say asset inventory is the most important part of your program, because if you don’t know what assets are out there, you’re never going to be able to protect your organization from somebody that maybe they know what’s out there and you don’t.

So asset inventory is critical. You cannot build upon your internal program without understanding what your attack surface is. I think another point is there are tools to help you.

This is not something that we need to do manually anymore. You do not have to go into cabinets and count every single blinky light. There are tools and you know products out there that will help us get closer to where we want to be.

And then at the end of the day, you still need an internal team that understands what the information coming back is. So if if you you know if you do need help in deploying these tools or selecting tools or understanding what the risk is, I’d be happy to help.

You can connect with me on LinkedIn. Brian Derrico, think I’m the only one. And I can help you with those problems because, again, once we once we conquer assets and get the tools in place, a lot of pieces of the program become a lot easier.

And my goal and what I love is just driving efficiency. So let’s automate, automate, automate, use tools to kind of help us see what we can and just do what we can to protect critical infrastructure.

Nathaniel Nelson
Andrew, that just about concludes your interview with Brian. Do you have any final thoughts about what he talked about there that you can leave our listeners with?

Andrew Ginter
I mean, I think what I took away from here is is, the importance of inventory and the need for automation. I mean, if a modern nuclear generator has, 10,000 plus devices in it that have CPUs in them that have to be managed, that have software that have to be managed, then you know I don’t know that a nuclear generator is that much more heavily instrumented than the average industrial thing. If you buy a steam turbine, it’s a modern turbine is heavily instrumented. If you buy any kind of physical equipment, it’s going to be heavily instrumented. This is you know There’s plus CPUs in a modern automobile.

And that’s, that’s something that fits in your living room. We’re talking about massive installations. I would imagine that a big refinery has as many as 100,000 plus devices if it’s been upgraded recently.

When was the last time you tried to manage a spreadsheet with 10,000 rows in it? When the last time you tried to manage a spreadsheet with 100,000 in it? Just manually counting the blinking lights takes a long time.

Automation to me is is essential. I mean, this is, you look at the NIST cybersecurity framework, sort of the grand compendium of everything that is cyber. What’s the first thing you do? Well, the first thing you do is figure out who’s responsible for the program and you know assign budget and responsibility.

What’s the second thing you do? You take asset inventory. You got to understand what you’re protecting. So, this this all makes sense that you need the inventory and in the modern world, you need automation. There’s no way you can do this anymore manually. So, my thanks to to Brian Derrico and, learn something here.

Nathaniel Nelson
Yes, our thanks to Brian and Andrew, as always, thank you for speaking with me.

It’s always a pleasure. Thank you, Nate.

Nathaniel Nelson
This has been the Industrial Security Podcast from Waterfall. Thanks to everyone out there listening.