I’m noticing a disturbing trend of late:
Some end-users are actively trying to impose security from outside staff upon operations. In fact, some vendors are suggesting that this is a good thing to do.
Sadly, imposing security on others is a doomed effort. They’re going to fail badly because they’re not thinking ahead of the problem. The reasons why they will fail are not hard for anyone to predict.
1. They do not involve the operators.
2. They do not know the operation.
3. They’re treating the technology instead of the whole process.
4. They’re more interested in detecting and reacting to the problem than preventing it.
I’ve been ranting for years that companies can’t bolt security on to an existing system. One might as well defend a straw house with a large safe door. All the residents have to go to great lengths to enter that safe door. Meanwhile the walls are still made of straw. An attacker doesn’t even have to go through the door. They only need a match to burn it down. To make matters worse, the security people want to put smoke detectors inside the straw house. That doesn’t prevent the house from burning down. The only thing that will really make a house more secure is to build it out of something that can’t easily be destroyed.
Unfortunately, there are no IT gods who can look over the shoulders of operators and magically know what they need to know to securify the operation. The solution is something that most managers do not want to hear.
The company needs to train the operators and technicians to do better OT. They need to know more than “if the green light is on, the network is good.” Devices and networks can no longer remain “clouds” of information. The traditional IT practice of compartmentalizing information is toxic in this application.
Start with monitoring network port activity. If a port lights up or shuts down unexpectedly, then an operator should go investigate, preferably while armed with a big pipe wrench. If network activity levels suddenly increase and nobody knows who might be working on the network (or even if they do) confirm where it’s coming from and why it’s happening.
If you’re thinking of using wireless, find something that can monitor the media and make sure that everyone is trained to recognize when extraneous, unexpected signals are present.
And this will also require significant design and testing updates to the SCADA/Control Systems. They should include monitoring firmware versions, Error rates, traffic levels, port status, PLC scan times, arp tables, and many more things. Operators need to understand at least some of those features, and know enough to be able to call someone to help.
Why do this?
Because control systems and SCADA system activity can change frequently depending on who is doing what work where. There is no way a security division can keep up with the goings on inside every plant, transmission, or distribution system. But operators are supposed to know this. It’s their responsibility, their licenses, and their reputation at stake, so they’d better know.
While appliances can assist knowledgeable people to secure their operation, that knowledge has to be there first.
Furthermore, if security gets in the way of operations, guess who wins? Hint: It’s not security. Again, there is no way an external securification can ever work well. And don’t even think of an appliance approach to security. Appliances don’t secure anything. While appliances can assist knowledgeable people to secure their operation, that knowledge has to be there first. Also, the idea that a centralized security Network Operations Center can oversee all the goings on, and know that it’s time to descend upon a plant to save the hapless operators from themselves is silly.
Designing more secure ICS/SCADA systems requires built-in self integrity in to every element of these systems. I’ll have more to say about some creative ideas to address some of these problems later.
There is another aspect to many ICS/SCADA security issues: They don’t look like security issues at first. Many look like bugs, poor scripting, broken instrumentation, or some other routine problem. It can take a long time to exhaust all the possibilities before the problem might be deemed worthy of a security investigation.
My advice to any company thinking of building a security division to impose security on their operations: Don’t. This is not one of those problems that one can fix with remote control office IT policies. The operations staff will revolt. And the C-Level people will be faced with a big bill and failure as far as the eye can see.
