Among the tribes of engineers, there are certain things we just have to learn by doing. One of them is PLC programming. Somehow, we engineers are expected to emerge from college knowing good practices for programming a PLC. Some of us older engineers learned to program using FORTRAN. If we were lucky, we learned about structured programming. The millennial engineers may have had the benefit of learning about object oriented programming. Maybe it was a class in C++. But data structures were something that they were just “supposed to know.” And engineering educations today? If we’re lucky, they’ll see a course in how to sling code in Python. That’s what my son did when he was studying in a pre-engineering course.
My point is that most engineers discover good programming practices the hard way. We learn on the job. I stumbled across this many years ago. I started collecting tips, tricks, and experiences from my colleagues on what we like to do when programming a PLC. Some of these ideas became obsolete when the PLC finally became obsolete. For example, though it was a highly regarded PLC in its day, I don’t think anyone still cares about programming tips for an Allen-Bradley PLC-2.
Meanwhile, when learning “How To Program A PLC” in one of those vendor classes, they just presume you know what to do with each of the four IEC 61131 programming languages in a PLC, what each of the commands are good for, how to use a set and a reset coil safely, and so on. They give background on using their products, but not a whole lot of experience on how it might be done better.
A few years ago I met Isiah Jones while working at Jacobs. He shared a paper he co-wrote with another engineer on the subject of tips that they had collected on PLC programming. The outline in the papers covered some fundamentals that I hadn’t considered, but didn’t get too deep in to the practicalities of how one actually programs. We both realized that this was a fairly large gap and that it was probably causing more trouble for the ICS security business than anyone realized.
So I scraped a few ideas from his paper and combined them with my notes and made a presentation at the S4x20 conference in January of 2020, just before the pandemic broke out. I didn’t have any idea how the presentation would be received. It came as a big surprise when Dale Peterson noticed it and mentioned it at the tail end of the conference as something we should pursue later.
As with many people during the beginning of the pandemic, my employment status was turbulent. I was busy staying afloat financially while trying to pay for two children in college. The TOP20 effort stagnated. I am not a great organizer. I think of things, write them up and then hope that people might find some use in it. But Sarah Fluchs of Admeritia, GmbH is a very capable organizer. She documented my scattered notes from the presentation. And then we had to weed out the stuff that wasn’t specifically PLC related. For example, I had a lot of notes regarding I/O design that aren’t strictly related to PLC programming.
You may be wondering; I usually discuss security. Why concern yourself with PLC programming? Most OT specialists stop at the PLC. It’s an embedded creature and it does stuff very closely related to the process. So the OT specialists let the engineers do whatever it is that they do. Unfortunately, as I pointed out above, engineers often don’t know a whole lot about secure programming practices.
While these tips won’t protect you from the truly nefarious exploits, at least they won’t leave you with comically trivial vulnerabilities. Take the recent intrusion into the City of Oldsmar Florida water utility SCADA system: Setting the Sodium Hydroxide dosing rate to a ridiculously high value of 11,100 ppm shouldn’t have been possible. The PLC should have rejected that value. But it didn’t. Why didn’t the PLC just ignore the information it received from the HMI? It wasn’t configured or programmed to do so. The features in the PLC programming language exist to do this. Yet, it probably didn’t occur to the integrators or engineers that someone would deliberately set things that way.
These programming tools and tips are not things that you will find in most control system narrative documents. The people designing the process are focused on how to make the process work and handle a few very basic failure modes. They do not get in to the gritty details of how the PLC should do what it does and what it should do if there are indications of integrity failure. They may not even get in to details of how to handle a situation where network segment connections need to be broken to handle a security problem.
Thus the Top20 list was created to help engineers by showing some of these practices and helping people make the most of the various features a PLC has. Since July of 2020, with backing from the ISA and Dale Peterson, Sarah Fluchs assembled an interesting group of people (among the many, I would like to give a shout out to Vivek Ponnada for the exemplary work he did) who have reviewed and added to the list of practices. The document is much better than the scattered notes that I had. It is our hope that vendors, integration firms, and engineering firms will see this list and start adding to it and perhaps add training around it.
And now that we have the PLC programming practices ball rolling, let me reiterate that many ideas were discarded because they relate to process and I/O design. That will probably be a subject of multiple future efforts.
Again, my thanks to Sarah Fluchs and to all the contributors of this effort. I hope this will be a living document and a cause for discussion about how we train the engineers to do this work. It is far more than I ever dreamed it would be.
