IT and OT networks are under continuing attacks with varying degrees of impacts. When the DHS CISA Alert was issued specifically identifying control systems, I had two questions: why now and what happened that was unique to control systems? For control system cyber security what is most important are the physical impacts from the control system incidents. The attached identifies actual control system cyber security incidents that can affect multiple facilities in multiple industries yet there has been no disclosure nor guidance offered (https://www.controlglobal.com/blogs/unfettered/information-sharing-on-control-system-cyber-incidents-is-not-working-and-that-can-be-deadly). The Alert would have been an ideal vehicle to have addressed these and other control system cases. Because the Alert did not address control systems, I went to colleagues in senior positions in industry, academia, and media asking if they would support me on a blog on the shortcomings of the Alert. They have. I also needed independent validation of my concerns about the Alert. The first was Executive Order 13920 as it was only about the control systems and hardware. However, as mentioned, the Alert didn’t address the control systems or reference the Executive Order. The second was an unclassified presentation by Dr. Kamal Jabbour, a senior information security scientist at the Air Force Research Laboratory (AFRL), who challenged CISA and the community to go beyond the corrective actions provided in the CISA Alert.
Response to the Alert
Federal agencies including DoD, DoE, and DHS, along with commercial critical infrastructure suppliers and owners have made significant progress in improving the cyber security and resilience of IT and OT networks by making them a priority. However, DHS CISA, like many organizations, falls short in addressing the facility hardware and control system devices. What makes this gap critical is that real physical damage, operational failures, and loss of life are often the result from compromised control system devices. Because of the plethora of ransomware and other network attacks, there is a hypersensitivity to network security resulting in control system security not being adequately addressed (or excluded). The reasons for this are almost mutually exclusive. While the fields of IT Security and analysis have matured tremendously in a highly competitive and high demand market, the pace has been much slower in securing a much smaller control device market which until the past decade were considered secure and “air gapped” from vulnerable IT systems. Network cyber security (whether IT or OT) has cyber forensics and cyber security training for network security personnel while adversaries have closely studied the vulnerabilities and gaps in logistics and supply chain management to identify new vulnerabilities to exploit. Consequently, IT security advances in network security now provide rapid detection and isolation techniques where anomalies and malicious code issues can be identified and mitigated. Unfortunately, control system cyber security has neither cyber forensics nor adequate cyber security training and education for the engineers. Consequently, control system cyber incidents are rarely fully analyzed and therefore not confirmed or identified as being cyber-related. The July 23, 2020, NSA and DHS CISA Alert AA20-205A: Recommend Immediate Actions to Reduce Exposure Across Operational Technologies and Control Systems shares this oversight. The security defenders tend to overlook control system security, while several national state “bad actors” have demonstrated a willingness to continue to exploit U.S. weakness by implementing hardware backdoors in U.S critical infrastructure including large electric transformers and essential manufacturing equipment.
Let’s explore areas where control system security is being overlooked in 2020 and what needs to be done to reverse these trends.
May 1st, 2020, Presidential Executive Order (EO) 13290 was issued, most likely the result of the discovery of implanted hardware devices creating backdoors into large electric transformers, thus bypassing all cyber network security mitigation. This should have been a rude and necessary wakeup call for industry and all federal agencies with infrastructure cybersecurity responsibilities. This clever tactic of using a transformer as a Trojan Horse and bypassing all internal utility networks is essentially the Chinese doing a reverse Stuxnet on critical US grid equipment. But despite this blatant warning the industry and some in our government remain fixated on network security. While an appropriate measure would be to hold bad actors accountable and impose cost for what amounts to an act of aggression, the response appears to be let’s continue to increase our network security. The myriad of webinars which have come out since the EO was issued either focused on “rip and replace” or network issues. The issue with “rip and replace” is replace with what? The network issues were explicitly excluded from the EO (see the detailed list of equipment addressed in the EO). But EEI, NERC, the recent CISA Alert, and majority of various industries continue to focus on network issues, because that is their comfort zone and area of expertise. May 22, 2020, DHS, DOE, and UK’s NCSC Issue Guidance on Protecting Industrial Control Systems (https://us-cert.cisa.gov/ncas/current-activity/2020/05/22/cisa-doe-and-uks-ncsc-issue-guidance-protecting-industrial-control). The document was issued three weeks after EO 13920. Yet the document was a network-only document making no mention of any hardware issues. Moreover, the leading recommendation about expeditiously patching known vulnerabilities is applicable to some control systems but certainly not to all control systems.
The SANS 20 Security Controls were developed in 2009 to promote continuous monitoring and increase network awareness. June 2015 DHS issued their May/June ICS-Monitor article – “If You’re Connected, Your Likely Infected!” (ICS-CERT_Monitor_May-Jun2015.pdf). OT networks should not be connected to IT networks and control system devices should never be connected directly to the Internet. So, what precipitated the July 23nd CISA Alert? What’s new is the discovery of hardware backdoors found in equipment from overseas including transformers, pharmaceutical equipment, etc. Yet, that wasn’t addressed in the Alert. As an example, neither the Alert nor the May 22nd guidance addressed the April 2019 spoofing tests against phasor measurement units (PMUs). The testing (https://gpspatrons.com/power-grid-spoofing/) demonstrated the vulnerability of PMUs to cyberattacks where a generator trip in an automatic control scheme could be falsely activated by the GPS spoofing possibly leading to cascading faults and a large-scale power blackout. These are engineering issues that require engineering participation.
FERC and NERC published a cybersecurity guide for the power sector which covers four noninvasive techniques that security professionals can use to identify chips produced by Huawei and ZTE (Joint Staff White Paper on Supply Chain Vendor Identification – Noninvasive Network Interface Controller issued July 31, 2020). The hardware backdoors (implants) in the large transformers that likely precipitated Executive Order 13920 have the same issues and potentially have the Huawei and ZTE chips. These are issues beyond just the electric sector.
Friday August 7, 2020, I attended an unclassified webinar by Dr. Kamal Jabbour, a senior research scientist from the Air Force Research Laboratory, on the CISA Alert. I found the presentation outstanding and found his insight into how we should be dealing with hardware and firmware aggression and exploitation by adversaries and bad actors of great interest. Dr. Jabbour’s presentation highlighted both successes and shortfalls in in the CISA Alert. Dr. Jabbour stated the proposed network hardening targets the lower layers of the Internet stack and has no effect on the upper layers where the attacks occur and provided his audience a proposal for addressing the Alert shortfalls.
The Alert utilized the MITRE ATT&CK methodology. The MITRE ATT&CK methodology is a community-sourced framework for identifying malicious threat behaviors, specifically the Tactics, Techniques, and Procedures (TTPs). The MITRE ATT&CK knowledge base can be used to better characterize and describe post-compromise adversary behavior. However, the ATT&CK tool has two flaws that can be addressed significantly improving the methodology.
The first is the ATT&CK methodology is based on network, not control system devices so the TTPs are network-focused. Control systems are different from IT/OT networks as unintentional issues are also important. Moreover, a sophisticated attacker can make a cyberattack look like a system malfunction. Additionally, control system cyberattacks can be at the equipment level which cannot be detected at the network layer. An example is the Aurora vulnerability. This means that TTPs for control systems are different than those for networks and need to be developed.
The second is with how the MITRE ATT&CK methodology deals with process sensor issues. Under the Modify Control Logic section: “An adversary can de-calibrate a sensor by removing functions in control logic that account for sensor error. This can be used to change a control process without actually spoofing command messages to a controller or device.” Even though MITRE’s statement that the sensors can be decalibrated by removing functions in the control logic is not accurate, MITRE’s statement that “untrusted” sensor readings can be used to compromise equipment is correct. In fact, compromising the Ethernet network to change apparent sensor readings on the operator displays was the tradecraft used in Stuxnet. It was also a factor in the Triton attack in Saudi Arabia so the operators would not inadvertently foil the attacks. Additionally, sending “compromised” sensor values may become the next preferred attack vector of adversaries using hardware backdoors installed in the large electric transformers to cause damage and/or impact grid reliability at the time and place of their choosing.
The primary source of data for the MITRE ATT&CK framework comes from publicly available cyber incident reports. Dr, Jabbour stated it is like the streetlight effect- you see where you look. This reinforces my experience with the ATT&CK tool. When I talked to the MITRE project manager, he had very few actual control system cyber incidents that affected control system devices and was looking for real cases. My database has more than 1,250 actual incidents with more than 1,500 deaths and more than $70 Billion in direct damages. I would encourage CISA to review these incidents to understand if we are addressing the appropriate threats.
The CISA Alert states that since the Ukraine cyber attack of 2015 organizations must assume in their planning of not only a malfunctioning or inoperative control system, but a control system that is actively acting contrary to the safe and reliable operation of the process. Consequently, organizations need an OT resilience plan. Dr. Jabbour again challenged his audience to make better use of engineering tools such as state estimation, stability analysis, and load synchronization that no operator can perform manually, which wasn’t addressed by CISA or DOE. Manual operation requires engineering participation, yet engineers haven’t been trained about cybersecurity. How will manual operation impact cyber mitigation? From my experience, many of the most impactful control system cyber incidents were not IP network-related. How will the table-top exercises reflect these actual cases? Any control system cyber response/resiliency plan needs engineering participation.
On a specific ICS project, AFRL indicated they had hired an independent assessment team to perform systems analysis, security architecture review, source code analysis, and hardware analysis in order to analyze the security posture of the design and implementation of the solution and evaluate the prototype system based on its operational goals. The prototype was evaluated against the assertion that it is intended to offer a secure, alternative data transmission path with improved confidentiality, integrity, and availability (CIA) over the legacy system. That same hardware analysis and approach used by AFRL would significantly improve the value of DOE and DHS testing and alerts.
It should be evident that control system cybersecurity is essential and needs to be addressed by both government and industry. It is necessary if we are to create and maintain secure and resilient critical infrastructures and protect and defend our citizens’ way of life in the face of adversaries who are more than willing to challenge American values.
