(cross-posted from the SCADASEC mailing list – posting was reply from Marina Krotofil)
I just went through the Dragos report, the Wired
article, the Ukrainian sources, Nozomi report, some more foreign articles,
Medium articles, Dale Peterson opinion piece, CERT-UA page, spoke to some
ICS folks who already looked into the sample, and I have so many questions.
1. Firstly, there is a mismatch in data of affected households and duration
of disruption. According to official Ukrainian sources, 324 Individual
Heating Units (IHU) were affected. The number 600 appears to come from the
SBU’s spokesperson who kindly replied to a Ukrainian journalist. However,
SBU mentioned 600 households and the report mentions 600 apartment
buildings. Given that one apartment building requires at least 1 or more
IHU, the number of affected buildings is lower than 600. Secondly, the heat
supply was restored in 6hrs to 50% and 13hrs to 100% (not 48 hrs as stated
in the report). It took slightly longer to restore the hot water supply
though – possibly due to longer restoration processes. It is stated that
Individual Heating Units (IHU) “stopped functioning” and basically
disconnected from the centralized heat supply and the maintenance personnel
re-started IHUs manually (by traveling to them) and with that restored
heating in individual buildings. This brings me to the next point.
In their interview with Wired, it was stated that the attackers “changed
temperature outputs to turn off the flow of hot water” and “malware altered
temperature readings to trick control systems into cooling the hot water
running through the building pipes”. These statements do not correspond to
what is written in the official statements on heating outage recovery.
Also, in the case of IHU usage like in Lviv, the IHU unit on the premises
regulates the final temperature to be sent through the building. The hot
water from the central supply comes with heat losses and therefore is no
longer hot. The alleged temperature value manipulation seems not to be
aligned with the hearing infrastructures in Ukraine. The logging
devices/controllers at the central heating station indeed measure the
temperature, pressure and volume of the supplied water but the attackers
would not be able to trick the heating system into cooling the water
neither centrally (they would need to hack much deeper for that) nor
remotely, at the individual building (they would need to hack into
individual IHU systems).
There is another statement in the Wired article that the FrostyGoop malware
was used to “turn off the flow of hot water”. I am a bit confused about
which of the statements is accurate: the attacker cooling the hot water or
switching off the supply of hot water. However, it appears that the ENCO
devices at LvivTeploEnergo are only used for taking measurements.
2. Firstly, the sample is indeed freely available on Virus Total. Many
other companies and threat hunters identified it and analyzed it. It is not
a ready-to-be-used malware but a generic modbus client with very few
generic capabilities. Dale wrote about the same. The accompanying json
configuration file contained a hardcoded IP address in Romania. There is no
indication that this or similar samples/modbus client was used in the
attack in Ukraine. In fact, there is no reliable information that exactly
the ENCO devices were affected in the attack. But theoretically they could
have been. One can find ENCO devices being exposed in Ukraine (and many
other countries) and one can find through the tender information that
LvivTeploEnergo uses these devices. However, according to the tender
documentation, these devices are meant to be used for measuring/monitoring.
If a device like ENCO is DoSed, the central heat supply company can no
longer conduct the billing and therefore the services must be stopped. This
is a normal practice in Ukraine. And in such a case, the remote IHU must be
re-connected manually. Again, there is no reliable information why 324 IHU
went into a fault/stand-by mode. Given how quickly the large number of
these units were restarted and re-connected indicates that nothing
damaging/unusual has happened. 175 remote units were restarted manually
between 2 am and 9.30 am.
3. I am very curious about Dragos’ collaboration with the SBU. Being a
National Secrete Services agency, they cannot easily collaborate with
foreign nationals. When I need to speak to them, there is a need to obtain
permission because I am no longer a Ukrainian citizen but a German citizen.
Also, every piece of information gathered by this organization is by
default a “state secret”. Sharing information even between different
national agencies requires having established and approved processes. I am
wondering how these processes were bypassed in the case of this incident.
Information about Industroyer 3 (2022 version) was classified (what exactly
has happened) and if SBU was involved in this incident, the data would
become inaccessible to anybody inside the country or outside, unless there
is a process for exchange. I contacted a few of my Ukrainian fellas who
work closely with the mentioned in the report situational center, but I
received two similar unsatisfactory responses: nobody will provide such
information to them. They will be asked why they need the information and
without absolutely valid reason and approval, the answer will be “no”.
Also, it would be a legitimate question, why SBU decided to discuss the
incident with nobody else within the country or outside, except Dragos. Was
the communication conducted via the official spokesperson? By the way, the
SBU’s spokesperson also mentioned that “The consequences of the cyberattack
were quickly neutralized, and services were restored. The company continued
to work as usual.” This statement is consistent with what was mentioned in
the official updates by the city of Lviv. Given how quickly services were
restored and life went on as usual, it is unclear what was the approach to
incident response and forensic evidence collection (if any).
4. It is unfortunate that Dragos report does not provide information on who
conducted incident response and forensic analysis. Given the quick
restoration, it seems like the evidence was not really collected. The
information about firmware downgrade also raises questions. It was stated
that “The adversaries downgraded the firmware on the controllers, deploying
a version that lacks monitoring capabilities employed at the victim
facility, resulting in the Loss of View. The adversaries caused the
controllers to report inaccurate measurements, resulting in the incorrect
operation of the system and the loss of heating to customers.” I am puzzled
about what firmware monitoring capabilities on the controller are
mentioned. Is there such? How did the attacker upgrade the firmware? Not
via Modbus! I mean there are function codes that allow this like Modicon
PLCs do). However, these function codes were not implemented in the
discovered sample (the sample has FC 3,6 and 16). Also, it did not make
sense to do so. Why would the attacker need to change any values on the
controller if the controller spoke simple Modbus and the readings could be
changed directly within the traffic? However, again, given that the
controllers were restarted, all the evidence was gone. So I am struggling
to understand how the alleged attack flow was reconstructed. The whole
incident response part seems to be disjoined. Given that the Dragos team
simply found a random sample of a Modbus client on VirusTotal, I am
struggling to see how the report progressed to the information on page 6,
especially given that no architecture drawings are provided.
5. Few miscellaneous points. The email is getting long and I am sure
missing few points to address. To wrap up. The provided Modbus monitoring
rules have been around since about 2004. In this regard, Nozomi provided a
more comprehensive and modern overview of how activities of this sample can
be detected. The sample was uploaded to VirusTotal in October 2023, but was
“discovered” by Dragos in April 2024, another OT company discovered it at
about the same or a bit earlier as part of the regular but in general
random VirusTotal threat hunting. I am genuinely curious about why nobody
discovered this sample earlier. And why was this reported just now? Not
really important but curious nevertheless 🙂 The other OT company who
discovered the sample simply treated it as a “modbus capable script” being
submitted to VT.
To conclude, the report raises more questions than provides answers. It is
hard to tie the discovered VT sample to what has happened in Ukraine. In
fact, the sample does not have any direct relation to the Ukrainian
incident. The discovered sample is also not a ready-to-use malware but
rather a “starting point” of something potentially more interesting. There
is a probability that ENCO data controllers were indeed affected, however,
likely in a much more simplistic way. Given quick restoration, nothing
extraordinary has happened which would prompt the need for engineering
investigation. The heating system was deemed operable and could be directly
restored to normal. At least this is what public information of Lviv city
tells us. And of course, I am also speculating as there are no detailed
reliable sources of information about the incident.
Some of the sources I reviewed are listed below. By the way, some of this
year info on threat actors targeting industrial infrastructure in Ukraine:
CERT-UA <https://cert.gov.ua/article/6278706>
– How Russia-Linked Malware Cut Heat to 600 Ukrainian Buildings in Deep
Winter | WIRED
<https://www.wired.com/story/russia-ukraine-frostygoop-malware-heating-utility/>
– (13) Florian Roth on X: “I think I found a sample that looks very much
like the FrostyGoop malware targeting ICS systems & used in an attack
against an energy company in Lviv, Ukraine They forgot to mention the hash
in their report. I also published a YARA rule to detect it. @DragosInc’s
report https://t.co/MBlCBWS6bP” / X
<https://x.com/cyb3rops/status/1815771782051237998?t=iao4A2E0c1cluf_jZgsVUw&s=19>
– VirusTotal – File –
5d2e4fd08f81e3b2eb2f3eaae16eb32ae02e760afc36fa17f4649322f6da53fb
<https://www.virustotal.com/gui/file/5d2e4fd08f81e3b2eb2f3eaae16eb32ae02e760afc36fa17f4649322f6da53fb>
– Protecting OT Systems Against FrostyGoop/BUSTLEBERM Malware
(nozominetworks.com)
<https://www.nozominetworks.com/blog/protecting-against-frostygoop-bustleberm-malware>
– Чому на Сихові немає опалення та гарячої води — пояснюють у
“Львівтеплоенерго” — Cуспільне Львів (suspilne.media)
<https://suspilne.media/lviv/667490-castina-lvova-zalisilas-bez-opalenna-ta-garacoi-vodi-slidci-pereviraut-versiu-pro-hakersku-ataku/>
– Через збій у системі теплопостачання Львівтеплоенерго частина Сихова
без опалення і гарячої води — Львівська міська рада (city-adm.lviv.ua)
<https://city-adm.lviv.ua/news/society/emergency/300007-cherez-zbii-u-systemi-teplopostachannia-lvivteploenerho-chastyna-sykhova-bez-opalennia-i-hariachoi-vody>
– Індивідуальний тепловий пункт – перший крок до модернізації
багатоповерхівки | Українська Енергетика (ua-energy.org)
<https://ua-energy.org/uk/posts/indyvidualnyi-teplovyi-punkt-pershyi-krok-do-modernizatsii-bahatopoverkhivky>
– Індивідуальний тепловий пункт — Вікіпедія (wikipedia.org)
<https://uk.wikipedia.org/wiki/%D0%86%D0%BD%D0%B4%D0%B8%D0%B2%D1%96%D0%B4%D1%83%D0%B0%D0%BB%D1%8C%D0%BD%D0%B8%D0%B9_%D1%82%D0%B5%D0%BF%D0%BB%D0%BE%D0%B2%D0%B8%D0%B9_%D0%BF%D1%83%D0%BD%D0%BA%D1%82#cite_note-:0-4>
– Тендер Контролери даних «Enco control» (opendatabot.ua)
<https://opendatabot.ua/tenders/UA-2024-02-14-004122-a>
– FrostyGoop: 2004 Is Calling | LinkedIn
<https://www.linkedin.com/pulse/frostygoop-2004-calling-dale-peterson-xpw7c/>
– How to Find and Probe ENCO PLCs on the Internet Just Like FrostyGoop
malware | by Sulaiman Alhasawi | Jul, 2024 | Medium
<https://alhasawi.medium.com/how-to-find-and-probe-enco-plcs-on-the-internet-just-like-frostygoop-malware-3546ba7dfce4>
– FrostyGoop malware left 600 Ukrainian households without heat this
winter (therecord.media)
<https://therecord.media/frostygoop-malware-ukraine-heat>
– 77.52.254.9 (shodan.io) <https://www.shodan.io/host/77.52.254.9>
– CERT-UA <https://cert.gov.ua/article/6278706>
Kind regards,
Marina
