Industrial control systems (ICS) are everywhere: they run factories, control power generation stations, steer ships, and run water treatment plants. When industrial control systems malfunction, consequences can be catastrophic, and purposeful exploits of these high-value systems pose a grave threat. Nation-states with virtually unlimited resources are adding infrastructure disruption to their arsenal of cyberweapons.

Industrial control systems are often insufficiently hardened to prevent virtual or physical tampering. Their long expected lifetimes (often measured in decades) further compound security issues.

In this article, we review current cybersecurity threats facing industrial and infrastructure operators. We look into risks posed by unmanaged Single Board Computers (SBCs), like the Raspberry Pi (RPi). Finally, we list best practices for securing mission-critical systems and discuss the principles device manufacturers need to use when building ICS products.

The growing threat of industrial exploits

Perhaps the most well-known example of industrial sabotage occurred over a decade ago. Stuxnet was an exploit designed by Western intelligence agencies to cause damage to Iran's nuclear program. The code hijacked an air-gapped SCADA system and caused centrifuges used for uranium enrichment to self-destruct.

Iranian President Mahmoud Ahmadinejad during a tour of centrifuges at Natanz in 2008.

Prior to Stuxnet, in 2007, Idaho National Laboratory conducted the Aurora Generator Test showing how a carefully crafted cyberattack can damage physical components on the power grid. Interestingly, while the attack targeted and successfully destroyed a diesel generator, it did so via a programmable digital relay on a circuit breaker, which then caused the generator to go out of phase. Even if a critical system is not vulnerable to direct attacks, ancillary systems can be exploited to cause failure through unplanned behavior.

A diesel generator destroyed in a mock cyberattack. Aurora Generator Test.

More recently, an attack of a water treatment plant in Oldsmar, Florida increased levels of lye (a caustic chemical otherwise known as sodium hydroxide) in the towns' water supply to 100x normal.  The facility used an outdated Windows 7 machine to access the plants' SCADA systems. The system had no firewall and used a shared password. TeamViewer was often used to remotely access the machine. Disturbingly, the attack was thwarted only because an employee noticed the mouse cursor moving by itself.

Non-nation state adversaries have been using ransomware to attack industrial targets. Colonial Pipeline operations were disrupted for a week in May of 2021, dramatically affecting gas prices in several states and causing shortages. This attack is clear evidence that cyberattacks can trigger civil unrest. Notably, this attack was not targeting the ICS infrastructure, but rather the billing system. Measures should be taken to assess the operational risk of peripheral systems.

Claroty, an industrial cybersecurity company, has released its biannual industrial control systems risk and vulnerability report, which found that the number of reported vulnerabilities increased by 25% when compared to 2019, with critical infrastructure areas like manufacturing, energy, water, and commercial facilities being most affected.

Breakdown of vulnerabilities in critical infrastructure sectors year-over-year. Source: Claroty

Unmanaged Single Board Computers (SBCs) – a potential attack vector

In recent years SBCs have gained wide popularity due to their low cost, relative power, and versatility. In industrial computing, they are often used as "industrial glue" connecting systems, gathering data, and providing access to legacy equipment. Unlike Programmable Logic Controllers (or PLCs) which are harder to program and lack flexibility, SBCs use standard operating systems like Linux and have popular development platforms such as Node-RED and Python.

This led many new devices and monitoring systems to be developed using SBCs. 44% of Raspberry Pis are sold to industrial customers. I have seen RPis hiding in semiconductor plant machinery, utility-scale energy generation systems, vital monitoring equipment, and even casinos. The ubiquity of the RPi has normalized its usage in many industrial settings, including interfacing with SCADA systems. There is a cottage industry of vendors making industrial Pi add-ons.

Raspberry Pi in a typical industrial application.

However, the current crop of SBCs trade flexibility for security. Here are the reasons why SBCs pose a significant unmitigated risk to critical industrial systems:

  • Lack of standardization in hardening SBCs. They are not managed remotely as normal enterprise devices, and there is no agreed-upon security framework endorsed by the equipment manufacturers. Patching is rarely done once a system is deployed. An SBC that is buried deep inside another product is likely to be running unpatched software with known vulnerabilities.  Insecure, easy-to-guess passwords are often used for convenience.
  • Many SBCs do not have Secure Boot capability. This is a critical security feature that is necessary to secure systems. Only cryptographically validated firmware should be able to boot on a device.
  • Adhoc SBCs are seldom hardened beyond the default OS image, which includes a wide range of software packages, libraries from external sources, and active services. This leaves a wide attack surface.
  • A compromised SBC is a perfect launchpad for attacks from inside the network. Its fast CPU, memory, and storage enable an adversary to run advanced penetration tools. Additional built-in connectivity options in the form of WiFi and Bluetooth provide further channels for propagation. When interfaced with SCADA, they provide an excellent staging area for an attack on high-value physical assets. A compromised SBC can potentially be used to bridge an air-gapped network.
  • Connected industrial devices are often placed outdoors, or in other areas where physical access is possible. An attacker has multiple entry points, including plugging a keyboard and a screen directly into the open ports.
  • An LTE modem can be plugged into a spare USB port or added as a hat to remotely access the SBC. Because they're typically in an enclosure, it would be impossible to tell if a device is compromised without looking inside.
  • The SD card can be easily tampered with and replaced.
  • Custom code running on these devices is likely to be written in Node or Python, which while being convenient to use, rely on open source library repositories such as npm and PyPi. Compromised code has and is being committed to open source repositories.
A .bash_history file being accessed by the package uploaded to npm.

Many of these issues are not exclusive to SBCs, but with approximately 15.4MM Raspberry Pi units sold to the industrial sector -- they stand out because of their huge install base. It is reasonable to assume that a significant number of units are deployed in sensitive areas such as infrastructure, aerospace, defense, and manufacturing.  This makes industrial devices built on SBCs enticing targets for cyberattacks.

Even without attacking vulnerabilities in the device itself (A Risk Assessment on Raspberry PI using NIST Standards), tampering and exploiting improperly configured SBCs is a serious threat.

For example, in 2018, hackers used a Raspberry Pi connected to the IT network at Jet Propulsion Laboratory and stole 500MB of data related to the Mars missions. The attack went undetected for over a year.

The April 2018 cyberattack exploited this particular weakness when the hacker accessed the JPL network by targeting a Raspberry Pi computer that was not authorized to be attached to the JPL network.

For industrial control systems, cyber espionage is the least worrisome threat. Compromised devices can lay dormant and stay undetected, awaiting the command to disrupt critical systems.

It should be assumed that many industrial and infrastructure systems have already been penetrated to some extend. According to Trend Micro, in 2021, 6% of ICSs in the US were infected with malware.

Percentage of infected ICS equipment by country in 2021. Source: Trend Micro.

Best practices for securing Industrial Control Systems (ICS)

Securing industrial control systems requires a multi-layered approach. Security is security, and there is no gap between physical security and cybersecurity.

Consider instituting the following recommendations:

  • The threat model must include any connected device with an embedded microcontroller.
  • Even if a critical system is not vulnerable to direct attacks, ancillary systems can be exploited to cause failure (ie Aurora Generator Test). Perform a top-down review of processes for exploitable vulnerabilities across multiple steps and systems.
  • Patch systems with security updates at a regular schedule, including air-gapped systems. End-of-life software should be decommissioned. Any unpatched device is a potential staging point for wider attacks.
  • Use network micro-segmentation to prevent lateral movement of attacks. Ensure industrial control systems are on a physically separate network from the rest of the organization. Inventory data flows between critical systems, and limit communication only to what is necessary for day-to-day operation.
  • Make sure ICS devices implement a Hardware Security Module (HSM) and use signed firmware to prevent tampering. Avoid using insure SBCs where possible.
  • Inventory all SBC devices, develop a consistent hardening profile, and perform frequent patches on all devices centrally. Install monitoring tools to detect tampering
  • Conduct regular penetration tests, including compromised physical security scenarios.
  • Deploy a Network Access Control (NAC) system to prevent unauthorized devices from being plugged into the network. Inventory all connected devices.
  • Support teams should implement procedures for updating equipment with vendor-provided patches in a timely manner.
  • Prioritize physical access and security. Disable USB/networking/WiFi/Bluetooth connectivity on all devices unless specifically required.
  • Use the principle of least access privilege (PoLP) – a user should be given a minimum level of access to perform his or her job function.
  • Limit the use of remote access. If access is required, consider strict role-based and time-based entitlements.
  • Employ strict deployment control of custom code. Audit for vulnerabilities.
  • Increase network surveillance with firewalls that have an Intrusion Detection System (IDS). Monitor log files and file systems for unusual changes. As the NASA breach demonstrates, the attackers are willing to be slow and deliberate with the rate at which they move data, making detection difficult. Network traffic patterns may differ from traditional exploits (the goal could be to disrupt rather than steal data).
  • Perform regular backups of critical systems with retention time-lock for ransomware recovery.

Best practices for designing secure industrial products

ICS equipment manufacturers must design products with security at their core. Consumer-grade, generic computing platforms (including the Raspberry Pi) are fine for prototyping, but cannot be used to shortcut product development and rush products to market. Security best practices should always be applied when designing ICS equipment, with 3rd party penetration testing being a part of the development cycle.

As supply chain cyberattack targets, industrial control device manufacturers must prioritize security in all aspects of their operations.

  • Consider yourself a prime target for cyberattacks. Supply chain attacks such as SolarWinds are highly effective at exploiting multiple sensitive targets at once. Secure your infrastructure and all aspects of the build process. Examine downstream supply chains (NCSC: Software Supply Chain Attacks).
  • Minimize the attack surface of your products. Generalized hardware such as Raspberry Pi have their place in the development lifecycle, but production industrial devices need be streamlined to perform only their intended function and stripped of all else. Unused USB ports and radios should be eliminated from the design. The operating system should similarly be streamlined with unnecessary services removed, and if possible, replaced with single-purpose firmware running on a real-time operating system (RTOS).
  • Build security into everything. Even if your system is non-critical, it can be used as a staging area to compromise other, more critical systems. Only use hardware components that implement robust security.
  • Sign all firmware. The device should also have a hardware module to validate the cryptographic signature of the firmware. Securely store all keys.
Industrial control systems must cryptographically validate the code they execute.
  • Do not rely on physical access control as part of your security model. Include physical tampering in threat analysis and design. For example, many production devices retain UART pinouts. That can be exploited to tamper with the system.
  • Audit 3rd party libraries for vulnerabilities. Avoid using code repositories directly. See: Several Malicious Typosquatted Python Libraries Found On PyPI Repository.
  • Log all system functions and integrate with logging tools such as Splunk.
  • Utilize safe languages. Consider replacing C/C++ with Rust for building software. Rust prevents many memory safety bugs which are commonly exploited.
  • Include penetration testing into the development lifecycle.

Conclusion

Adversaries are growing bolder and more sophisticated in their attacks. Recent events have shown that cyberattacks can effectively incite social unrest. Nation-states are embracing cyber warfare as a cost-effective weapon, with critical industrial and infrastructure systems as targets.

Attacks will inevitably succeed, so operational resilience to cyberthreats must be derived from how attacks are managed and their impact minimized. Operators must take a security-first posture. They must plan and implement security measures that assume their systems have already been compromised.

Industrial control system networks must be segmented from other systems, with any access to the network tightly controlled and continuously monitored. Attack surface must be minimized by eliminating devices and services that are non-essential. Lateral access within the network should be minimized through micro-segmentation. Always use the principle of least access privilege (PoLP) for users and services. Do not rely on lack of physical access as a security measure.

All critical systems must be continuously monitored for threats and attacks. An incident response plan should be maintained and tested regularly. Processes must be put in place for patching systems as soon as vulnerabilities emerge. During the procurement of ICS equipment, security should be heavily scrutinized including supply chain risks.

Combined with principled and rigorous implementation, these measures can greatly minimize the risk and impact of cyberattacks.

--

Please leave a comment below and thanks for reading!