The short answer
Firmware Management
Firmware management is the process by which the software inside controllers and devices is kept at versions that are supported, free of known exploitable vulnerabilities where a fix exists, and consistent across the fleet, without an update ever stopping the process unexpectedly. It rests on the inventory of versions, the vendor lifecycle and security notices, and a decision per device family about the target version and the trigger for moving to it: a security advisory with exposure, a defect that affects the plant, an end of support, or a fleet standardization. The update itself is a change under change management performed at a planned outage: the program and configuration backed up and verified, the firmware image downloaded from the vendor and verified by hash, the update tested on a spare or a test unit of the same model, the process placed in manual, the update applied per the vendor procedure, the program reloaded or converted if the version requires it, the device tested against the functional description, and the inventory and baseline updated. Devices whose firmware cannot be updated are protected by compensating controls and scheduled for replacement.
Key points
- Firmware lags because updates need outages and can change behavior; it still matters because advisories name versions.
- Per device family: a target version, a trigger for moving, and a compensating control for the gap.
- Every update is a planned outage with a backup, a verified image, a test on a spare, a manual fallback, and a rollback.
- Some controller updates require the program to be converted or re-downloaded; know before the outage.
- Communication modules and network devices are updated more often than processors; they are the exposed parts.
- Update the inventory, the baseline, and the signature in the SCADA as the closing step.
Why firmware is different
A server patch is applied in a window and rolled back from a snapshot if it fails. A controller firmware update takes the controller out of service, may require the program to be converted to a new version of the programming software and re-downloaded, may change the behavior of an instruction or a communication timeout in a way the program did not expect, and cannot be rolled back on some platforms without a vendor procedure. Drives lose their parameters on some updates. Switches drop every connected device for the minutes the update takes. None of that means firmware is left alone; it means firmware is updated deliberately, with a reason, at an outage, with preparation.
Deciding
| Trigger | Response | Timing |
|---|---|---|
| Security advisory with a fix, and the vulnerable interface exposed | Update, or apply a compensating control and schedule the update | Emergency if exploited and exposed; otherwise the next outage |
| Security advisory with a fix, interface not exposed | Schedule the update; record the exposure analysis | Next planned outage for that process |
| Defect notice affecting a function the plant uses | Update | Next outage, sooner if the defect is active |
| End of support for the current version | Update to a supported version, or plan the replacement | Before support ends |
| Fleet standardization | Bring identical devices to one version so that spares and programs interchange | Over a year of outages |
| New features not needed | Do nothing | A version that works and is supported is left alone |
The update
- 1
Read the release notes
Every version between the current and the target: behavior changes, program conversion requirements, programming software version required, known issues, and the update procedure. Some updates must be applied in sequence through intermediate versions.
- 2
Back up and verify
The program and the configuration uploaded from the device and compared with the baseline; the drive parameters exported; the switch configuration saved. Verified, not assumed.
- 3
Obtain and verify the image
From the vendor site through the controlled transfer path, with the hash checked against the vendor publication. An image from an email or a contractor laptop is not applied.
- 4
Test on a spare
The update applied to a spare or a test unit of the same model, the program loaded, and the functions exercised. The spare then becomes the tested fallback unit for the outage.
- 5
Schedule the outage
A change record; operations agree the window; the process in manual or on the backup control per the manual operation procedure.
- 6
Apply
Per the vendor procedure, from the engineering workstation, with the console cable connected and the power stable; a power loss during a firmware write is the way to make a controller a paperweight.
- 7
Reload and test
The program re-downloaded or converted as the release notes require, the configuration restored, and the device tested against the functional description: I/O, communications, sequences, alarms.
- 8
Return to service
The process returned to automatic with the operator watching the first cycles.
- 9
Close
The inventory version, the baseline, the SCADA signature value, and the change record updated; the release notes filed.
By device
- Controller processors
- Rarely, deliberately, at a major outage, with conversion planned. The processor is usually reachable only from the engineering zone, which lowers the urgency.
- Communication modules and Ethernet cards
- More often; they face the network and carry most of the advisories. Some can be updated without stopping the processor.
- I/O modules
- Rarely; when a defect affects a channel type or a fleet standardization calls for it.
- Drives
- When a defect or advisory affects the drive, with the parameter set exported first and re-imported after, and a check of every parameter that the release notes say changed default.
- HMI panels
- With the runtime and the project; the project may need conversion like a controller program.
- Switches, firewalls, radios, modems
- On advisories and on a quarterly review; firewalls facing outward are the emergency tier. Configuration backed up, the window agreed, the connected devices expected to drop.
- Instruments
- When the vendor advises; the configuration exported first; the device description for the configuration tool updated to match.
When it cannot be updated
A controller family past its end of support, a firmware that the vendor has withdrawn, a device whose update requires hardware the utility does not have: the record says so, a compensating control restricts who can reach the vulnerable interface, change detection watches it, and the replacement is on the capital plan with a date. The risk assessment carries it every year until it is gone, and the fleet standardization that comes with the replacement is planned so that the next generation is easier to manage than the last.
Frequently asked questions
- Should every controller be on the latest firmware?
- No. Every controller should be on a supported version free of exploitable vulnerabilities in the interfaces it exposes, and identical to its siblings where practical. The latest version has the newest defects; a version a year old with a clean advisory record is often the better target.
- The vendor update tool wants internet access. How do we do it?
- Download the image on a workstation outside the control network, verify the hash, transfer it through the controlled path, and apply it from the engineering workstation with the offline procedure the vendor provides. Vendors provide offline procedures because plants ask for them.
- How do we roll back a controller firmware update that broke something?
- On platforms that allow downgrade, by applying the previous image per the vendor procedure and reloading the backed-up program; on platforms that do not, by swapping in the tested spare on the old version, which is why the spare is tested before the outage. The rollback plan is in the change record before the update starts.
- Can the integrator do the updates?
- Under the utility change process, with the utility backup, the utility test, and the utility record. The integrator brings the platform expertise and the conversion experience; the utility owns the decision, the outage, and the result.
Related topics
- Documenting Firmware VersionsWhy the firmware version of every controller, module, drive, switch, radio, and instrument belongs in the inventory with the date it was read: advisories are issued by version. Where to read each one, the several versions one controller has, and the cadence.
- Risk-Based PatchingDeciding what to patch first when everything cannot be patched at once: priority from exposure, exploitation, and consequence rather than scores, tiers with timelines, different rules for servers, network gear, and controllers, and the written accept decision.
- Change Management for OTA change process sized for a utility control system: what counts as a change, the request with risk and rollback, owner approval, the window, backup before and verification after, the baseline update, the emergency path, and the record.
- PLC Program BackupsWhat a complete controller backup contains, how often to take one, where to keep it, why the upload from the running controller is not the master, and the verification that turns a folder of files into a recovery plan.
- Compensating ControlsWhat to do about a vulnerability that cannot be patched now or ever: restricting who can reach it, disabling the service, blocking dangerous protocol functions, allowlisting, monitoring, physical and procedural controls, matched to the attack path.
- Controller HardeningWhat can be locked down on a PLC or RTU: keyswitch and mode, credentials, unused services, access lists, firmware, and the engineering workstation that is the real target.
Direct contact
Have a controls question?
Reach Eric Sullivan directly about anything on this site, a controls or automation topic, or one of his personal projects.