The short answer
SCADA Backups
A SCADA backup is several backups: an export of the application project in the vendor format after every change, an image of each server on a schedule and before every patch, the databases that hold the alarm journal and configuration daily, the historian archive files as they close, the license files and activation records, certificates and keys, scripts, report templates, and custom code, and the configuration files of the firewalls, switches, and radios that the system depends on. Each has its own frequency and retention, multiple versions are kept so that a backup taken after a compromise does not overwrite the last clean one, copies are kept in three places including one offline, the backups are encrypted and their completion is monitored, and a restore procedure is written and tested so that the backups can actually be used.
Key points
- The project export and the server image are different backups with different uses; keep both.
- Licenses, activation records, certificates, and installers are part of the backup, or the restore stops at the license prompt.
- Historian archives are regulatory records; they are backed up as files as they close.
- Keep versions, not just the latest; the latest may already be compromised.
- Three copies, two kinds of media, one offline; monitored, encrypted, and documented.
- A backup without a tested restore procedure is a file with hope in it.
What has to be in it
| Item | Method | Frequency | Retention |
|---|---|---|---|
| Application project | Export in the vendor project format, with displays, tags, alarms, scripts, and security configuration | After every change, and weekly | Every version for a year; milestones forever |
| Server images | Full image of each virtual machine or physical server | Weekly, and before every patch or upgrade | Several weeks of versions; one per quarter for a year |
| Databases | Database backup of the alarm journal, audit trail, configuration, and reporting databases | Daily, with transaction logs where the database supports them | Thirty days online; monthly for the record period |
| Historian archives | File backup of archive files as they close, plus the active archive daily | Daily | The retention policy for the data, which may be years |
| Licenses and activation | License files, activation codes, vendor account details, dongle records | On every change | Forever, with the images |
| Certificates and keys | Server certificates, private keys, OPC UA certificates, VPN keys, with their passwords in the vault | On every change | Until expiry plus one year |
| Scripts, reports, custom code | File backup of everything not in the project export | After every change | Every version |
| Network device configurations | Firewall, switch, router, and radio configuration exports | After every change, and monthly | Every version |
| Installers and media | The installation packages for the SCADA version in use, the operating system, and third-party software | When a version is installed | Until the version is retired |
| Documentation | The functional description, drawings, IP schedule, restore procedure, and the vault export | On every change | Every version |
Two kinds of backup
A project export is what the vendor software can import: it restores the application onto a freshly installed server, survives a version change, and can be inspected and compared. A server image is what the hypervisor or the imaging tool can restore: it brings the whole machine back in minutes, exactly as it was, on the same or compatible hardware. The image is the fast path after a hardware failure or a bad patch; the project export is the path after a compromise, when the image cannot be trusted, and after a platform upgrade, when the image is the wrong version. A system with only images cannot be rebuilt clean; a system with only project exports takes a day to rebuild. Keep both.
Versions and places
Ransomware often sits in a system for weeks before it acts, and a backup taken during that time contains it. Keeping only the latest backup means the last clean copy has been overwritten by the time anyone notices. Retention keeps versions going back months, with milestones kept indefinitely. The three-two-one rule places three copies on two kinds of media with one off site, and the control system version adds one offline: a copy that no network, no credential, and no process on the control network can reach, which is the copy that survives the day everything else is encrypted.
Mechanics
- Backups are automated and scheduled; a backup that depends on someone remembering is taken until they forget.
- Completion and size are monitored and alarmed; a job that fails silently for a month is a month with no backup.
- Backup data is encrypted, and the key is in the vault and in the break-glass envelope, not on the backup server.
- The backup target is on the control network or the DMZ with a one-way push, not on the office file server.
- Backup credentials are unique to the backup service and are not the domain administrator.
- A restore procedure is written for each item, tested, and kept where it can be read with the servers down.
The restore procedure
For each item, the procedure says where the backup is, what it is restored onto, in what order, with what credentials and licenses, and how the result is verified. For a server it includes the license reactivation with the vendor contact, the archive re-registration for the historian, the reconnection of clients, and the alarm state check. For the project export it includes the fresh installation of the platform at the right version. The procedure is exercised at least once a year, timed, and corrected, and the time it takes is the recovery time the utility should plan around.
Frequently asked questions
- How often should the project be exported?
- After every change, as part of the change record, and on a weekly schedule to catch what was missed. Exports are small and versions are cheap; a change that was made and not exported is a change that has to be made again after a restore.
- Do virtual machine snapshots count as backups?
- No. A snapshot lives on the same storage as the machine, grows until it hurts performance, and is lost with the storage. Snapshots are for rolling back a patch within hours; backups are copies elsewhere. Take a backup before the patch as well as the snapshot, and delete the snapshot afterward.
- What about the operator consoles?
- An image of the standard client build, kept current, plus the project deployment. Individual consoles are not backed up; they are reimaged from the standard build, which is faster and cleaner than restoring a machine that has accumulated years of drift.
- Who is responsible?
- A named person, with a schedule, a monitoring alarm that reaches them, and an annual test that management sees the result of. Backups at most utilities fail through neglect rather than technology, and the fix is a name.
Related topics
- What to Back Up in a Control SystemWhat a utility must be able to restore after a failure or attack: controller programs and firmware, SCADA and historian, HMI panels, drive and instrument parameters, network configurations, licenses, installers, and documentation, and how often each changes.
- Backup TestingWhy backups that have never been restored fail when needed, and how to find out: spot restores, full restores of a server to an isolated machine, controller program restores to a spare, archive reads, timing each one, and the findings and record that follow.
- Offline CopiesThe backup copy that survives ransomware: why anything reachable over the network will be encrypted too, what offline and immutable mean, options from rotated drives to immutable storage, what goes in the offline set, and the restore test that uses it.
- PLC Program BackupsWhat a complete controller backup contains, how often to take one, where to keep it, why the upload from the running controller is not the master, and the verification that turns a folder of files into a recovery plan.
- Server FailureHow SCADA servers fail and what to do when one does: failure modes from a full disk to a dead hypervisor host, what the plant experiences with and without redundancy, the recovery sequence from restore to alarm verification, and the monitoring that warns.
- Long-Term StorageKeeping historian data for years: a retention policy by data class that follows the record rules, tiered storage from online to offline copies, downsampling, archive management, exports that outlive the historian, and the annual proof that old data reads.
Direct contact
Have a controls question?
Reach Eric Sullivan directly about anything on this site, a controls or automation topic, or one of his personal projects.