The short answer
Credential Storage
Credentials for a control system live in a credential vault, a password manager built for teams or an enterprise secrets system, with an individual login for each person, access by role so that a technician sees the device passwords they maintain and not the administrator passwords, and a log of every retrieval. Service account credentials live in the platform credential stores that the SCADA and the operating system provide, not in scripts. A sealed break-glass copy of the administrative and vault recovery credentials is kept in a physical safe under a procedure. The vault is backed up off the network and the recovery is tested, so that the departure of the person who set it up does not lock the utility out. Credentials are removed from every other place they have accumulated: spreadsheets, sticky notes, drawings, controller project files, integrator archives, and email.
Key points
- One vault, individual logins, role-based access, and a log of who retrieved what.
- Service account credentials in the platform credential store, never in a script or a configuration file.
- A sealed break-glass copy in a safe, with a procedure for opening it and a rotation after every use.
- The vault is backed up off the network, and recovery is practiced before the vault administrator leaves.
- Hunt down the copies: spreadsheets, notes, drawings, project files, integrator archives, email.
- Contractors get credentials through the vault, temporarily, and the credentials are rotated when they finish.
Where credentials are found today
| Place | Why it is there | Problem |
|---|---|---|
| A spreadsheet on the shared drive | The integrator left one at commissioning; it was convenient | Readable by everyone with the drive, copied to laptops, emailed, never updated |
| Sticky notes and the inside of the panel door | The console password had to be remembered by everyone | Visible to every visitor and every photograph |
| Drawings and specifications | The device password was on the network drawing | Drawings are shared with bidders, contractors, and consultants |
| Controller and HMI project files | The HMI project stores the SCADA password; the controller project stores the device password | Project files are backed up, emailed, and kept by every integrator who touched them |
| Scripts and configuration files | A database password in a report script; a service password in a driver configuration | Anyone who reads the file has the credential; rotation breaks the script |
| Email and text messages | A password was sent to a contractor in a hurry | Retained forever in mailboxes on both ends |
| One person memory | The administrator knows them all | The administrator leaves, is ill, or is on vacation |
The vault
A credential vault is a purpose-built store: encrypted, with individual accounts, groups or roles that decide which entries each person can see, an audit log of every view and change, sharing that does not involve copying the password into a message, and a way to store notes, keys, and certificates as well as passwords. Team password managers do this at a cost a small utility can afford; enterprise secrets systems do it at scale. Either is run on infrastructure the utility controls or in a service the utility has evaluated, with its own multi-factor authentication, and its master credentials are the most protected thing the utility owns.
- 1
Choose and set up
A vault product with team features, individual accounts, roles, an audit log, and an offline export for backup. Multi-factor authentication on every vault account.
- 2
Structure it
Folders or groups by system and site: SCADA, network, controllers by site, radios, vendors. Each entry with the device, the address, the account, the password, and the date it was last changed.
- 3
Load it
Every credential from every place it currently lives, changing each one as it is loaded if it has ever been shared widely.
- 4
Grant access
Each person sees what their role needs. Administrators see administrator entries; technicians see the devices they maintain; nobody sees everything except the two people who administer the vault.
- 5
Move service accounts
Credentials used by software into the platform credential stores, with the vault holding a copy for recovery.
- 6
Break-glass
The vault recovery credentials and the top administrative credentials printed, sealed, signed across the seal, dated, and locked in a safe with a log of who opens it. Rotated after any opening.
- 7
Back up
An encrypted export of the vault on a schedule, kept offline with the break-glass envelope, and a restore tested once.
- 8
Clean up
Delete the spreadsheet, shred the notes, remove passwords from drawings and project file comments, and ask the integrator to confirm they have removed them from their copies.
Service accounts and software
A report script that connects to the database, a driver that logs into a device, a collector that authenticates to the historian: each needs a credential, and the credential does not belong in the script. The SCADA platform, the operating system, and the database each provide a credential store or a managed account mechanism that keeps the secret out of the file and lets it be rotated in one place. Where an older product insists on a password in a configuration file, the file is protected by permissions so that only the service can read it, the password is unique to that use, and the fact is recorded in the vault entry so that rotation includes it.
Contractors
An integrator or a vendor needs credentials for a project. They receive them through the vault, as a share to a named account with an expiry, or in person for devices, and the record shows what they were given. When the engagement ends, every credential they received is rotated, and the vault log makes the list. Credentials sent in an email to a contractor are in that contractor mailbox forever; credentials shared through the vault can be withdrawn.
Frequently asked questions
- Is a cloud password manager acceptable for a water utility?
- Reputable team password managers use end-to-end encryption so that the service cannot read the entries, and many utilities use them. The utility evaluates the product, enforces multi-factor authentication on it, keeps an offline encrypted export, and decides for itself whether the most sensitive credentials, such as the break-glass set, stay offline in the safe instead. A self-hosted vault avoids the question at the cost of running it.
- What goes in the safe?
- The vault recovery credentials, the top administrative credentials for the SCADA and the directory, the firewall administrator credentials, and anything else without which the system could not be recovered if the vault were lost, each in a sealed, signed, dated envelope, with a log of openings. Not every device password; the vault holds those.
- The controller project file contains the HMI password in a comment. Does that matter?
- Yes: the project file is backed up, emailed to the integrator, and kept for years by everyone who worked on it. Remove credentials from comments and descriptions, rotate the credential, and treat project files as sensitive documents from then on.
- How do I hand a password to a technician in the field?
- They retrieve it from the vault on their own device with their own login, which logs it. If the field has no connectivity, they retrieve it before leaving, and the vault entry is rotated on a schedule appropriate to how often it is retrieved. Reading it over the radio or texting it is not a method.
Related topics
- Password Policy for OTA password policy that fits a control system rather than an office: length over complexity, rotation on events, screening against breached lists, multi-factor authentication, no lockouts that take control from an operator, and unique vaulted device passwords.
- Shared Account ProblemsWhy the shared operator login, the vendor account everyone knows, and the single PLC password are the weakest point in most utilities: no accountability, no offboarding, no detection. Containing what cannot be avoided and migrating the rest.
- Default CredentialsThe passwords printed in the manual: where they hide in a control system, why they are the first thing an attacker tries, how to find every one on site, and the procedure for changing them without locking yourself out.
- What to Back Up in a Control SystemWhat a utility must be able to restore after a failure or attack: controller programs and firmware, SCADA and historian, HMI panels, drive and instrument parameters, network configurations, licenses, installers, and documentation, and how often each changes.
- Vendor Remote AccessIntegrators and equipment vendors need to get in. How to let them without leaving a permanent door open: utility-controlled sessions, named accounts, MFA, a jump host, session recording, and what to do about the modem you did not know was there.
- OT Incident Response PlanThe plan for the day the control system is not trusted: who decides, the triggers, the first hour, isolating without stopping treatment, running by hand, preserving evidence, who to notify and when, recovery from backups, and the exercise that makes it real.
Direct contact
Have a controls question?
Reach Eric Sullivan directly about anything on this site, a controls or automation topic, or one of his personal projects.