The short answer
Manual Operation Procedures
Manual operation procedures describe how each process is run when automation is unavailable, at three levels: without the SCADA but with the controllers running, without a controller so that equipment is operated in hand at the panel, and without power beyond the generator. For each process they name the local instruments to read, the settings to make by hand, the checks and their frequency, the paper log, the dose arithmetic for chemical feeds from a flow reading, the conditions under which a process is stopped, the staffing to sustain it around the clock, and how long it can be sustained. They are kept on paper in the control room and at each site, they are the fallback for cyber incidents and for equipment failures alike, and they are practiced on a schedule so that operators have done it before the day they must.
Key points
- Three levels: no SCADA, no controller, no power. Each is a different procedure for the same process.
- Per process: what to read, what to set, how often to check, what to write down, when to stop.
- Chemical feed by hand is arithmetic from a flow reading; the tables are on the procedure.
- Staffing and duration are part of the procedure; a plan that needs eight people has to say so.
- Paper: the procedures, the log sheets, and the phone list live in the control room and at the sites.
- Drill it: a manual shift once a year, with the SCADA screens turned off, finds what the procedure forgot.
Three levels
| Level | What still works | What the operator does |
|---|---|---|
| No SCADA | Controllers run the process on local logic; local HMI panels work; no central displays, alarms, remote control, or history | Rounds of the plant and the sites, reading local HMIs and instruments; setpoint changes at the local HMI; alarms by observation; paper log |
| No controller | Equipment runs in hand at the panel: pumps started and stopped with the HOA switch, valves operated by hand, feed pumps set manually | Operates equipment by hand against readings from gauges, sight glasses, and portable instruments; continuous attendance at critical processes |
| No power beyond the generator | Whatever the generator carries; often the controllers and some pumps | Prioritizes loads per the generator plan; runs the remaining processes by the level above |
What each procedure contains
- The process and the level the procedure covers, and the trigger for using it.
- The instruments to read by hand: which gauge, which sight glass, which portable meter, where the sample point is.
- The settings to make: which switch to which position, which valve to which position, which feed pump at which stroke and speed, in the order to do them.
- The checks and their frequency: level every 30 minutes, residual every 2 hours, and the limits that require action.
- The dose arithmetic: a table of feed pump settings against plant flow for the current chemical strength, or the formula and a worked example.
- The stop conditions: the readings at which the process is shut down rather than run by hand, and how to shut it down safely.
- The log sheet: a paper form with the readings, the settings, the time, and the initials.
- Staffing: how many people, where, and the maximum shift; the rounds route for remote sites and the driving time.
- Communications: radio channels, the phone tree, and who is told what and when.
- Return to automatic: how to hand the process back to the controller or the SCADA without a bump.
Examples
- Lift station without SCADA
- The controller runs the pumps on level. The operator visits on a schedule set by the wet well volume and the inflow, reads the level and the pump status at the panel, confirms the floats are in place as backup, and logs it. High level is found by the visit or by a local dialer; the schedule tightens in wet weather.
- Lift station without a controller
- Pumps in HAND at the panel with the floats providing high-level protection where they are wired to the starters directly, or an operator at the station running pumps by the level in the well. The procedure says how long a pump may run, how to alternate, and when to call for a tanker.
- High service pumping without SCADA
- The operator reads the discharge pressure gauge and the clearwell level at the plant, and the tank levels by visiting or by a local gauge, and starts and stops pumps at the panel to hold the pressure band and keep the tanks between limits. A schedule based on the normal demand pattern, adjusted by the readings.
- Chlorination by hand
- The feed pump set from the flow read at the meter and a table of pump settings per flow for the current hypochlorite strength; residual sampled at the contact tank outlet on a schedule with a portable colorimeter and the setting adjusted; the log records flow, setting, and residual. Loss of feed is found by the residual, so the sampling interval is short.
- Filter backwash by hand
- The operator runs the sequence from the local valve controls: influent closed, drain to level, air scour timed by a watch, wash at the rate read on the local flow indicator, settle, filter to waste until the portable turbidimeter reads below the setpoint, return to service. The written sequence with durations is on the procedure.
Writing them
- 1
List the processes
Every process and every remote site, with the level of automation each depends on.
- 2
Write the no-SCADA procedure first
It is the most likely case and the simplest: rounds, local HMIs, paper log, and communications.
- 3
Write the no-controller procedure for the critical processes
Disinfection, high service, lift stations with overflow consequences, chemical feeds with overdose consequences. The others can wait for a later revision.
- 4
Make the local readings possible
A process that can only be read through the SCADA has no manual procedure. Add the gauge, the sight glass, the local indicator, or the portable instrument, and the sample point.
- 5
Do the arithmetic
Dose tables per chemical at the current strength, pump setting tables, generator load lists.
- 6
Staff it
Count the people needed for 24-hour manual operation, and where the shortfall would be filled: mutual aid, the integrator, retired operators, the on-call list.
- 7
Print it
The procedures, the log sheets, and the phone tree in binders in the control room, at each site, and in the safe. Update them when the plant changes.
Duration
Manual operation is sustainable for a period that the procedure states honestly: a few days at most utilities with normal staffing, longer with mutual aid. The incident response plan uses that number to decide how fast recovery has to happen and when to call for help. A procedure that assumes manual operation can go on indefinitely has not counted the people, and a recovery plan that assumes the SCADA will be back in a day has not tested the restore.
Frequently asked questions
- Is manual operation only for cyber incidents?
- No. The same procedures cover a server failure, a lost network, a lightning strike on the radio system, and a controller that died. Cyber incidents are the case where every automated system may be unavailable at once and for longer, which is why the procedures have to cover the no-controller level for the critical processes.
- Our operators have never run the plant by hand. Where do we start?
- With the no-SCADA procedure for the whole plant, which is mostly rounds and paper logs, and a manual shift to practice it. Then the no-controller procedure for disinfection and the highest-consequence lift station. Two procedures and one drill change the utility posture more than any purchase.
- What about regulatory monitoring during manual operation?
- It continues by grab sample and portable instrument on the schedule the permit requires, and the paper log is the record. The procedure lists the samples, the instruments, the intervals, and the reporting; the regulator is told that the plant is in manual operation and how compliance is being demonstrated.
- How do we hand back to automatic?
- Process by process, with the controller or the SCADA confirmed clean and its setpoints checked against the manual settings, the equipment placed in AUTO with the operator watching the first cycles, and the manual log closed with the time. The procedure describes it so that the handover is not the moment something is left in HAND by mistake.
Related topics
- OT Incident Response PlanThe plan for the day the control system is not trusted: who decides, the triggers, the first hour, isolating without stopping treatment, running by hand, preserving evidence, who to notify and when, recovery from backups, and the exercise that makes it real.
- Isolating a Compromised SystemCutting a compromised machine, segment, or site off from the control system without stopping the process: who may decide, isolation points planned in advance, disconnecting the network rather than the power, running on manual control, and preserving evidence.
- Lift Station Backup ControlHow a lift station keeps pumping when the transmitter, the PLC, or the SCADA link is gone: float backup logic, the control transfer relay, redundant-off protection, and how to test it with the controller actually dead.
- Hand-Off-Auto SelectorsWhat each position of an HOA selector does, how it is wired into the starter circuit, which protections must survive Hand, what the controller should see, and the mistakes that let a pump run with nobody watching.
- Chemical FeedChemical feed from the control side: the chemicals and where they go, metering pumps and dry feeders, flow pacing with residual trim, the dose arithmetic, day tanks and drawdown tests, loss-of-feed detection, interlocks, and overfeed protection.
- Recovery and RestorationBringing a control system back after a compromise without reintroducing it: controllers verified against baselines first, servers rebuilt from clean media and project exports, every credential changed, hardening, staged reconnection, and end-to-end tests.
Direct contact
Have a controls question?
Reach Eric Sullivan directly about anything on this site, a controls or automation topic, or one of his personal projects.