Last week, we mapped the operational terrain, establishing that risk management is the "Where" of your Safety Management System. We discussed how to spot the "snakes in the grass" at the "pointy end" of the operation—identifying everything from physical hazards to operational frictions like equipment downtime and supply chain vulnerabilities. But identifying a hazard is only the first step. This week, we transition to the active engine of the SMS: assessing the risk itself. In the many years I’ve been a Safety Consultant, I’ve reviewed thousands of risk assessments from the hangar floor to the boardroom. I can tell you firsthand that most of them suffer from the same fatal flaw: they are subjective guessing games disguised as objective data.
The industry standard for assessing risk is evaluating Likelihood against Severity. However, when this is done without strict, pragmatic guidance, it is wildly unreliable. The same hazard, provided to different groups, will result in varying levels of risk based purely on the specific experience, complacency, or lack of knowledge (two of the classic Human Factors "Dirty Dozen") of the people performing the assessment.
The Problem with "Likelihood"
Let’s start with what works. Severity, especially when assessing the potential recurrence of an incident, is fairly straightforward. What is the credible impact? Is it a fatality, a lost-time injury (LTI), damage to an aircraft or facility, regulatory fines, or a loss of reputation? An organization can clearly define these parameters and draw lines in the sand for what is acceptable, low, medium, high, or unacceptable. These values normally reflect the potential impact on personnel and the organization.
Likelihood, on the other hand, is a trap.
Commonly, likelihood is taught as a blended assessment of two things: the frequency of exposure to the hazard, and the effectiveness of the control measures in place. There are massive problems with the latter half of this equation.
- Do you actually have authority over those control measures?
- Do you know what the existing control measures really are on the night shift?
- Are there external influences?
Consider driving a company vehicle in city traffic. You know how to drive, and you understand the local traffic regulations (your controls). But you have zero control over the competency, sobriety, or distraction levels of the other drivers on the road. If you assess likelihood based on your own controls while ignoring external reality, your assessment is a localized fantasy.
The Fix: Decoupling Frequency and Effectiveness
You can immediately improve the reliability of a risk assessment by breaking "Likelihood" into its actual components: Frequency of Exposure and Control Effectiveness, and evaluating them separately.
Step 1: Establishing Initial Risk Frequency can be easily and objectively defined. How often are personnel actually exposed to this hazard under normal working conditions? Organizations can define parameters ranging from "extremely unlikely" to "certain to occur" based strictly on the production schedule or task inventory.
I use a standard 5x5 grid to evaluate what I call Initial Risk. This is calculated using only Frequency of Exposure multiplied by Severity of Outcome, assuming no controls are in place. This provides a raw numerical score from 1 to 25.
I break that score down into risk values from acceptable to unacceptable. The sole intent of this Initial Risk score is to prioritize the organization’s response. Risk must always be managed to a condition of As Low As Reasonably Practicable (ALARP). At the lower end, an organization must be able to document that further risk reduction is not worth the operational friction. At the upper end (Unacceptable), it is a clear trigger to stop work immediately and implement interim measures before the task can resume.
Step 2: Auditing Control Effectiveness to find Residual Risk Here is where the paradigm shifts. Assessing the effectiveness of control measures shouldn't be a guess—it must be an audit process.
To determine Residual Risk, I don't ask a committee how they "feel" about a control. I ask seven definitive questions to determine if the control measures have been implemented, are effective, and are managed to ensure continuous adherence.
- Documentation: Is the control measure formally defined in standard operating procedures (SOPs)?
- Training: Have all exposed personnel been explicitly trained and evaluated on this specific control?
- Resourcing: Are the physical tools, equipment, or PPE required for this control readily available at the point of use?
- Verification: Is there evidence (via frontline supervisor oversight) that the control is actually being used as intended?
- Hierarchy: Does the control rely on engineering/elimination rather than human behavioral compliance?
- Independence: Is the control free from external, uncontrollable variables (e.g., public interference, weather)?
- Monitoring: Is there a scheduled maintenance or review process to ensure the control does not degrade over time?
Each question answered positively reduces the Initial Risk score by a set percentage. The resulting numerical score is then applied to the risk matrix to determine a risk value.
The Result: Operational Integrity
By treating control effectiveness as an audit rather than an assumption, we remove the guesswork. The resulting Residual Risk score provides clear, data-driven guidance to the organization regarding the effort required to ensure the risk remains managed in the future.
When we stop trying to manage the abstract concept of "Safety" and start measuring the actual conditions of the work, we bridge the gap between regulatory compliance and operational reality. We stop policing paperwork and start protecting people.