A useful tool is not the same as a competent decision-maker
AI is already moving into the everyday machinery of work. It can help organisations analyse incident data, identify recurring hazards, monitor equipment, flag unusual patterns and produce first drafts or supporting material for risk assessments, procedures or training materials. Used well, these tools can reduce administrative burden and help safety professionals focus their attention where it is most valuable.
The difficulty is that AI does not need to be obviously broken to be dangerous. A system may produce an answer that is polished, specific and plausible while omitting a critical control, misreading a legal requirement or applying information that does not fit the actual task, workforce or site. In workplace safety, an error that sounds authoritative may be more hazardous than one that is visibly nonsensical because it is less likely to be questioned.
The human factor: automation bias
People tend to place undue confidence in recommendations produced by automated systems, particularly when the technology appears sophisticated or has been correct before. This is known as automation bias. Time pressure, workload and unclear accountability can strengthen the effect: checking the output begins to feel like duplication, and the human reviewer gradually becomes a rubber stamp.
This creates a subtle shift in responsibility. Instead of using AI as one source of information, the organisation starts treating it as the decision-maker while a person simply approves the result. The label ‘human in the loop’ offers little protection if that person lacks the technical competence, operational knowledge, time or authority to intervene. Effective oversight must include the ability to recognise limitations, seek contrary evidence, ask for specialist input and stop the process when something does not look right.
AI does not remove employer duties
Technology does not displace an employer’s existing duties. Under the Safety, Health and Welfare at Work Act 2005, employers must, so far as is reasonably practicable, ensure employees’ safety, health and welfare, provide safe systems of work, and identify hazards and assess risks. An AI-generated risk assessment may support that process, but it cannot inspect the workplace, understand informal workarounds, consult employees or accept legal responsibility on the employer’s behalf.
The EU AI Act points in the same direction. Its AI-literacy provisions require providers and deployers to take measures that support the development of AI literacy among staff and others dealing with AI systems on their behalf. For high-risk systems, human oversight requirements are intended to help ensure that people understand system limitations, remain alert to automation bias, interpret outputs correctly and override or stop the system where needed. Not every safety-related AI tool will fall into the high-risk category, and relevant high-risk tools have later application dates, but the principles offer a benchmark for organisations considering their approach to AI oversight and governance.
From nominal oversight to meaningful control
Before AI is used in a safety-critical process, employers should define what the tool may do, what it must never decide alone and who remains accountable. Controls should be proportionate to the potential consequences of error. Drafting a toolbox talk, for example, does not carry the same risk as recommending whether machinery is safe to restart or prioritising corrective actions after a serious incident.
In practice, meaningful human oversight should be built around six core controls:
- Set approved uses and clear boundaries. Identify where AI may assist and where qualified human judgement is mandatory.
- Require verification against authoritative sources, current legislation, manufacturer instructions and site-specific evidence.
- Assign a competent reviewer with enough time and authority to challenge, amend, escalate or reject the output.
- Protect the quality and confidentiality of data entered into AI systems, and test outputs for gaps, bias, false confidence and unintended impacts on workers.
- Record material AI use and the basis of safety-critical decisions so that the process can be explained, audited and improved.
- Consult workers and safety representatives, then monitor how the system affects workload, behaviour, reporting and risk in practice.
Keeping people at the centre of safer work
The strongest approach is neither to reject AI nor to trust it reflexively. It is to combine the speed and pattern-recognition of technology with the experience, curiosity and moral responsibility of people. Organisations that establish this discipline early will be better placed to benefit from AI without allowing convenience to weaken their safety systems.
The essential question is not simply whether an AI tool can produce an answer. It is whether the organisation can verify that answer, explain the decision that follows and identify the person who remains responsible. In workplace safety, accountability cannot be automated away.
How NFP can help
As AI becomes part of everyday work, organisations need practical governance that connects new technology with existing safety responsibilities. NFP can support employers in reviewing risk-management arrangements, strengthening policies and procedures, clarifying roles and building proportionate controls that keep competent human judgement at the centre of safety-critical decisions.