PPE Detection Accuracy: How a Pilot Should Measure It

At the end of a PPE detection pilot, an HSE director should hold three numbers for every rule on every camera: how often the alerts were right, how many real violations were caught, and how many false alarms the control room received per day. With those numbers, deploy, adjust or stop is a one-meeting decision. Without them, it is months of arguing over a demo video.

This article sets out how PPE detection accuracy should be measured on a Saudi industrial site: plants, terminals, fabrication yards, turnarounds and shutdowns. It is written for HSE, operations and IT leaders who have seen a polished demo and want to know whether it will hold up on their own cameras.

Why PPE detection is an HSE question first

PPE in the Kingdom is a legal duty on both sides of the employment relationship. Under the Labor Law, as published by the Ministry of Human Resources and Social Development:

• Article 122 requires every employer to take the precautions needed to protect workers from hazards arising from work and machinery, and prohibits charging workers for that protection.
• Article 123 requires the employer to inform workers of the hazards of their occupation before they start, oblige them to use the prescribed protective equipment, provide appropriate PPE and train them in its use.
• Article 124 requires workers to use the protective equipment designated for each operation and to follow the instructions issued for their safety.

Many industrial operators also run an occupational health and safety management system aligned with ISO 45001, which is built on hazard identification, risk assessment, worker participation and incident investigation.

PPE detection does not change these duties. It gives the employer continuous evidence of whether the controls are working, zone by zone and shift by shift, instead of a snapshot from the last safety walk.

Define the rule before you test the model

Most disputes in a pilot are not about the model. They are about what counts as a violation. Write the rules down before the first clip is reviewed.

• What does "worn" mean? A hard hat on the head, not in the hand or clipped to a belt. A vest fastened, not draped over a shoulder.
• Where does each rule apply? A PPE matrix by zone: hard hat and high-visibility vest across the process area; safety glasses at grinding stations; harness at height; full coveralls inside specific units.
• Who is exempt? Visitors on an escorted route, drivers who stay in the cab, staff in a marked walkway.
• How long before it counts? A worker who takes off a helmet to wipe sweat for five seconds is not the same as one working without it for five minutes. Agree a minimum duration.

Then check what the cameras can physically see.

PPE item	Detectable from typical site CCTV?	Notes
Hard hat	Yes, at most working distances	Colour can also indicate role if your site uses a colour code
High-visibility vest	Yes	Reflective strips under night infrared need specific testing
Coveralls	Usually	Depends on contrast between coverall and surroundings
Safety glasses	Only on close cameras	Small and often hidden by a helmet brim from overhead angles
Gloves	Only on close cameras	Best checked at fixed workstations
Harness	Often, at height	Lanyard attachment is harder to confirm than the harness itself

A rule that the camera cannot see is not a model problem. It is a camera placement decision, and it should be scoped as one.

The measures that matter

Ground truth comes first

Accuracy is always measured against something. In a pilot, that something is a sample of your own footage reviewed by people. Each event is labelled: a real violation, a compliant worker, or unclear. The review should follow a written labelling guide, ideally with two reviewers and a third to settle disagreements. Without this, every accuracy figure is an opinion.

Precision, recall and why "accuracy" misleads

• Precision answers: when the system raised an alert, how often was it a real violation? Low precision means the control room drowns in false alarms.
• Recall answers: of all real violations in the sample, how many did the system catch? Low recall means violations go unseen while the dashboard looks calm.

A single "accuracy" figure hides this trade-off. On a well-run site, most workers are compliant most of the time. A system that never raises an alert would score very high "accuracy" and catch nothing. Ask for precision and recall separately, per rule.

Detection models also report where they see an object, as a box around it. Matching those boxes to the reviewed boxes uses an overlap measure called intersection over union. For an HSE decision, though, what matters is the event: did a real violation produce one timely alert?

False alarms per camera per day

This is the operational measure that decides adoption. Every false alarm costs an operator attention, and multiplied across dozens of cameras it adds up fast. Past a point the channel is ignored, and real alerts are ignored with it. Agree the target with the people who will receive the alerts, not with the vendor.

Time to alert and uptime

• Time to alert. Seconds from the violation starting to the alert arriving. For a person entering a live lifting zone, seconds matter. For a daily compliance report, they do not.
• Usable hours. The share of hours each camera produced footage the system could analyse. Dust, glare, a lens covered in grime and a camera knocked out of position all count against it.

Building a fair test

A pilot is only as honest as its sample. Insist on the following.

• Your footage, your cameras. Not a curated clip set from another site.
• All conditions. Day and night shifts, summer heat haze, dust events, low sun at the start and end of the day, rain on the lens in winter.
• All populations. Your own staff and contractors, in the PPE colours and styles actually issued. Turnarounds bring many contractors in unfamiliar kit; include that period if you can.
• Cameras held back. At least one camera the system was not tuned on, to show how it performs on a new view.
• No cherry-picking. The sample is drawn by time window, agreed in advance, not chosen after results are known.

Reading the results

Look at results per camera and per rule, not just the site average. A strong average can hide one camera that generates most of the false alarms.

Then read the failure analysis. Every missed violation and every false alarm should be tagged with a cause: occlusion behind pipework, distance, backlight, a reflective surface mistaken for a vest, a worker partly out of frame. Each cause points to a fix: a camera angle, a zone boundary, a minimum duration, or retraining on site footage.

Finally, choose the operating point. Raising the confidence threshold improves precision and lowers recall; lowering it does the opposite. That choice belongs to HSE, informed by the data, and may differ by zone. An exclusion zone near a crane may warrant more alerts; a general walkway may warrant fewer.

From alert to action

Detection is half the system. The other half is what happens next.

• Routing. Alerts go to the control room, the area supervisor on WhatsApp or SMS, or both, by zone and severity.
• Human check. A short clip lets the receiver confirm the event in seconds before acting.
• Record. Confirmed events flow into your HSE register, with time, zone and clip, faces blurred.
• Review. Weekly trends feed toolbox talks and the PPE matrix itself.

SDAIA's AI Ethics Principles call for AI systems to be monitored periodically and continuously, and for human oversight where automated decisions are hard to reverse or could affect lives. The NIST AI Risk Management Framework organises the same idea under its govern, map, measure and manage functions. In practice: keep measuring precision, recall and false alarms after go-live, not only in the pilot.

Site constraints: networks, privacy and approvals

• Operational technology networks. The NCA's Operational Technology Cybersecurity Controls extend the Essential Cybersecurity Controls to industrial control systems. Edge hardware that reads cameras should sit on a segment agreed with your OT security team, never bridged onto control networks.
• Security camera law. Oil and petrochemical facilities are listed among the categories covered by the Law of Using Security Surveillance Cameras (official Arabic text). Its Implementing Regulation permits entities that are not otherwise authorised to analyse security camera recordings only where people are not identified or disclosed, after approval from the Public Security Directorate. Your legal team should confirm how that applies to the cameras in the pilot.
• Personal data. The Personal Data Protection Law treats photos and videos of an individual as personal data.

Our Vision AI practice is built for these conditions. Models run on edge hardware on site. Only events and short clips leave the site. Faces are blurred by default, and there is no face matching unless requested and lawful. Hosting is on premises or in a Saudi data centre. The system is designed to support PDPL and NCA requirements.

A pilot plan in four to six weeks

1. Week 1. Agree rules, zones, the PPE matrix and targets for precision, recall and false alarms per camera per day. Connect to the selected cameras.
2. Weeks 2 to 3. Run on live footage. Build the reviewed ground-truth sample across shifts and conditions.
3. Weeks 4 to 5. Measure, tag every failure by cause, adjust angles and thresholds, and re-measure.
4. Final week. Report per camera and per rule, with a recommendation: deploy, adjust and re-test, or stop.

The scope and price are fixed and agreed before the pilot starts. The assessment before it is free. For more on how vision supports HSE across plants, terminals and yards, see computer vision for oil, gas and industrial sites, and for non-PPE uses of the same cameras, read what your existing CCTV cameras can already tell you.

Show us one camera

Pick one camera over a zone where PPE matters. Send us ten minutes of footage. We will return a free read-out: which PPE items the camera can see, the measures it can support and the limits of its angle and light.

Show us one camera and get your first accuracy conversation grounded in your own footage.