KPI
Detection improvement: AI benchmark
Increase in genuine fraud, risk or defects detected.
How to measure it
True positives detected per period or recall on a labelled sample, before and after.
Across the library
Median 82.5% across 4 deployments, reported range 25% to 300%.
Median 5x across 3 deployments, reported range 2.5x to 14x.
Higher is better. Unit: percent.
Reported values by use case
AI scam intervention for instant payments
- Starling Bank300% · vendor claim
- Vodafone30% · organization claim
AI computer vision for store shelf and stock monitoring
- Schnuck Markets14x · organization claim
AI for application and identity fraud detection
- Department for Work and Pensions2.5x · organization claim
AI for benefit fraud and error detection in social security
- Department for Work and Pensions2.5x · organization claim
AI for IT incident triage and root cause analysis (AIOps)
- TD Bank25% · vendor claim
AI for merchant underwriting and risk monitoring
- Visa5x · organization claim
AI for telecom fraud detection (SIM swap, IRSF and Wangiri)
- Vodafone30% · organization claim
Real time fraud scoring for card and instant payments
- NatWest Group135% · vendor claim
Use cases that should track detection improvement
- AI assisted proctoring and integrity monitoring for remote exams
- AI clinical trial patient matching and prescreening
- AI computer vision for store shelf and stock monitoring
- AI content moderation for trust and safety at online platforms
- AI early warning and covenant monitoring for loan portfolios
- AI early warning for sepsis and in hospital clinical deterioration
- AI for AML transaction monitoring alert triage
- AI for application and identity fraud detection
- AI for benefit fraud and error detection in social security
- AI for customs risk targeting, container selection and valuation checks
- AI for fee and interest leakage detection
- AI for healthcare claims fraud, waste and abuse detection
- AI for insurance claims fraud detection
- AI for IT incident triage and root cause analysis (AIOps)
- AI for market abuse surveillance alert triage
- AI for merchant underwriting and risk monitoring
- AI for money mule account and network detection
- AI for predictive network maintenance in telecom
- AI for risk based inspection prioritization in food safety, workplace and environmental regulation
- AI for tax compliance risk scoring and audit selection
- AI for telecom fraud detection (SIM swap, IRSF and Wangiri)
- AI predictive maintenance for industrial and energy assets
- AI prioritization of radiology and imaging worklists
- AI quality inspection on the production line
- AI risk scoring in child welfare intake and investigations
- AI scam intervention for instant payments
- AI screening of trade finance transactions for trade based money laundering
- AI supply chain risk and disruption monitoring
- AI support for emergency call triage (112 and 911)
- AI video analytics and screening for physical security
- Dynamic AML customer risk rating with machine learning
- Real time fraud scoring for card and instant payments