What 97% reconciliation accuracy really takes
Most document-automation demos quietly skip the accuracy problem. Here's the rule-based engine behind a 97%-accurate air-cargo reconciliation, and why good-enough OCR isn't.
- Raw OCR on messy operational documents lands around 80–90% field accuracy, nowhere near safe enough to reconcile money without a human checking every line.
- The jump to 97% comes from a validation layer on top of OCR, not a better OCR model. It cross-checks against source systems, applies business rules, and scores confidence so only the doubtful lines go to a human.
- The goal is not 100% automation. It's automating the 90%+ of lines the system is confident about and escalating the rest, so one person reviews exceptions instead of keying everything.
- For one logistics client this turned a 5-day, twice-monthly reconciliation into a 2-hour job, and freed a full-time person for work that needs judgment.
Everyone selling document automation shows you the same demo: a clean invoice goes in, perfect structured data comes out. Then you point it at your actual paperwork (a smudged airway bill, a scanned statement, a form someone filled in by hand) and the accuracy falls off a cliff.
So when a client asks whether AI can “just read our documents and reconcile them,” the honest answer is: reading them is the easy 80%. The last 17 points, the ones that decide whether you can trust the numbers with money, are the whole job. Here’s what getting to 97% on a real air-cargo reconciliation actually took.
Why raw OCR gets you to 85%, not 97%
Modern OCR is genuinely good. On a clean, templated document it will pull fields with high accuracy. But operational documents are not clean or templated. Airway bills come from dozens of carriers in different layouts. Numbers get handwritten over printed fields. Scans are skewed, faxed, photographed at an angle.
On that kind of input, field-level OCR accuracy realistically lands somewhere around 80–90%. That sounds close. It isn’t. In a reconciliation, one wrong digit in a charge or a weight doesn’t round away: it becomes a discrepancy someone has to hunt down, or worse, a number that quietly ships wrong. At 85% accuracy across a document with a dozen fields, almost every document has at least one error. You haven’t removed the manual work. You’ve moved it from typing to hunting.
The mistake is treating OCR as the product. OCR is a raw input. The accuracy lives in what you build on top of it.
The validation layer that closes the gap
The jump from ~85% to 97% didn’t come from a better OCR model. It came from a rule-based validation engine sitting between extraction and the ledger. Three things do the work:
Cross-checks against source systems
Every extracted field that can be verified against another system, is. In one build, extracted airway-bill data was reconciled line by line against the operational source-of-truth system. When two independent sources agree, confidence in that line is high. When they disagree, you’ve found exactly the line that needs a human, automatically.
Business rules that encode what “valid” means
An experienced clerk knows a charge that’s an order of magnitude off is a typo, that certain fields must sum to a total, that a date can’t precede a shipment. Those instincts become explicit rules: range checks, checksum-style totals, format and cross-field constraints. Most OCR errors violate a rule the moment they appear, so the rule catches them before they reach the ledger.
A confidence score that decides what a human sees
Every reconciled line gets a confidence score from the checks above. High-confidence lines flow straight through. Low-confidence lines are routed to a person. This is the piece most demos skip, and it’s the most important: the system knows what it doesn’t know, and escalates only that.
Why 100% automation is the wrong target
Chasing 100% is how these projects blow their budget. The last handful of genuinely ambiguous documents (the water-damaged scan, the non-standard carrier, the handwritten correction) cost more to automate reliably than a person spends handling them in a minute.
So the target was never “no humans.” It was: automate every line the system is confident about, and put one person on the exceptions. That’s what 97% buys you. Not a system that never needs a human, but one where a human reviews the 3% that’s genuinely doubtful instead of re-keying 100% of the work. The savings come from changing the ratio, not from removing people entirely.
What it changed for the client
The reconciliation ran twice a month and took a full-time person the best part of five days each cycle: pulling airway bills, keying them, matching every line against the operational system by hand, chasing discrepancies.
After the build, that same cycle runs in about two hours. The pipeline extracts, reconciles and cross-checks the full volume, holds 97% accuracy through the validation layer, and surfaces only the flagged lines for review. The person who used to do the keying moved to work that actually needs their judgment. Scope to deployed took under two months.
None of that is because the OCR was magic. It’s because the accuracy problem was treated as the real problem, and solved with cross-checks, rules, and honest confidence scoring rather than a hope that the model would be right.
How to judge an automation pitch
If you’re evaluating a document-automation project, the tell is simple. Ask what happens to the documents the system gets wrong. If the answer is a hand-wave about model accuracy, keep your money. If the answer is a specific story about validation, cross-checks and how low-confidence cases get escalated to a person, you’re talking to someone who has actually shipped one.
Good-enough OCR isn’t good enough. The engine around it is the product.
Frequently asked questions
- Can OCR alone reach 97% accuracy on reconciliation?
- Rarely. On clean, templated documents OCR can be very accurate, but operational paperwork like airway bills, invoices and statements is messy, so raw OCR typically lands around 80–90% field accuracy. Reaching 97% comes from a validation layer on top of OCR (cross-checks against source systems, business rules, and confidence scoring), not from the OCR model by itself.
- What does "97% accuracy" actually mean here?
- It means 97% of extracted-and-reconciled line items matched the source system correctly without human intervention. The remaining 3% weren't wrong and shipped anyway. They were flagged by the confidence engine and routed to a person to check, so errors were caught rather than silently passed through.
- Why not aim for 100% automation?
- Because the last few percent of messy, ambiguous documents cost far more to automate than a human spending a few minutes on them. Automating the confident 90%+ and escalating the doubtful lines gives you most of the savings at a fraction of the risk and build cost.
- How long does a build like this take?
- The reconciliation went from scope to deployed in under two months, working as a forward-deployed partner embedded with the team.