Task-specific evaluation
We evaluate the structures that matter
An OCR system can score well on general text metrics and still fail the business workflow.
Two systems may extract almost identical text from a table while one places values under the wrong columns.
- Text quality
- Character error rate, word error rate, missing text, duplicated content, and reading order where relevant.
- Table quality
- Correct rows, columns, headers, empty cells, merged structures, and value placement.
- Extraction quality
- Accuracy of the specific fields, entities, values, or classifications needed by the workflow.
- Visual information
- Whether important charts, diagrams, and other non-text elements are captured in a usable form.
- Operational performance
- Processing latency, cost, failure rate, throughput, and dependence on external APIs.
Evaluation should reflect the actual document structure and downstream use case.