AI Agents
AI in Industries
AI Models
AI Hardware
Agentic AI Frameworks
GenAI Applications
AI Foundations
Data Security
Identity & Access Management
Security Tools
Network Security
Threat Detection and Response (TDR)
Web Proxies
Web Data Scraping
Data Collection
Data Science
Synthetic Data
E-Commerce
Workload Automation
IT Automation
Process Improvement
RMM
Managed File Transfer
Helpdesk Software
OCR accuracy is critical for many document processing tasks, and SOTA multi-modal LLMs are now offering an alternative to OCR. We benchmarked leading OCR services in DeltOCR Bench to identify their accuracy levels in different document types:
The full names of the above products and their versions in use as of November 2025 are listed below. Our study covers both easily accessible API services and solutions requiring on-premises infrastructure, comparing key models in the market in a deep test environment.
The following models were included in our benchmarking list due to both their ease of access and performance.
Microsoft Azure Document Intelligence API is part of the Azure Cognitive Services family.
Testing these models is more challenging than API solutions due to installation, dependency management, and hardware requirements. All local tests were conducted in a dedicated server environment.
We calculated the accuracy of results as the cosine similarity score for printed text, printed media, and handwriting. Each score visible in the chart represents the performance of the corresponding model within that category.
During our testing, we observed that the Nanonets-OCR2-3B model delivered the weakest performance in the benchmark, achieving the lowest scores. Generally, we found that some models struggled particularly with cursive handwriting and disorganized text layouts (mixed line ordering, inconsistent capitalization). Similar performance issues also emerged in the printed media category, especially with low-resolution images and those containing multiple font styles.
We used a total of 300 documents in this benchmark, with 100 documents per category across 3 categories:
Printed text includes letters, website screenshots, emails, reports, etc.
Printed media includes posters, book covers, advertisements, etc. We aimed to see the success of the OCR tools in different text fonts and placements.
Files in these 2 categories were sourced from the Industry Documents Library (IDL).1
Handwriting: In the handwritten category, as some IDL documents were not easy to read, our team generated documents similar to the IDL documents. We manually prepared samples of human-legible handwriting. All samples were in a cursive handwriting style.
This benchmark focuses on the text extraction accuracy of the products.
Preprocessing is performed only for the handwriting category. We took pictures of handwritten documents with our smartphones and used a mobile scanner app:
OCR: We ran all the products on the same dataset and generated text outputs as raw text (.txt) files. Then, we manually prepared the ground truth including the correct text in all of these files. The ground truth was verified twice by humans.
Comparison: We measured the accuracy of OCR solutions by comparing their outputs with the original texts. For this purpose, we used the Sentence-BERT (SBERT) framework to compute cosine similarity scores. In the benchmark, we used the high-performing multilingual paraphrase model, MiniLM-L12-v2, to compute the similarity score between each product’s output and the ground-truth texts. This score represents the text accuracy level.
The similarity function uses a cosine distance metric to calculate the similarity between two texts. We did not use Levenshtein distance for this benchmark because different products output texts in different orders.2
While Levenshtein distance takes these differences into account, we are only looking for how accurately the text is detected, but not where it is located. The cosine distance has negligible penalties for such cases, so we decided to use it in this benchmark.
There are many OCR products on the market. We need to focus on the ones that can output raw text results. The products for this benchmark are chosen based on:
This is not a comprehensive market review, and we may have excluded some products with significant capabilities. If that is the case, please leave a comment, and we will be happy to expand the benchmark.
Advanced capabilities such as text location detection, key-value pairing, and document classification were not evaluated in this benchmark.
The sample size will be increased in the next iteration. If you are looking for OCR for handwriting, see our handwriting OCR benchmark with 50 samples.
You can also see our invoice OCR benchmark and receipt OCR benchmark if you are interested.
Google Cloud Platform’s Vision OCR tool achieves the highest text accuracy of 98.0% when the entire data set is tested. While all products perform above 99.2% with Category 1, where typed texts are included, the handwritten images in Category 2 and 3 create the real difference between the products.
The overall results show that GCP Vision and AWS Textract are the dominant OCR products, with the highest accuracy in recognizing the given text.
Notes from the overall results:
As mentioned in the overall results, there was a single “outlier” image where AWS Textract could not recognize any text. While the product shows more than 95% text accuracy in all other images, this instance reduced AWS’s performance and widened its confidence interval.
As this instance might be an exception, we also wanted to compare the products without it. We called this image the “troublemaker” and re-ran our results to see if they made a difference.
Here are the new results after excluding the “trouble-maker” from the dataset.
When the “trouble-maker” is excluded, AWS Textract becomes the top performer by an almost perfect (99.3%) text accuracy level with a narrow confidence interval. While the scores do not change much, GCP Vision and AWS Textract remain the top 2 products, with better text accuracy than the others.
The main factor reducing the text accuracy of certain products is the presence of handwriting in images. Thus, we excluded all images (all of category 2 and 6 images from category 3) and re-evaluated the text accuracy performance, again.
The results are more head-to-head when handwritten images are excluded. AWS Textract and GCP Vision remain the top 2 products in the benchmark, but ABBYY FineReader also performs very well (99.3%) this time. Although all products achieve over 95% accuracy when handwriting is excluded, Azure Computer Vision and Tesseract OCR still struggle with scanned documents, putting them behind in this comparison.
We tested five OCR products to measure their text accuracy performance. We used versions available as of May/2021. Used products are:
Although there are many image datasets for OCR, these are
Thus, we decided to create our own dataset under three main categories:
All input files are in .jpg or .png format.
We use other market data (e.g. software reviews, customer case studies) to rank software providers. However, since most corporates use the term “OCR” when searching for data extraction solutions (i.e., including those that generate machine-readable data), our list has a larger scope and more companies than those presented in this benchmarking exercise.
Optical Character Recognition (OCR) is a field of machine learning that specializes in distinguishing characters within images like scanned documents, printed books, or photos. Although it is a mature technology, there are still no OCR products that can recognize all kinds of text with 100% accuracy. Among the products that we benchmarked, only a few products could output successful results from our test set.
OCR tools are used by companies to identify texts and their positions in images, classify business documents according to subjects, or conduct key-value pairing within documents. Based on OCR results, other technology companies build applications like document automation. For all these business cases, accurate text recognition is critical for an OCR product.
Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.
HTML, for blog posts, LinkedIn articles & newsletters. Recommended.
For academic papers and analyst reports following APA 7th style.
For LaTeX documents and academic reference managers.
Your email address will not be published. All fields are required. Comments are left in their original language.
Did you ever think of oncluding multimodal llms in your comparison, like gpt4o, llama 3.2. gemini, claude etc.?
Hi Serhat and thank you for your comment, Yes, we added those for which we have API access like Claude and GPT-4o.
Just stumbled on this milestone assessment update. Could you kindly elaborate further on the three revised datasets: Thanks for this work. Character Sets When someone refers to 'handriting', that can mean many things: 'handwriting style' typefaces (per Docusign, etc.), and hand-printed (block printing and mixed-case printing) as often found in combs and box delineators, and finally, cursive or longhand writing (exclusive of signatures). Character Context Structured content, semi-structured content, and unstructured content. Image Qualities (bitonal, greyscale, full colour, spatial dpi, from a scanner/cell-phone/native rendering, image 'enhancements' prior to OCR (thickening, local gamma, background dropout, sharpening, smoothing, noise removal, etc.) These can have significant impacts, and some don't realize the importance of including these benchmark differentiators.
Hi there, thank you for the detailed comment, we are updating the article to include these details.
Hello, great work! Just curious, did you use a trained Tesseract when making these testing?
Hi, Webster. Glad you enjoyed the article. The tools we tested were: ABBYY FineReader 15 Amazon Textract Google Cloud Platform Vision API Microsoft Azure Computer Vision API Tesseract OCR Engine Hope this answers your question.
The graph images are not working for me at the moment. Otherwise great
Thank you Bobby! We have a glitch in the CMS and we are fixing it. Apologies for the issue, it should be fixed next week.
Thanks for sharing, can you add a free OCR for everyone to use? https://www.geekersoft.com/ocr-online.html
Hi Samsun, unfortunately, we don't share all OCR providers on this page, there are thousands of them. We tried to put together the largest ones in terms of market presence. If you have evidence that your solution is one of the top 10 globally, please share it with us at info@aimultiple.com so we can consider it.
What version of Tesseract did you test with? They recently released v5.
Hi Scott, we did the benchmarking before Tesseract 5. We will redo it soon and include the versions in the methodology section as well.
This is very informative, nice work. I assume your tests used documents/images in English? I've been experimenting with OCR tools on other languages and finding relatively poor accuracy.
Exactly, all text were in English. I hear similar things about OCR on non-Latin characters. We have an Arabic speaker in the team who claims that accuracy in Arabic is much lower compared to English. We can do a benchmark on non-Latin characters if there is demand for it.
interesting post!!! do you have any suggestion about improving accuracy on scanned image ? i'm using tesseract right now. anyway , great work!
Thank you for the comment. There are pre-processing approaches that can be implemented to improve image quality. But such approaches may already be used in Tesseract. A detailed research into Tesseract image processing would be helpful in your case.
We follow ethical norms & our process for objectivity. This research does not feature any customers of AIMultiple.