FineBooks benchmarks 14 open OCR models on historical texts to fix LLM training data degradation
A joint initiative by Hugging Face and EleutherAI titled FineBooks evaluated 14 open-source OCR models across 2,165 historical book pages to solve data degradation in LLM pre-training corpora. Legacy OCR errors on public domain books have been shown to degrade language model efficiency by up to 30% compared to clean text. The benchmark revealed that modern vision-language models achieve over 97% character accuracy at a processing cost under $2 per 1,000 pages. The top performer was dots.mocr, a 3-billion parameter model that outperformed models three times its size, followed by OvisOCR2 operating at just $0.46 per 1,000 pages. The project aims to re-process massive open archives like the Biodiversity Heritage Library's 64 million pages, making high-quality historical texts available for open-source AI training without relying on degraded scans.
Old OCR text cripples language model training, and FineBooks wants to fix that at scale