baidu/Unlimited-OCR: A New Option for Long-Document OCR

Baidu recently open-sourced a new OCR model: baidu/Unlimited-OCR.

It is not a lightweight OCR tool for simply recognizing text in one image. It is closer to long-document parsing: converting papers, reports, books, and scanned PDFs into readable, editable text or Markdown while preserving as much structure as possible.

If you care about PDF-to-Markdown, long-document OCR, or private local document parsing, this model deserves a place on your shortlist.

It Solves Long-Document Parsing, Not Single-Page OCR

Traditional OCR often handles one image at a time: screenshots, IDs, invoices, forms, and receipts.

Real document parsing is often harder:

• A PDF may have dozens or hundreds of pages.
• Pages may contain multi-column layout, tables, titles, footnotes, and references.
• Content may mix Chinese, English, formulas, code, and captions.
• Users do not want isolated text boxes; they want structurally coherent Markdown.

This is where baidu/Unlimited-OCR fits. It is closer to a document transcription model. The goal is not only recognizing characters, but also preserving reading order and structure.

The paper Unlimited OCR Works positions it as one-shot long-horizon parsing: handling longer document parsing tasks in one pass rather than focusing only on single-page image recognition.

References:

• Hugging Face: baidu/Unlimited-OCR
• GitHub: baidu/Unlimited-OCR
• Paper: Unlimited OCR Works

Key Strength: More Stable Long Output

A core issue in long-document OCR is that the longer the output, the slower inference becomes and the more GPU memory it consumes.

Many multimodal models perform well on the first few pages, then slow down, lose structure, repeat content, miss sections, or become unstable under context and KV-cache pressure.

An important design in baidu/Unlimited-OCR is Reference Sliding Window Attention, mentioned in the paper. In simple terms, it aims to control KV-cache growth during long output so decoding pressure remains more stable.

This means the focus is not only “can it recognize accurately?” but also “can it keep outputting across long documents?”

For real use, that matters. The biggest problem in PDF parsing is often not failing on the first page, but breaking halfway through a long document.

Model Size and GPU Requirements

baidu/Unlimited-OCR is a 3B MoE model. The paper and official notes mention around 0.5B active parameters during inference.

This means it does not require the same GPU threshold as some large multimodal models. According to the vLLM official recipe, a single GPU with more than 8GB VRAM can run BF16 inference.

Reference:

• vLLM Recipe: baidu/Unlimited-OCR

A practical memory estimate:

So if you are testing locally, an 8GB GPU can try it. If your goal is stable long-PDF parsing, I would start from 16GB VRAM.

The reason is simple: being able to run the model does not mean your real task is comfortable. Image resolution, page count, max_tokens, concurrency, and vLLM KV-cache reservation all affect actual memory usage.

Benchmarks Look Good, But Test Your Own Documents

The official paper reports strong results on OmniDocBench.

On OmniDocBench v1.5, the paper reports an overall score of 93.23; on v1.6, 93.92. The same table compares it with DeepSeek-OCR, DeepSeek-OCR 2, Qianfan-OCR, Logics-Parsing-v2, and others.

This shows it is competitive on public document parsing benchmarks.

But OCR has an old problem: good benchmark results do not guarantee good results on your documents.

Real documents often include:

• Low-resolution scans.
• Skewed pages.
• Watermarks.
• Faint table lines.
• Mixed two-column or three-column layouts.
• Heavy image compression.
• Handwritten notes.
• Formulas, footnotes, headers, and footers.

All of these can significantly affect results. Whether it fits your use case should be decided with your own sample set.

Good Use Cases

I would first test baidu/Unlimited-OCR in these scenarios:

1. PDF to Markdown

This is the most interesting use case.

For papers, reports, ebooks, white papers, and manuals, if you want to convert them into Markdown for RAG, knowledge bases, search indexing, or editing, Unlimited-OCR is worth testing.

2. Private Local Document Parsing

Many documents should not be uploaded to cloud OCR: contracts, internal reports, financial materials, technical documents.

If you have a local GPU and are willing to deploy your own inference service, it gives you more control than cloud OCR.

3. Long-Document Batch Processing

If your task is not occasional image recognition but continuous PDF processing, its long-output design, vLLM support, and relatively low VRAM threshold are attractive.

4. OCR as RAG Preprocessing

The first step in many RAG projects is not vector retrieval. It is turning PDFs into usable text.

Poor document parsing damages chunking, retrieval, and Q&A later. Unlimited-OCR can be tested as an OCR / parsing component in a RAG ingestion pipeline.

Poor Fits

It is not universal.

1. Simple Screenshot OCR

For one UI screenshot, a short paragraph, or a few lines of error text, system OCR, PaddleOCR, Tesseract, cloud OCR, or general multimodal models are often enough. A long-document model may be unnecessary overhead.

2. Invoice, ID, or Form Field Extraction

Unlimited-OCR is more about transcription and parsing than specialized structured document extraction.

If you need stable extraction of invoice numbers, amounts, tax IDs, ID numbers, or contract fields, you will usually need downstream structured extraction, rule validation, or business templates.

3. Zero Tolerance for Deployment Complexity

This is a model that requires engineering deployment. Official examples involve Transformers, vLLM, SGLang, prompt formats, and inference parameters.

If you want a web product where you upload a PDF and get a result immediately, this is not an out-of-the-box SaaS.

4. Production OCR with Strong SLA

It is a newly open-sourced model, and ecosystem, issue feedback, and best practices are still developing. Production use should be rolled out carefully and not replace existing pipelines only because the paper score looks good.

How to Compare It with PaddleOCR and DeepSeek-OCR

Roughly:

• PaddleOCR: mature traditional OCR toolkit, good for detection, recognition, layout analysis, and structured task composition.
• DeepSeek-OCR: multimodal OCR direction, suitable when OCR and semantic understanding need to work together.
• baidu/Unlimited-OCR: emphasizes long-document parsing and long-output stability, suitable for PDFs and multi-page documents to Markdown.

It does not simply replace the others. It fills a specific position: long-document OCR / document parsing.

If your main task is “what text is in this image?”, it may not be the lightest choice.

If your main task is “turn this dozens-page PDF into readable Markdown,” it is worth trying.

Details to Watch During Use

1. Follow the official prompt format
   The vLLM recipe notes that the prompt must start with <image>, otherwise output may be empty.

2. Do not blindly increase max_tokens
   Long documents need enough output length, but excessive max_tokens increases memory and latency pressure.

3. Control image resolution first
   When converting PDF pages to images, too low DPI hurts recognition, while too high DPI increases compute cost.

4. Keep a failure sample set
   Maintain hard samples: low resolution, skew, watermarks, complex tables, multi-column layout. Use them for regression whenever the model changes.

5. Post-process structured tasks
   If the final goal is field extraction, do not rely only on OCR output. Add JSON schema extraction, rule validation, and human sampling.

My Take

baidu/Unlimited-OCR is an open-source OCR model worth watching, but its strongest area is long-document parsing, not simple screenshot recognition.

Its strengths are clear:

• Strong long-document transcription.
• Good public benchmark performance.
• 3B MoE with around 0.5B active parameters, making GPU requirements relatively friendly.
• 8GB VRAM can try it; 16GB is better for stable long PDFs.
• Supports Transformers, vLLM, SGLang, and other inference paths.
• MIT License, convenient for open-source and commercial evaluation.

Its limits are also clear:

• New model, still accumulating ecosystem maturity.
• Deployment and parameter tuning require engineering experience.
• Not necessarily ideal for simple OCR or strongly structured invoice extraction.
• Real performance must be validated on your own document set.

If your goal is to convert PDFs, papers, reports, and books into Markdown for knowledge bases or RAG, baidu/Unlimited-OCR should be tested early.

If your goal is only screenshot OCR or invoice field extraction, it may not be the simplest choice.

The selection rule is simple: try it for long-document parsing; do not blindly use it for ordinary OCR; before production replacement, benchmark with your own samples.