Unlimited-OCR Deep Dive: Baidu's Open Source Long-Document OCR with vLLM + Docker

1. Introduction: Why Unlimited-OCR Deserves Your Attention

In June 2026, Baidu open-sourced Unlimited-OCR — an OCR model with just 3B parameters that can parse a 100-page PDF in a single pass. As of now, GitHub stars have surpassed 26,000, making it the most watched open-source project in the OCR field.

Traditional OCR tools (Tesseract, PaddleOCR) process long documents using a "page-by-page independent recognition" strategy, clearing context with each page turn. This leads to: - Context fragmentation: Cross-page tables and paragraphs cannot be understood coherently - Redundant computation: Each page rebuilds feature extractors from scratch - KV Cache explosion: Attention cache grows linearly with document length

Unlimited-OCR solves this problem completely through an innovative R-SWA (Reference Sliding Window Attention) mechanism. This article provides deep analysis of its technical principles, latest developments (vLLM integration, enterprise services), deployment guide, and practical use cases.

Related Reading: - AI Model Deployment Guide Series - Local LLM Inference Optimization in Practice

2. Core Technology: R-SWA Attention Mechanism

2.1 The Problem: KV Cache Disaster for Long Documents

When traditional Transformers process long documents, KV Cache (Key-Value cache) grows linearly with document length. When processing a 100-page PDF, KV Cache can occupy tens of GB of VRAM, causing: - Out of memory (OOM) errors - Dramatically increased inference latency - Inability to run on consumer-grade GPUs

2.2 The Solution: R-SWA Mechanism

Unlimited-OCR's core innovation is R-SWA (Reference Sliding Window Attention), inspired by how humans read long documents:

When humans read long documents:
1. Maintain a "sliding window" of working memory (the few pages currently being read)
2. Use reference mechanisms to recall key information (important content from earlier)
3. Don't forget everything before just because we turned the page

R-SWA technical implementation:

# R-SWA core logic pseudocode
class ReferentialSlidingWindowAttention(nn.Module):
    def __init__(self, window_size=2048, ref_size=512):
        self.window_size = window_size  # Current window (local details)
        self.ref_size = ref_size        # Reference window (global context)

    def forward(self, hidden_states, past_kv):
        # 1. Current window attention (local details)
        local_attn = self.local_attention(hidden_states[-self.window_size:])

        # 2. Reference window attention (global context)
        ref_attn = self.ref_attention(past_kv[-self.ref_size:])

        # 3. Gated fusion: local + global
        gate = self.sigmoid(self.gate_proj(hidden_states))
        fused = gate * local_attn + (1 - gate) * ref_attn
        return fused

Key advantages: - Fixed memory usage: Regardless of document length, attention cache only maintains window_size + ref_size - Continuous context: Cross-page information stays coherent through reference window - Efficient inference: At 32K context length, single forward pass completes dozens of pages of parsing

2.3 MoE Architecture: 3B Total Parameters, 0.5B Activated

Unlimited-OCR uses Mixture of Experts (MoE) architecture:

MoE benefits: large model capacity (3B) but low inference cost (only 0.5B activated per step). This means it runs on consumer GPUs (like RTX 3090/4090).

3. Latest Developments: vLLM Integration and Enterprise Services

3.1 vLLM Community Support (2026-06-28)

On June 28, 2026, the vLLM community officially merged support for Unlimited-OCR, enabling vLLM inference acceleration.

vLLM advantages: - PagedAttention: Efficient KV Cache management, 2-4x VRAM utilization improvement - Continuous Batching: Dynamic batching, 3-5x throughput improvement - Tensor Parallelism: Multi-GPU parallel inference

Deploying Unlimited-OCR with vLLM:

# Install vLLM
pip install vllm

# Start vLLM service
python -m vllm.entrypoints.openai.api_server \
  --model baidu/Unlimited-OCR \
  --tensor-parallel-size 1 \
  --max-model-len 32768 \
  --port 8000

# Call API (OpenAI-compatible format)
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "baidu/Unlimited-OCR",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "image_url", "image_url": {"url": "file:///path/to/document.pdf"}},
          {"type": "text", "text": "Please parse this document"}
        ]
      }
    ]
  }'

3.2 Baidu AI Open Platform Enterprise Service

Baidu has launched Unlimited-OCR enterprise service, providing: - Managed API: No need to build your own GPU servers - SLA guarantee: 99.9% availability - Enterprise security: Data encryption, access control, audit logs

Use cases: - Teams without GPU hardware - Production environments requiring high availability guarantees - Rapid prototype validation

3.3 Community Feedback and Ecosystem

As of October 2026, Unlimited-OCR community ecosystem: - GitHub Stars: 26,000+ - Hugging Face downloads: 500,000+ - Community contributions: - Docker images (official + community versions) - LangChain integration - LlamaIndex document loader - Gradio Web UI

4. Performance Evaluation: Comparing with Commercial OCR Services

4.1 OmniDocBench Benchmark Tests

4.2 Long Document Parsing Capability Comparison

4.3 Speed Comparison

50-page PDF processing time: - Unlimited-OCR (vLLM): ~15 seconds (1 inference) - Unlimited-OCR (native): ~45 seconds (1 inference) - Google Vision API: ~50 seconds (50 API calls) - PaddleOCR: ~750 seconds (50 inferences) - Tesseract: ~1250 seconds (50 inferences)

Conclusion: vLLM-accelerated Unlimited-OCR is fastest for long document scenarios, and lowest cost.

5. Deployment Guide: Three Methods

5.1 Method 1: Hugging Face Transformers (Simplest)

# Install dependencies
pip install transformers torch pillow

# Python code
from transformers import AutoModelForVision2Seq, AutoProcessor
from PIL import Image
import torch

# Load model
model = AutoModelForVision2Seq.from_pretrained(
    "baidu/Unlimited-OCR",
    torch_dtype=torch.float16,
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("baidu/Unlimited-OCR")

# Process document
image = Image.open("document.pdf")  # Supports PDF, images
inputs = processor(images=image, text="Parse this document", return_tensors="pt")
inputs = {k: v.to(model.device) for k, v in inputs.items()}

# Inference
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=32768)

result = processor.decode(outputs[0], skip_special_tokens=True)
print(result)

5.2 Method 2: vLLM Acceleration (Recommended for Production)

# Install vLLM
pip install vllm

# Start service
python -m vllm.entrypoints.openai.api_server \
  --model baidu/Unlimited-OCR \
  --tensor-parallel-size 1 \
  --max-model-len 32768 \
  --port 8000

# Batch processing script
import requests
import base64
import os

def ocr_document(pdf_path):
    with open(pdf_path, "rb") as f:
        pdf_base64 = base64.b64encode(f.read()).decode()

    response = requests.post(
        "http://localhost:8000/v1/chat/completions",
        json={
            "model": "baidu/Unlimited-OCR",
            "messages": [{
                "role": "user",
                "content": [
                    {"type": "image_url", "image_url": {"url": f"data:application/pdf;base64,{pdf_base64}"}},
                    {"type": "text", "text": "Parse this document to Markdown format"}
                ]
            }]
        }
    )
    return response.json()["choices"][0]["message"]["content"]

# Batch processing
for pdf_file in os.listdir("./documents"):
    if pdf_file.endswith(".pdf"):
        result = ocr_document(f"./documents/{pdf_file}")
        with open(f"./results/{pdf_file}.md", "w") as f:
            f.write(result)
        print(f"Processed: {pdf_file}")

5.3 Method 3: Docker Containerized Deployment

# Pull official image
docker pull baidu/unlimited-ocr:latest

# Start service
docker run -d \
  --name unlimited-ocr \
  --gpus all \
  -p 8000:8000 \
  -v $(pwd)/models:/models \
  -v $(pwd)/results:/results \
  baidu/unlimited-ocr:latest

# Process document
docker exec unlimited-ocr python infer.py \
  --model /models/Unlimited-OCR \
  --input /results/document.pdf \
  --output /results/output.md \
  --mode multi

# Docker Compose configuration
cat > docker-compose.yml << 'EOF'
version: '3.8'
services:
  unlimited-ocr:
    image: baidu/unlimited-ocr:latest
    ports:
      - "8000:8000"
    volumes:
      - ./models:/models
      - ./results:/results
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    environment:
      - MODEL_PATH=/models/Unlimited-OCR
      - MAX_MODEL_LEN=32768
EOF

docker-compose up -d

6. Practical Use Cases

6.1 Case 1: 100-Page Technical Manual Parsing

Scenario: Convert a 100-page scanned technical manual to editable Markdown.

# Input: scanned_manual.pdf (100 pages)
# Output: editable_manual.md

python infer.py \
  --model baidu/Unlimited-OCR \
  --input scanned_manual.pdf \
  --output editable_manual.md \
  --mode multi \
  --format markdown \
  --context-length 32768

# Time: ~90 seconds (vLLM accelerated)
# Output quality: Heading hierarchy, code blocks, table structures all correctly recognized

6.2 Case 2: Financial Report Cross-Page Table Extraction

Scenario: Financial reports contain large tables spanning multiple pages — traditional tools truncate them.

from unlimited_ocr import UnlimitedOCR

ocr = UnlimitedOCR(model_path="baidu/Unlimited-OCR")

# R-SWA's continuous context ensures complete cross-page table extraction
result = ocr.extract_tables(
    input_path="financial_report.pdf",
    output_format="csv",
    context_length=32768
)

# Output: Complete cross-page tables, no truncation
result.to_csv("extracted_tables.csv")

6.3 Case 3: Batch Academic Paper Parsing

Scenario: Batch parse 100 academic paper PDFs, extracting abstracts, methods, and experimental results.

#!/bin/bash
# batch_parse_papers.sh

INPUT_DIR="./papers"
OUTPUT_DIR="./parsed"
MODEL="baidu/Unlimited-OCR"

mkdir -p "$OUTPUT_DIR"

for pdf in "$INPUT_DIR"/*.pdf; do
    filename=$(basename "$pdf" .pdf)
    echo "Processing: $filename"

    python infer.py \
      --model "$MODEL" \
      --input "$pdf" \
      --output "$OUTPUT_DIR/${filename}.md" \
      --mode multi \
      --extract-structure  # Extract titles, abstracts, sections

    echo "Done: $filename"
done

echo "All papers processed."

7. Summary and Recommendations

7.1 Unlimited-OCR's Core Advantages

• ✅ R-SWA attention mechanism: 32K context continuous parsing, solving long document OCR pain points
• ✅ MoE architecture: 3B total parameters, 0.5B activated, runs on consumer GPUs
• ✅ SOTA accuracy: OmniDocBench 93.92%
• ✅ vLLM acceleration: 3-5x throughput improvement for production environments
• ✅ Fully open source: Apache 2.0 license, no API call fees

7.2 Use Cases

Recommended for: - Long document batch parsing (technical manuals, financial reports, academic papers) - Cross-page table extraction - Multilingual mixed document recognition - Scenarios with high data privacy requirements (local deployment)

Not recommended for: - Single-page simple text (Tesseract is lighter) - Real-time video stream OCR (high latency) - No GPU environments (CPU inference too slow, recommend Baidu enterprise service)

7.3 Next Steps

1. Quick experience: Use Hugging Face online Demo
2. Local deployment: Refer to Section 5 deployment guide
3. Production environment: Use vLLM acceleration + Docker containerization
4. Enterprise service: Visit Baidu AI Open Platform

Reference links: - GitHub repository: baidu/Unlimited-OCR - Hugging Face model: baidu/Unlimited-OCR - vLLM documentation: vLLM official docs - Baidu enterprise service: Unlimited-OCR Enterprise Edition