Document → Markdown extraction (docling-serve)

Layout-aware “file in → clean Markdown out” for Sunny uploads, publications/ literatures, and unattended jobs (retailer onboarding workbooks, supplier datasheets). Built 2026-07-17 from the AI-resources review pick (Basecamp todo).

Spike: docling vs markitdown (2026-07-17)

Both candidates were run over six real Heatwave files — a WarmlyYours spec sheet and the Ember radiant-panel installation manual (public publications), a Wayfair invoice from a support case, a scanned tax-exemption certificate, and two retailer workbooks (wayfair.ca.xlsx, home-depot-aged-invoices.xlsx).

File (type)	markitdown 0.1.x	docling 2.107
Spec sheet PDF (label/value spec table)	Table structure lost — labels and values emitted as disconnected paragraph runs; pairing unrecoverable	Proper 2-column Markdown table, every label↔value pair intact
Installation manual PDF (12 pp, diagrams)	Plain text, ~3 s	Full structure, headings + tables, ~28 s CPU
Scanned PDF (tax certificate)	0 bytes — no OCR at all	OCR’d automatically (easyocr); text + checkbox items usable, minor glyph noise
Support-case invoice PDF	Text only (had a text layer)	Invoice fields + line-item table reconstructed; confidence “good”
Retailer workbooks (.xlsx)	Clean tables, slightly nicer float rendering	Clean tables (raw floats occasionally verbose, e.g. 1012.2249999999999)
Speed	<1 s/file	4–28 s/file (CPU)

Decision: docling. It wins the two things the use-cases actually need — table fidelity (“answer questions over a supplier PDF’s spec tables”) and scanned-document OCR (no OCRmyPDF pre-processing step needed; the todo’s fallback item is moot). markitdown’s only wins were speed and float rendering.

Deployment sealed it: the app runs in Kamal-managed containers with no Python, and docling ships an official server image (docling-serve) that drops straight in as a Kamal accessory — zero custom packaging. markitdown has no server image; we’d have to build and maintain one (or bake a Python toolchain into the Rails image). The original “uv-pinned CLI the job shells out to” idea died for the same reason: there is no uv inside the production containers.

docling-serve responses also carry a per-document confidence report (mean_grade: poor/fair/good/excellent) we can surface later if garbage extractions ever become a problem.

Architecture

Same dev↔prod philosophy as the Playwright browser server (PlaywrightRuntime):

                         DOCLING_SERVER_URL
Rails app (no Python) ──────HTTP POST──────▶ docling-serve container
  DoclingClient.convert(path)                 /v1/convert/file → md_content

• Production: docling Kamal accessory (config/deploy.yml) on the primary host — ghcr.io/docling-project/docling-serve-cpu:v1.26.0 (models baked in, ~7 GB image, ~2–4 GB RAM under load, CPU inference). App env: DOCLING_SERVER_URL: http://heatwave-docling:5001.
• Staging: mirrored accessory block (config/deploy.staging.yml), heatwave-staging-docling. Shares the box with prod — boot only when staging needs extraction.
• Development: part of the default compose stack — docker compose up -d starts it and bin/dev exports DOCLING_SERVER_URL=http://localhost:5001 automatically.
• Unset URL (test, fresh dev): DoclingClient.configured? is false and convert raises DoclingClient::NotConfigured with instructions. Nothing else breaks.

Accessories are never booted by kamal deploy; first-time setup is:

mise exec -- bundle exec kamal accessory boot docling               # production
mise exec -- bundle exec kamal accessory boot docling -d staging    # staging (optional)

Version bumps: image tag is pinned in deploy.yml, deploy.staging.yml, and docker-compose.yml — bump all three together (kamal accessory reboot docling after).

DOCLING_SERVE_MAX_SYNC_WAIT=300 on the accessory keeps the synchronous convert endpoint aligned with DoclingClient::CONVERT_TIMEOUT (300 s); the server default (120 s) can be exceeded by big scanned PDFs on CPU. If volume ever outgrows the sync endpoint, docling-serve also exposes an async task API (/v1/convert/file/async + poll) — switch DoclingClient then, not before.

Rails surface

Piece	What it does
DoclingClient	HTTP client. convert(path, filename:) → Markdown. SUPPORTED_EXTENSIONS gates input types.
uploads.extracted_markdown (text)	Cached extraction, on the upload row (migration 20260717103419).
Upload#extract_markdown!(force: false)	Extract + cache (update_column — derived data, skips validations/audit). Returns cache when present.
UploadMarkdownExtractionWorker	Sidekiq (pdf queue). perform_async(upload_id, {'force' => true}). Skips missing/unstored/unsupported uploads.
Publication::MarkdownExtractionHandler	RES subscriber on Events::PublicationPdfChanged — pre-extracts every publication/literature PDF on link/replace (sibling of Publication::VisionAnalysisHandler).
extract_document (Sunny tool)	8th tool in Assistant::PdfToolBuilder (pdf_tools service, all roles). Source = conversation-attached upload id or public URL; paginates long output via offset/next_offset (12 000-char slices); caches on the upload.

Unattended usage (retailer onboarding, one-off scripts):

UploadMarkdownExtractionWorker.perform_async(upload.id)
# …or inline:
upload.extract_markdown!            # cached after the first call

Publication semantic-search embeddings

docling is also the document-text backbone of publication embeddings. The prior pipeline embedded only Claude’s vision paraphrase (pdf_image_descriptions) once a publication was analyzed — so verbatim spec text and tables were absent from the vector, and an exact model-number/spec query could miss. The old fallback (PDF::Reader) produced garbled/concatenated text, which is why vision had displaced it.

Now the two layers are merged, not either/or (Publication#content_for_embedding):

• Document text — docling markdown (verbatim body, model numbers, spec tables), cached in items.search_text. This also feeds the lexical half of hybrid search and the embeddability gate.
• Visual content — Claude’s vision descriptions of the diagrams/photos that docling reduces to <!-- image --> placeholders.

A query for an exact spec hits the docling text; a query for “how do I wire the thermostat” hits the vision description of the diagram — either source alone loses one.

Wiring (all async — no docling call in any web save path):

PublicationPdfChanged event
  ├─ Publication::VisionAnalysisHandler  → PublicationVisionWorker    → pdf_image_descriptions
  └─ Publication::MarkdownExtractionHandler → UploadMarkdownExtractionWorker
        → literature.extracted_markdown  (docling)
        → items.search_text              (mirror)
        → EmbeddingWorker (chunked)       → merged docling + vision vector(s)

Publications always take the chunked embedding path (Item#embeddable_chunked? → Embeddable#generate_chunked_embeddings!): merged content routinely exceeds the ~8k-token window, so a single vector would truncate the tail of long installation manuals. The chunked path self-degrades to a single primary row for short documents, and keeps a record in exactly one shape (single XOR *_chunk_* rows) so retrieval never carries a stale duplicate vector.

PDF::Reader is removed from the publication path (the gem stays for Gamma import + Pdf::Toolkit). Reindexing existing publications after this change is a bulk op — enqueue UploadMarkdownExtractionWorker per literature (it extracts → mirrors search_text → re-embeds), gated by the count-first + two-confirmation protocol.

Gotchas

• Filename extension matters — docling-serve detects the input format from the uploaded filename. DoclingClient.convert forwards Upload#attachment_name; URL sources must end in a supported extension.
• Image handling — we request image_export_mode=placeholder (<!-- image -->), never embedded: the docling CLI’s default embeds base64 images and ballooned a 393 KB manual into 1.3 MB of Markdown in the spike.
• OCR noise — scanned-document output is usable but imperfect (logo art extracted as garbage strings, glyph slips). The Sunny system prompt’s extraction-caveat section covers judging artifacts.
• PII — extractions inherit their upload’s sensitivity (support-case invoices contain customer addresses). extracted_markdown lives on the same row with the same access story as the file itself; the Sunny tool only reads conversation-attached uploads.