Document → Markdown extraction (docling-serve)

Layout-aware "file in → clean Markdown out" for Sunny uploads, publications/
literatures, and unattended jobs (retailer onboarding workbooks, supplier
datasheets). Built 2026-07-17 from the AI-resources review pick
(Basecamp todo).

Spike: docling vs markitdown (2026-07-17)

Both candidates were run over six real Heatwave files — a WarmlyYours spec
sheet and the Ember radiant-panel installation manual (public publications), a
Wayfair invoice from a support case, a scanned tax-exemption certificate, and
two retailer workbooks (wayfair.ca.xlsx, home-depot-aged-invoices.xlsx).

File (type)	markitdown 0.1.x	docling 2.107
Spec sheet PDF (label/value spec table)	Table structure lost — labels and values emitted as disconnected paragraph runs; pairing unrecoverable	Proper 2-column Markdown table, every label↔value pair intact
Installation manual PDF (12 pp, diagrams)	Plain text, ~3 s	Full structure, headings + tables, ~28 s CPU
Scanned PDF (tax certificate)	0 bytes — no OCR at all	OCR'd automatically (easyocr); text + checkbox items usable, minor glyph noise
Support-case invoice PDF	Text only (had a text layer)	Invoice fields + line-item table reconstructed; confidence "good"
Retailer workbooks (.xlsx)	Clean tables, slightly nicer float rendering	Clean tables (raw floats occasionally verbose, e.g. 1012.2249999999999)
Speed	<1 s/file	4–28 s/file (CPU)

Decision: docling. It wins the two things the use-cases actually need —
table fidelity ("answer questions over a supplier PDF's spec tables") and
scanned-document OCR (no OCRmyPDF pre-processing step needed; the todo's
fallback item is moot). markitdown's only wins were speed and float rendering.

Deployment sealed it: the app runs in Kamal-managed containers with no Python,
and docling ships an official server image
(docling-serve) that drops
straight in as a Kamal accessory — zero custom packaging. markitdown has no
server image; we'd have to build and maintain one (or bake a Python toolchain
into the Rails image). The original "uv-pinned CLI the job shells out to" idea
died for the same reason: there is no uv inside the production containers.

docling-serve responses also carry a per-document confidence report
(mean_grade: poor/fair/good/excellent) we can surface later if garbage
extractions ever become a problem.

Architecture

Same dev↔prod philosophy as the Playwright browser server (PlaywrightRuntime):

                         DOCLING_SERVER_URL
Rails app (no Python) ──────HTTP POST──────▶ docling-serve container
  DoclingClient.convert(path)                 /v1/convert/file → md_content

• Production: docling Kamal accessory (config/deploy.yml) on the primary
  host — ghcr.io/docling-project/docling-serve-cpu:v1.26.0 (models baked in,
  ~7 GB image, ~2–4 GB RAM under load, CPU inference). App env:
  DOCLING_SERVER_URL: http://heatwave-docling:5001.
• Staging: mirrored accessory block (config/deploy.staging.yml),
  heatwave-staging-docling. Shares the box with prod — boot only when
  staging needs extraction.
• Development: part of the default compose stack — docker compose up -d
  starts it and bin/dev exports DOCLING_SERVER_URL=http://localhost:5001
  automatically.
• Unset URL (test, fresh dev): DoclingClient.configured? is false and
  convert raises DoclingClient::NotConfigured with instructions. Nothing
  else breaks.

Accessories are never booted by kamal deploy; first-time setup is:

mise exec -- bundle exec kamal accessory boot docling               # production
mise exec -- bundle exec kamal accessory boot docling -d staging    # staging (optional)

Version bumps: image tag is pinned in deploy.yml, deploy.staging.yml, and
docker-compose.yml — bump all three together
(kamal accessory reboot docling after).

DOCLING_SERVE_MAX_SYNC_WAIT=300 on the accessory keeps the synchronous
convert endpoint aligned with DoclingClient::CONVERT_TIMEOUT (300 s); the
server default (120 s) can be exceeded by big scanned PDFs on CPU. If volume
ever outgrows the sync endpoint, docling-serve also exposes an async task API
(/v1/convert/file/async + poll) — switch DoclingClient then, not before.

Rails surface

Piece	What it does
DoclingClient	HTTP client. convert(path, filename:) → Markdown. SUPPORTED_EXTENSIONS gates input types.
uploads.extracted_markdown (text)	Cached extraction, on the upload row (migration 20260717103419).
Upload#extract_markdown!(force: false)	Extract + cache (update_column — derived data, skips validations/audit). Returns cache when present.
UploadMarkdownExtractionWorker	Sidekiq (pdf queue). perform_async(upload_id, {'force' => true}). Skips missing/unstored/unsupported uploads.
Publication::MarkdownExtractionHandler	RES subscriber on Events::PublicationPdfChanged — pre-extracts every publication/literature PDF on link/replace (sibling of Publication::VisionAnalysisHandler).
extract_document (Sunny tool)	8th tool in Assistant::PdfToolBuilder (pdf_tools service, all roles). Source = conversation-attached upload id or public URL; paginates long output via offset/next_offset (12 000-char slices); caches on the upload.

Unattended usage (retailer onboarding, one-off scripts):

UploadMarkdownExtractionWorker.perform_async(upload.id)
# …or inline:
upload.extract_markdown!            # cached after the first call

Publication semantic-search embeddings

docling is also the document-text backbone of publication embeddings. The
prior pipeline embedded only Claude's vision paraphrase
(pdf_image_descriptions) once a publication was analyzed — so verbatim spec
text and tables were absent from the vector, and an exact model-number/spec
query could miss. The old fallback (PDF::Reader) produced garbled/concatenated
text, which is why vision had displaced it.

Now the two layers are merged, not either/or (Publication#content_for_embedding):

• Document text — docling markdown (verbatim body, model numbers, spec
  tables), cached in items.search_text. This also feeds the lexical half of
  hybrid search and the embeddability gate.
• Visual content — Claude's vision descriptions of the diagrams/photos that
  docling reduces to <!-- image --> placeholders.

A query for an exact spec hits the docling text; a query for "how do I wire the
thermostat" hits the vision description of the diagram — either source alone
loses one.

Wiring (all async — no docling call in any web save path):

PublicationPdfChanged event
  ├─ Publication::VisionAnalysisHandler  → PublicationVisionWorker    → pdf_image_descriptions
  └─ Publication::MarkdownExtractionHandler → UploadMarkdownExtractionWorker
        → literature.extracted_markdown  (docling)
        → items.search_text              (mirror)
        → EmbeddingWorker (chunked)       → merged docling + vision vector(s)

Publications always take the chunked embedding path
(Item#embeddable_chunked? → Embeddable#generate_chunked_embeddings!): merged
content routinely exceeds the ~8k-token window, so a single vector would truncate
the tail of long installation manuals. The chunked path self-degrades to a
single primary row for short documents, and keeps a record in exactly one
shape (single XOR *_chunk_* rows) so retrieval never carries a stale
duplicate vector.

PDF::Reader is removed from the publication path (the gem stays for Gamma
import + Pdf::Toolkit). Reindexing existing publications after this change is a
bulk op — enqueue UploadMarkdownExtractionWorker per literature (it extracts →
mirrors search_text → re-embeds), gated by the count-first + two-confirmation
protocol.

Gotchas

• Filename extension matters — docling-serve detects the input format from
  the uploaded filename. DoclingClient.convert forwards
  Upload#attachment_name; URL sources must end in a supported extension.
• Image handling — we request image_export_mode=placeholder
  (<!-- image -->), never embedded: the docling CLI's default embeds
  base64 images and ballooned a 393 KB manual into 1.3 MB of Markdown in the
  spike.
• OCR noise — scanned-document output is usable but imperfect (logo art
  extracted as garbage strings, glyph slips). The Sunny system prompt's
  extraction-caveat section covers judging artifacts.
• PII — extractions inherit their upload's sensitivity (support-case
  invoices contain customer addresses). extracted_markdown lives on the same
  row with the same access story as the file itself; the Sunny tool only reads
  conversation-attached uploads.