Skip to content

MarkItDown Sidecar

Jump to bottom
lqdflying edited this page Aug 8, 2026 · 18 revisions

MarkItDown sidecar

An optional companion container that converts uploaded documents to Markdown before the Knowledge Base chunks and embeds them. It widens the set of formats Knowledge accepts, and makes the formats that already worked arrive as structured Markdown — headings and tables survive chunking instead of collapsing into flat text.

It wraps Microsoft MarkItDown 0.1.7 in a two-endpoint HTTP API.

Why it is a separate container

MarkItDown is Python. The ChatHub runtime image is distroless — FROM scratch plus busybox and a copied node binary, with no interpreter and no package manager. MarkItDown's full converter set also needs lxml, pandas, pdfminer, pdfplumber and Magika's ONNX model, plus ffmpeg and exiftool as system binaries for audio transcription and image metadata.

Image Size (uncompressed virtual)
lqdflying/chathub 564 MB
Sidecar — Debian build (this page's Dockerfile, default args) ~1.07 GB
Sidecar — Debian build, INSTALL_OCR_PLUGIN=true ~1.15 GB
Sidecar — alternative recipe (Chainguard Wolfi base + multi-stage venv) ~730 MB

The size depends on the base image and on the optional OCR plugin. The Debian slim base alone is ~380 MB versus ~17 MB for a Wolfi/apko base, and building the Debian recipe with INSTALL_OCR_PLUGIN=true (which adds the markitdown-ocr plugin, including its PyMuPDF rendering dependency) adds ~80 MB. INSTALL_OCR_PLUGIN=false is the default in the Debian Dockerfile ARG and the Compose args:; the current prebuilt Wolfi image does not define that arg at all, so it ships without the plugin too. See OCR for scanned documents and pictures for when you would turn it on.

Folding it in would roughly triple the ChatHub image for every user, including those who never upload a spreadsheet, and would mean giving up distroless. So it runs alongside instead, and deployments that do not set MARKITDOWN_SERVICE_URL never build or pull it. This mirrors how ChatHub already treats UNSTRUCTURED_SERVER_URL.

What it adds

Formats the Knowledge Base accepts only when this service is configured:

Kind Formats
Spreadsheets .xlsx .xls — one Markdown table per sheet
Mail .msg (Outlook)
Notebooks .ipynb — cells as fenced code blocks
Archives .zip — converts each member and concatenates
Feeds .rss .atom .xml
Images .png .jpg .jpeg — EXIF metadata, plus an LLM-generated description when configured; OCR is available separately through supported Azure paths
Audio .wav .mp3 .m4a .mp4 — metadata plus speech transcription
Other .htm .jsonl .text

Formats that already worked keep working, now via MarkItDown: PDF (with geometric table detection), .docx (including equations, as LaTeX), .pptx (per-slide headings, speaker notes, chart data), .epub, .csv, .html.

.doc and .docm remain unsupported — MarkItDown has no converter for legacy Word either.

This table is the authority for which extensions ChatHub offers in its upload picker. MARKITDOWN_EXTENSIONS in packages/utils/src/isChunkableFile.ts mirrors it; if you change what the sidecar converts, update that set too.

Quick start

Two ways to run the sidecar. Both use the same markitdown.env for runtime configuration and the same ChatHub-side wiring. Pick one.

markitdown.env

Every sidecar variable lives in this file beside the Compose file. Nothing here is mandatory from the image's perspective — it boots with built-in defaults. What becomes mandatory depends on your topology and which capabilities you turn on; see Configuration reference for the full breakdown.

# ===== MANDATORY — when the port is reachable beyond loopback =====
# Bearer token required on /convert. Must equal ChatHub's MARKITDOWN_API_KEY.
# Leave empty only when the sidecar is bound to 127.0.0.1 AND nothing else
# can route to it.
MARKITDOWN_API_KEY=

# ===== NON-MANDATORY — uncomment to override built-in defaults =====
# Cap upload size in bytes. Default 104857600 (100 MiB). Keep ChatHub's
# MARKITDOWN_MAX_FILE_SIZE less than or equal to this sidecar's, or ChatHub
# will ship files the sidecar rejects.
# MARKITDOWN_MAX_FILE_SIZE=104857600

# Per-conversion log level. Default INFO; DEBUG logs each conversion.
# MARKITDOWN_LOG_LEVEL=INFO

# Load third-party markitdown.plugin entry points. Default false.
# MARKITDOWN_ENABLE_PLUGINS=false

# Inline images as base64 data URIs. Default false — base64 wastes embedding
# tokens. Turn on only if you need rendered Markdown with visible images.
# MARKITDOWN_KEEP_DATA_URIS=false

# ===== OPTIONAL — image captioning + (with the OCR plugin) scanned-PDF OCR =====
# Set MARKITDOWN_LLM_API_KEY and MARKITDOWN_LLM_MODEL together.
# Set MARKITDOWN_LLM_BASE_URL only when using a custom or non-OpenAI endpoint
# that exposes an OpenAI-compatible API.
# Scanned-PDF OCR additionally requires the image built with
# INSTALL_OCR_PLUGIN=true AND MARKITDOWN_ENABLE_PLUGINS=true — see
# "Build-time only" under Configuration reference.
# MARKITDOWN_LLM_BASE_URL=
# MARKITDOWN_LLM_API_KEY=
# MARKITDOWN_LLM_MODEL=
# Override the captioning prompt. Always optional.
# MARKITDOWN_LLM_PROMPT=

# ===== OPTIONAL — Azure OCR (alternative to the LLM-OCR path) =====
# AZURE_API_KEY is shared between Doc Intel and Content Understanding. Set
# one of the endpoints (or both — MarkItDown picks one based on the format).
# AZURE_DOC_INTEL_ENDPOINT=
# AZURE_CU_ENDPOINT=
# AZURE_CU_ANALYZER_ID=
# AZURE_API_KEY=

Option A — Run a prebuilt image

# markitdown.compose.yml — pull a prebuilt image. Env comes from
# markitdown.env beside this file so the compose carries no secrets.
services:
  markitdown:
    image: img.aksg.net/markitdown/markitdown:0.1.7 # or your own registry
    container_name: chathub-markitdown
    restart: always
    env_file:
      - markitdown.env
    ports:
      # Remove the 127.0.0.1: prefix only when unavoidable; pair it with
      # MARKITDOWN_API_KEY in markitdown.env so the open port is not anonymous.
      - '127.0.0.1:5000:5000'
    security_opt:
      - no-new-privileges:true
docker pull img.aksg.net/markitdown/markitdown:0.1.7
docker compose -f markitdown.compose.yml up -d

If ChatHub already runs under Compose, copy the markitdown service into ChatHub's compose (alongside chathub / chathub-db) and put both containers on one network — then ChatHub reaches the sidecar at http://markitdown:5000 and you can drop the ports: publish entirely. Keep env_file: [markitdown.env] when you copy the block in: component-scoped env stays per-service even inside a shared compose file.

Option B — Build from source

Create a directory with these four files. Nothing else is needed, and it does not have to live inside the ChatHub checkout.

Dockerfile

# MarkItDown conversion sidecar for the ChatHub Knowledge Base.
#
# Build from the directory holding these files:
#   docker build -t chathub-markitdown .
FROM python:3.12-slim-bookworm

ARG USE_CN_MIRROR

# `exiftool` supplies image and audio metadata; `ffmpeg` lets pydub transcode
# mp3/mp4 to wav for speech transcription. Without them those converters still
# run, they just have far less to report.
RUN \
    if [ "${USE_CN_MIRROR:-false}" = "true" ]; then \
        sed -i "s/deb.debian.org/mirrors.ustc.edu.cn/g" /etc/apt/sources.list.d/debian.sources; \
    fi \
    && apt-get update \
    && apt-get install --no-install-recommends -y exiftool ffmpeg \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

ENV EXIFTOOL_PATH="/usr/bin/exiftool" \
    PIP_DISABLE_PIP_VERSION_CHECK="1" \
    PIP_NO_CACHE_DIR="1" \
    PYTHONDONTWRITEBYTECODE="1" \
    PYTHONUNBUFFERED="1"

WORKDIR /app

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

# LLM-vision OCR for scanned PDFs and pictures embedded in Office documents.
# Off by default: it is installed from a git subdirectory and adds the
# markitdown-ocr plugin (which pulls PyMuPDF as a page-rendering dep), so
# it should not be able to break the base build.
ARG INSTALL_OCR_PLUGIN=false
RUN if [ "${INSTALL_OCR_PLUGIN}" = "true" ]; then \
        pip install --no-cache-dir \
        "markitdown-ocr @ git+https://github.com/microsoft/markitdown.git@v0.1.7#subdirectory=packages/markitdown-ocr"; \
    fi

COPY server.py ./

# Conversion reads whatever the process can reach, so drop privileges and keep
# no writable application directory.
USER nobody:nogroup

EXPOSE 5000

HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
    CMD python -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:5000/health', timeout=4).status == 200 else 1)"

CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "5000"]

requirements.txt

# Pinned so the converter behaviour matches what this page documents.
markitdown[all]==0.1.7

fastapi==0.115.12
uvicorn[standard]==0.34.0
python-multipart==0.0.20

# Used only when MARKITDOWN_LLM_* is configured, for image captioning and — with
# the markitdown-ocr plugin — OCR of scanned pages.
openai==1.68.2

The HTTP wire format (multipart POST /convert, JSON {markdown,title,…} response, the status codes in the HTTP contract) that ChatHub depends on is the stable surface; the dependency versions are not part of that contract. Dependency upgrades for CVE remediation are supported, but the resulting image must be validated with HTTP-contract tests, representative conversion fixtures, and a fresh vulnerability scan — FastAPI, Starlette, Pydantic, OpenAI, MarkItDown and their transitive dependencies may change behaviour between minor versions, so identical conversion output cannot be assumed. Top-level version pins alone do not guarantee byte-for-byte reproducibility either; that requires a complete dependency lock, a pinned base-image digest, a pinned upstream source revision, and a controlled build environment.

server.py

"""HTTP wrapper around Microsoft MarkItDown for the ChatHub Knowledge Base.

MarkItDown is a Python library and the ChatHub runtime image is distroless
(node only), so conversion runs here, out of process. ChatHub POSTs a document
to /convert and embeds the Markdown that comes back.

Configuration (all optional except where noted):
  MARKITDOWN_API_KEY        Bearer token required on /convert when set.
  MARKITDOWN_MAX_FILE_SIZE  Reject larger uploads, in bytes. Default 100 MiB.
  MARKITDOWN_ENABLE_PLUGINS Load third-party `markitdown.plugin` entry points.
  MARKITDOWN_KEEP_DATA_URIS Inline images as base64 data URIs. Off by default:
                            base64 blobs waste embedding tokens.
  MARKITDOWN_LOG_LEVEL      Server log level. Default INFO.
  MARKITDOWN_LLM_BASE_URL   OpenAI-compatible endpoint used to caption images
  MARKITDOWN_LLM_API_KEY    and, with the markitdown-ocr plugin installed, to
  MARKITDOWN_LLM_MODEL      OCR scanned PDFs and embedded pictures.
  MARKITDOWN_LLM_PROMPT     Override the captioning prompt.
  AZURE_DOC_INTEL_ENDPOINT  Route supported formats through Azure Document
                            Intelligence (prebuilt-layout) instead.
  AZURE_CU_ENDPOINT         Azure Content Understanding endpoint, and
  AZURE_CU_ANALYZER_ID      an optional custom analyzer id.
"""

from __future__ import annotations

import io
import logging
import os
import secrets
import threading
import time

from fastapi import Depends, FastAPI, File, Form, Header, HTTPException, Request, UploadFile
from fastapi.responses import JSONResponse
from markitdown import (
    FileConversionException,
    MarkItDown,
    MissingDependencyException,
    StreamInfo,
    UnsupportedFormatException,
    __version__ as markitdown_version,
)

logging.basicConfig(level=os.environ.get("MARKITDOWN_LOG_LEVEL") or "INFO")
logger = logging.getLogger("markitdown-server")


def _flag(name: str, default: bool = False) -> bool:
    raw = os.environ.get(name)
    if raw is None:
        return default
    return raw.strip().lower() in {"1", "true", "yes", "on"}


API_KEY = os.environ.get("MARKITDOWN_API_KEY") or None
MAX_FILE_SIZE = int(os.environ.get("MARKITDOWN_MAX_FILE_SIZE") or 100 * 1024 * 1024)
ENABLE_PLUGINS = _flag("MARKITDOWN_ENABLE_PLUGINS")
KEEP_DATA_URIS = _flag("MARKITDOWN_KEEP_DATA_URIS")

app = FastAPI(
    description="Converts documents to Markdown for the ChatHub Knowledge Base.",
    title="MarkItDown service",
)


# Refuse a body whose declared size is already over the cap before the form
# parser consumes it — this is what keeps an oversized upload from being spooled
# to disk at all. Multipart framing adds a little on top of the file itself,
# hence the slack; the exact cap is enforced on the file bytes in `convert`,
# which also catches chunked bodies that carry no Content-Length.
@app.middleware("http")
async def reject_declared_oversize(request: Request, call_next):
    declared = request.headers.get("content-length")
    if declared and declared.isdigit() and int(declared) > MAX_FILE_SIZE + 1_048_576:
        return JSONResponse(
            {"detail": f"{declared} bytes exceeds MARKITDOWN_MAX_FILE_SIZE ({MAX_FILE_SIZE})"},
            status_code=413,
        )
    return await call_next(request)


# One MarkItDown per worker thread. Each instance loads a Magika ONNX model, so
# it is too expensive to build per request, and sharing one across threads would
# mean sharing its detector. FastAPI reuses threadpool threads, so this warms up
# once per thread and then costs nothing.
_local = threading.local()


def _build_llm_client():
    """An OpenAI-compatible client for image captioning, if one is configured."""
    base_url = os.environ.get("MARKITDOWN_LLM_BASE_URL")
    api_key = os.environ.get("MARKITDOWN_LLM_API_KEY")
    model = os.environ.get("MARKITDOWN_LLM_MODEL")

    if not (api_key and model):
        return None, None

    try:
        from openai import OpenAI
    except ImportError:
        logger.warning("MARKITDOWN_LLM_* is set but the openai package is missing")
        return None, None

    return OpenAI(api_key=api_key, base_url=base_url or None), model


def _converter() -> MarkItDown:
    existing = getattr(_local, "markitdown", None)
    if existing is not None:
        return existing

    kwargs: dict[str, object] = {"enable_plugins": ENABLE_PLUGINS}

    llm_client, llm_model = _build_llm_client()
    if llm_client is not None:
        kwargs["llm_client"] = llm_client
        kwargs["llm_model"] = llm_model
        prompt = os.environ.get("MARKITDOWN_LLM_PROMPT")
        if prompt:
            kwargs["llm_prompt"] = prompt

    docintel_endpoint = os.environ.get("AZURE_DOC_INTEL_ENDPOINT")
    if docintel_endpoint:
        kwargs["docintel_endpoint"] = docintel_endpoint

    cu_endpoint = os.environ.get("AZURE_CU_ENDPOINT")
    if cu_endpoint:
        kwargs["cu_endpoint"] = cu_endpoint
        analyzer_id = os.environ.get("AZURE_CU_ANALYZER_ID")
        if analyzer_id:
            kwargs["cu_analyzer_id"] = analyzer_id

    instance = MarkItDown(**kwargs)
    _local.markitdown = instance
    logger.info(
        "MarkItDown ready (plugins=%s, llm=%s, docintel=%s, content_understanding=%s)",
        ENABLE_PLUGINS,
        bool(llm_client),
        bool(docintel_endpoint),
        bool(cu_endpoint),
    )
    return instance


def require_api_key(authorization: str | None = Header(default=None)) -> None:
    if not API_KEY:
        return
    expected = f"Bearer {API_KEY}"
    # Compare as bytes: only compare_digest gives the constant-time path, and its
    # str form raises on non-ASCII input, which a caller controls.
    if not (
        authorization
        and secrets.compare_digest(authorization.encode("utf-8"), expected.encode("utf-8"))
    ):
        raise HTTPException(status_code=401, detail="invalid or missing bearer token")


@app.get("/health")
def health() -> dict[str, object]:
    return {
        "keepDataUris": KEEP_DATA_URIS,
        "maxFileSize": MAX_FILE_SIZE,
        "plugins": ENABLE_PLUGINS,
        "status": "ok",
        "version": markitdown_version,
    }


# Deliberately a `def`, not `async def`: MarkItDown blocks, and FastAPI runs sync
# endpoints in its threadpool so one slow PDF cannot stall the event loop.
@app.post("/convert", dependencies=[Depends(require_api_key)])
def convert(
    file: UploadFile = File(...),
    filename: str | None = Form(default=None),
    mime_type: str | None = Form(default=None),
) -> dict[str, object]:
    # The framework has already spooled the body (memory below ~1 MiB, disk
    # above), so read it back in bounded pieces: the cap is enforced as bytes
    # accumulate, and an over-cap upload — or one whose Content-Length lied past
    # the middleware — is refused without ever sitting in RAM in one piece.
    pieces: list[bytes] = []
    received = 0
    while chunk := file.file.read(1024 * 1024):
        received += len(chunk)
        if received > MAX_FILE_SIZE:
            raise HTTPException(
                status_code=413,
                detail=f"file exceeds MARKITDOWN_MAX_FILE_SIZE ({MAX_FILE_SIZE})",
            )
        pieces.append(chunk)
    data = b"".join(pieces)

    if not data:
        raise HTTPException(status_code=400, detail="empty upload")

    name = filename or file.filename or "document"
    # StreamInfo wants the extension with its leading dot, and an empty string
    # would be treated as a real hint, so normalise to None instead.
    _, _, tail = name.rpartition(".")
    extension = f".{tail.lower()}" if tail and tail != name else None
    mimetype = (mime_type or file.content_type or "").split(";")[0].strip() or None

    stream_info = StreamInfo(
        extension=extension,
        filename=name,
        mimetype=mimetype,
    )

    started = time.monotonic()
    try:
        result = _converter().convert_stream(
            io.BytesIO(data),
            keep_data_uris=KEEP_DATA_URIS,
            stream_info=stream_info,
        )
    except UnsupportedFormatException as error:
        logger.info("unsupported format for %s: %s", name, error)
        raise HTTPException(status_code=415, detail=f"unsupported format: {error}") from error
    except MissingDependencyException as error:
        logger.error("missing dependency converting %s: %s", name, error)
        raise HTTPException(status_code=501, detail=str(error)) from error
    except FileConversionException as error:
        logger.error("failed to convert %s: %s", name, error)
        raise HTTPException(status_code=422, detail=str(error)) from error

    markdown = result.markdown or ""
    duration_ms = int((time.monotonic() - started) * 1000)
    logger.info(
        "converted %s (%s bytes, %s) to %s chars in %sms",
        name,
        len(data),
        mimetype or extension or "unknown type",
        len(markdown),
        duration_ms,
    )

    return {
        "chars": len(markdown),
        "durationMs": duration_ms,
        "markdown": markdown,
        "title": result.title,
    }

docker-compose.yml

Runtime configuration comes from markitdown.env (shared with Option A). The build arg INSTALL_OCR_PLUGIN stays here because it is a build-time concern, not a runtime env.

# Build and run from source. Runtime env comes from markitdown.env.
#
#   docker compose up -d --build
#
# If ChatHub already runs under Compose, copy the `markitdown` service into
# that file instead so both containers share a network, and use the service
# name in the URL (MARKITDOWN_SERVICE_URL=http://markitdown:5000).
services:
  markitdown:
    build:
      context: .
      dockerfile: Dockerfile
      args:
        # Set to "true" to bake in LLM-vision OCR (needs MARKITDOWN_ENABLE_PLUGINS=true
        # and MARKITDOWN_LLM_* at runtime — see OCR for scanned documents and pictures).
        INSTALL_OCR_PLUGIN: ${INSTALL_OCR_PLUGIN:-false}
    container_name: chathub-markitdown
    restart: always
    env_file:
      - markitdown.env
    ports:
      # Bound to loopback: the service has no authentication unless
      # MARKITDOWN_API_KEY is set, and it converts whatever it is given.
      - '127.0.0.1:5000:5000'
    security_opt:
      - no-new-privileges:true
docker compose up -d --build

Connect ChatHub

Once the sidecar is running, point ChatHub at it and restart ChatHub.

Network topology

The URL depends on where ChatHub itself runs, because the Compose files above bind the sidecar port to the host's loopback interface only.

ChatHub runs… MARKITDOWN_SERVICE_URL
in a container on the same Docker network (recommended — copy the markitdown service into ChatHub's compose) http://markitdown:5000
directly on the host (pnpm dev, or node outside Docker) http://127.0.0.1:5000
in a container but you cannot join networks http://<host-LAN-IP>:5000 — must drop the 127.0.0.1: port binding AND set MARKITDOWN_API_KEY

Do not use http://127.0.0.1:5000 from a containerized ChatHub: inside a container that address is the container itself, so the request never reaches the sidecar. The Docker bridge gateway (172.17.0.1) does not work either while the port is bound to 127.0.0.1 — put both containers on one network instead, or change the binding and set MARKITDOWN_API_KEY.

ChatHub-side environment

Add to ChatHub's own env — the .env next to ChatHub's compose, distinct from the sidecar's markitdown.env:

# Required — switches MarkItDown on as the preferred converter with the
# built-in loaders kept as the fallback. This also flips enabledMarkItDown
# in GlobalServerConfig, which is what makes the upload picker offer the
# extra formats.
MARKITDOWN_SERVICE_URL=http://markitdown:5000

# Required only when markitdown.env set MARKITDOWN_API_KEY. Must match exactly.
MARKITDOWN_API_KEY=<same-value-as-in-markitdown.env>

# Both default to sensible values; uncomment to override.
# MARKITDOWN_MAX_FILE_SIZE=104857600   # bytes (100 MiB); keep ≤ the sidecar's cap
# MARKITDOWN_TIMEOUT=180000            # ms (3 min)

See Knowledge Base and RAG for FILE_TYPE_CHUNKING_RULES to override the default routing per format — e.g. pin pdf=unstructured,default, or xlsx=markitdown to skip the fallback for spreadsheets. A FILE_TYPE_CHUNKING_RULES entry that pins a format to markitdown alone opts out of the fallback, so its errors surface directly rather than degrading to the built-in loaders.

Restart and verify

docker compose up -d --force-recreate chathub # or your equivalent restart

Server config is read at process start, so a plain restart is enough on ChatHub's side. The sidecar itself only needs --force-recreate when its own markitdown.env changes (MARKITDOWN_ENABLE_PLUGINS, MARKITDOWN_KEEP_DATA_URIS, any MARKITDOWN_LLM_* or AZURE_* value — these are read at container start, not per request).

Verify the sidecar is alive and converting:

curl -s http://127.0.0.1:5000/health # should print "version":"0.1.7"
curl -s -F file=@some.xlsx http://127.0.0.1:5000/convert | head -c 400

Then in ChatHub, upload a .xlsx / .msg / .ipynb into a Knowledge Base file. The picker now accepts formats the built-in loaders rejected.

Viewing the converted content

The Knowledge Base shows what the LLM actually sees — the converted/chunked content — not the original document:

  • MarkItDown icon (a document glyph, shown on any MarkItDown-convertible file when the sidecar is configured): re-parses the file through MarkItDown and opens the converted content. Use it to (re)convert a specific file via MarkItDown.
  • Filename or card in the Knowledge file manager: opens the chunk drawer for every file, including HTML, text, Markdown, and Office files. The drawer keeps a compact Basic Information section above search and chunk pages, with file dates, format, size, chunk count, embedding status, and a download action. The Chunks tag opens the same drawer.
  • The drawer displays one Markdown-rendered chunk per page, with a compact numbered pager and first/last jumpers. Headings, emphasis, lists, and well-formed GFM tables render as document content instead of literal Markdown punctuation. Each page has a stable size target: a large spreadsheet table is split into multiple valid Markdown pages, each no longer than 1,000 characters, with its header and delimiter repeated. A single giant cell becomes bounded, column-labeled continuation content so embedding limits are not exceeded. Each page has a provenance badge (MarkItDown when it came from the sidecar, LangChain for the built-in loader), and its detail button opens the full chunk text plus metadata — converted_by reads markitdown and type is MarkItDownElement.
  • The chunk drawer is a content-only viewer — it shows chunked / converted text, not the original file. This shared local path also avoids Microsoft Office Online, which needs a publicly reachable file URL and fails on a private host. If a file has not been parsed yet, the viewer presents a Parse action only when the current deployment supports that format. DOCX/PPTX work without the sidecar; XLS/XLSX need the sidecar; legacy DOC/ODT/PPT and sidecar-dependent files without the sidecar show an explanatory state instead of a failing action.

Existing documents do not auto-migrate. To re-chunk an already-uploaded file through MarkItDown, click its MarkItDown icon, or open the file row's actions menu (⋯) and choose Re-parse chunks — the file is re-parsed via the configured routing (MarkItDown first, with the built-in loaders as fallback) and its existing chunks are replaced. Re-parse files whose stored tables show literal pipes or a detached separator row: the table-preserving split and bounded page rules apply to newly parsed Markdown and cannot repair already-stored chunks.

Alternative images

The Debian Dockerfile above is one recipe. Prebuilt or Chainguard-Wolfi / multi-stage variants (e.g. img.aksg.net/markitdown/markitdown:0.1.7) produce a smaller, often 0-CVE image with the same HTTP contract and are a valid drop-in replacement.

The INSTALL_OCR_PLUGIN build arg is defined only by the Debian Dockerfile; the current prebuilt Wolfi image does not contain the OCR plugin, and its Wolfi build recipe must be extended before that build arg can be used.

Component Debian default build Current Wolfi prebuilt image
All text-extraction converters (xlsx/xls, docx, pptx, pdf, epub, html, msg, ipynb, zip, rss/atom, csv, code) ✓ present ✓ present
Audio transcription + image/audio metadata ✓ present ✓ present
Image captioning via OpenAI-compatible API (openai + MARKITDOWN_LLM_*) ✓ present (runtime opt-in) ✓ present (runtime opt-in)
Azure OCR (azure/msal + AZURE_*) ✓ present (runtime opt-in) ✓ present (runtime opt-in)
Scanned-PDF / embedded-raster OCR (markitdown-ocr plugin) opt-in via INSTALL_OCR_PLUGIN=true (off by default) not installed; Wolfi recipe must be extended first

A default build supports the same native-text input formats as a plugin-enabled build. Enabling the plugin adds raster/OCR processing; use representative conversion fixtures when exact output equivalence matters. Picture/raster handling is a purely optional capability; see OCR for scanned documents and pictures for the only inputs it affects.

To probe an existing image for the plugin without converting a file, check for the plugin's own module (not just PyMuPDF, which is only a rendering dependency):

docker run --rm img.aksg.net/markitdown/markitdown:0.1.7 \
  /opt/venv/bin/python -c "import markitdown_ocr" 2>&1 | head -1
# ImportError: No module named 'markitdown_ocr'  → plugin not baked in
# (silent)                                         → plugin present

/health does not surface the plugin's presence — it only reports the MarkItDown library version — so a missing plugin is otherwise invisible until a scanned PDF is uploaded.

Configuration reference

All sidecar vars live in markitdown.env (see Quick start). Every variable is optional from the image's perspective — it boots with built-in defaults — but your topology and which capabilities you turn on make some mandatory.

Mandatory

Variable When mandatory Purpose
MARKITDOWN_API_KEY When the port is reachable by anything other than ChatHub alone — i.e. you dropped the 127.0.0.1: port prefix, or another host can route to it Bearer token required on /convert. Must equal ChatHub's MARKITDOWN_API_KEY.

If the sidecar is bound to loopback and only ChatHub (on the same host) reaches it, nothing here is mandatory — the image's built-in defaults are sufficient and the sidecar refuses nothing on its own.

Conditional feature groups

Set each group fully, or not at all.

Group Variables What it enables
Auth MARKITDOWN_API_KEY (alone) Bearer-protected /convert.
Standalone image captioning MARKITDOWN_LLM_API_KEY + MARKITDOWN_LLM_MODEL (+ MARKITDOWN_LLM_BASE_URL only for a custom or non-OpenAI endpoint that exposes an OpenAI-compatible API). MARKITDOWN_LLM_PROMPT is always optional. Captions standalone PNG/JPEG via MarkItDown's built-in ImageConverter. No plugin or rebuild needed.
OCR plugin (scanned PDFs + embedded pictures) Same LLM vars as standalone captioning, plus MARKITDOWN_ENABLE_PLUGINS=true, and the image must have been built with INSTALL_OCR_PLUGIN=true. OCRs scanned PDFs and pictures embedded in PDF/DOCX/PPTX/XLSX via the markitdown-ocr plugin. Does not affect standalone images.
Azure Doc Intel AZURE_DOC_INTEL_ENDPOINT + AZURE_API_KEY Azure Document Intelligence prebuilt-layout for PDFs/images. Alternative to the LLM-OCR path.
Azure Content Understanding AZURE_CU_ENDPOINT + AZURE_API_KEY. AZURE_CU_ANALYZER_ID is optional even when CU is configured. Azure's multimodal analyzer.

The LLM and Azure paths are mutually independent — run both, either, or neither.

Non-mandatory

Variable Default Purpose
MARKITDOWN_MAX_FILE_SIZE 104857600 (100 MiB) Reject larger uploads, in bytes. Enforced twice — once by a content-length middleware, once by a byte-count read. Keep ChatHub's MARKITDOWN_MAX_FILE_SIZE ≤ this sidecar's, or ChatHub will ship files the sidecar refuses.
MARKITDOWN_ENABLE_PLUGINS false Load third-party markitdown.plugin entry points. Required true for the markitdown-ocr plugin to register.
MARKITDOWN_KEEP_DATA_URIS false Inline images as base64 data URIs in the Markdown. Off: base64 blobs waste embedding tokens. Turn on only if you need to render the converted Markdown with images intact.
MARKITDOWN_LOG_LEVEL INFO Python log level (DEBUG / INFO / WARNING / ERROR). DEBUG logs each conversion with duration and size — useful when diagnosing.
AZURE_CU_ANALYZER_ID (Azure-supplied default) Custom Content Understanding analyzer. Only meaningful when AZURE_CU_ENDPOINT is set.

Build-time only

INSTALL_OCR_PLUGIN is a Docker build arg, not a runtime env. It is defined by the Debian Dockerfile (default false); the current prebuilt Wolfi image does not define it, so passing --build-arg INSTALL_OCR_PLUGIN=true to that recipe has no effect until the Wolfi Dockerfile is extended.

Installing the plugin alone is not enough — it is discovered via the markitdown.plugin entry point, and MarkItDown only loads entry points when enable_plugins=True. The sidecar sets that from MARKITDOWN_ENABLE_PLUGINS, which defaults to false. The plugin's OCR engine is an OpenAI-compatible vision LLM (configured at runtime via MARKITDOWN_LLM_*); PyMuPDF is only a page-rendering dependency, not the OCR engine itself. The full scanned-PDF OCR path therefore needs all three:

# Build time (Debian recipe)
INSTALL_OCR_PLUGIN=true

# Runtime (markitdown.env)
MARKITDOWN_ENABLE_PLUGINS=true
MARKITDOWN_LLM_API_KEY=...
MARKITDOWN_LLM_MODEL=...
# Required only for a custom or non-OpenAI endpoint that exposes an
# OpenAI-compatible API:
# MARKITDOWN_LLM_BASE_URL=...

Without MARKITDOWN_ENABLE_PLUGINS=true, the installed markitdown-ocr entry point is never registered, so scanned-PDF OCR does not run. Building with the plugin but leaving MARKITDOWN_LLM_* unset also produces no OCR — there is no engine to call. Conversely, setting MARKITDOWN_LLM_* at runtime without the plugin still captions standalone images (.png/.jpg) via the LLM, but cannot get text out of scanned PDFs because the page-rendering pipeline is absent.

ChatHub-side variables

ChatHub reads four vars, set in ChatHub's own env — the .env next to ChatHub's compose, distinct from the sidecar's markitdown.env:

Variable Required? Purpose
MARKITDOWN_SERVICE_URL Yes, to switch MarkItDown on Where ChatHub POSTs conversion requests. Switches enabledMarkItDown in GlobalServerConfig, which is what makes the upload picker offer the extra formats.
MARKITDOWN_API_KEY Yes when the sidecar sets it Bearer token; must match the sidecar's MARKITDOWN_API_KEY exactly.
MARKITDOWN_MAX_FILE_SIZE No (default 104857600) Skip conversion above this size. Keep ≤ the sidecar's cap.
MARKITDOWN_TIMEOUT No (default 180000) Give up after this many ms.

exiftool and ffmpeg are installed in the image, so image/audio metadata and speech transcription work without further setup.

OCR for scanned documents and pictures

OCR of scanned PDFs and raster pictures embedded in Office documents is the only input class that any default build does not handle. The behaviour without OCR differs by input:

  • A scanned PDF (image-only pages) typically produces empty Markdown — the text extractors (pdfminer/pdfplumber/pypdfium2) find no text layer. Verified output of the current production image for an image-only PDF: {"chars": 0, "markdown": "", "title": null}. ChatHub reports NoChunkError on the empty result.
  • A standalone PNG or JPEG may return only EXIF or image-size metadata (via the exiftool converter), since there is no text layer to extract.

EXIF metadata is an image-input concept, not a PDF one — do not expect it from a scanned PDF. This whole section is an optional capability: if your deployment never ingests scanned or picture-only input, you can ignore it entirely and the sidecar is complete as built.

Two independent paths exist, with different coverage:

Path 1 — markitdown-ocr plugin (LLM-vision OCR)

Requires all three: build with INSTALL_OCR_PLUGIN=true (Debian recipe), runtime MARKITDOWN_ENABLE_PLUGINS=true, and runtime MARKITDOWN_LLM_*. The plugin registers enhanced converters for PDF, DOCX, PPTX, and XLSX only, so it OCRs scanned PDFs and pictures embedded in those document types. It does not register a standalone PNG/JPEG converter — standalone images are handled separately by MarkItDown's built-in ImageConverter when an LLM client and model are configured (see the "Standalone image captioning" row under Conditional feature groups). The distinction matters: standalone image output is an LLM-generated description from the built-in converter, while document-image OCR comes from markitdown-ocr.

Path 2 — Azure Document Intelligence

Set AZURE_DOC_INTEL_ENDPOINT + AZURE_API_KEY in markitdown.env. No rebuild. This OCRs scanned PDFs and supported standalone images (PDF/JPEG/PNG/BMP/TIFF), but it is not equivalent to markitdown-ocr for pictures embedded in Office documents — in MarkItDown v0.1.7 the DocumentIntelligenceFileType enum runs DOCX, PPTX, XLSX and HTML through Azure with OCR explicitly disabled (the enum marks them # No OCR), so embedded Office images are not OCR'd via this path.

Azure Content Understanding (AZURE_CU_ENDPOINT) is a separate multimodal path and should not be described as identical to Document Intelligence; treat it as its own capability with its own coverage.

The HTTP contract

ChatHub's client (src/libs/markitdown/index.ts) depends on this wire format. It is not covered by any test in the ChatHub repository — the client's tests mock fetch, and the chunking tests mock the client — so changing either side silently breaks conversion while every test still passes. Treat it as frozen unless you change both.

GET /health

No authentication — safe as a container healthcheck.

{
  "keepDataUris": false,
  "maxFileSize": 104857600,
  "plugins": false,
  "status": "ok",
  "version": "0.1.7"
}

POST /convert

Multipart request:

Field Required Purpose
file yes The document bytes
filename no Extension hint; improves format detection
mime_type no MIME hint; improves format detection

Send Authorization: Bearer $MARKITDOWN_API_KEY when that variable is set.

Response:

{
  "chars": 808,
  "durationMs": 78,
  "markdown": "## Sheet1\n| Alpha | Beta |\n| --- | --- |\n...",
  "title": null
}
Status Meaning
400 Empty upload
401 Missing or wrong bearer token
413 Larger than MARKITDOWN_MAX_FILE_SIZE
415 No converter for this format
422 Conversion failed
501 A converter's optional dependency is missing

With the default routing, ChatHub treats every one of them as "fall back to the built-in loaders"; when that fallback also fails, the sidecar's reason is carried into the surfaced error, so a sidecar that is down reports as unreachable rather than as an unsupported file type. A FILE_TYPE_CHUNKING_RULES entry that pins a format to markitdown alone opts out of the fallback — its errors surface directly.

Update it

MarkItDown is pinned in requirements.txt so conversion output does not change under you. To move to a newer release:

  1. Check what changed upstream in the MarkItDown releases — converter output can shift between versions, which changes the chunks that get embedded.

  2. Bump the pin, e.g. markitdown[all]==0.1.8. If you build with INSTALL_OCR_PLUGIN=true, bump the @v0.1.7 git ref in the Dockerfile to match, or the plugin and the library will disagree.

  3. Rebuild without the layer cache, so pip actually re-resolves:

    docker compose build --no-cache
docker compose up -d

  4. Confirm the running version:

    curl -s http://127.0.0.1:5000/health # "version" must show the new release

  5. Re-convert a document of each format you rely on, and reindex anything whose output you want re-embedded.

Configuration is read at container start, not per request, so changing markitdown.env values needs docker compose up -d --force-recreate, not just editing the file.

To stop the sidecar — note that a container started from a Compose file you have since moved or deleted must be removed by name:

docker compose down             # from the directory with docker-compose.yml
docker rm -f chathub-markitdown # if the Compose file is no longer there

Security

Conversion reads whatever the process can reach, so treat this as an internal service:

  • The Compose files bind the port to 127.0.0.1 and the Debian image drops privileges to nobody:nogroup. Do not expose it publicly.
  • Set MARKITDOWN_API_KEY if anything other than ChatHub can route to it — the service is unauthenticated otherwise.
  • Audio transcription sends audio to Google's free Web Speech endpoint, and the YouTube converter fetches transcripts, so those paths leave your network. Everything else converts locally.

ChatHub Wiki

Clone this wiki locally