How do I finetune an entire pdf document?

[digsganit98]

Hi, how do I finetune an entire pdf document? soup data forge, somehow doesn't work with anthropic or ollama or vllm, and using those I get connection refused error.

data generate, takes a single prompt instead of document, so I feel I am missing out on the context.

Is there a simpler way to ingest and process the data without hassle?

[MakazhanAlpamys]

Hi @digsganit98, welcome! Soup has no single "PDF in, model out" command today, but the pieces exist.

PDF to text. soup data ingest writes JSONL, one row per page. It needs pypdf, which is not in any Soup extra:

pip install pypdf
soup data ingest book.pdf --output book.jsonl

Why "connection refused". data forge and data generate call a model server that you run yourself. Soup's defaults are http://localhost:11434 (ollama) and http://localhost:8000 (vllm). If nothing listens there, you get exactly that error: start Ollama (with the model pulled) or your vLLM server first, or pass --judge-base-url (forge) / --api-base (generate). anthropic is not local; it needs ANTHROPIC_API_KEY set, plus a model name via --judge-model (forge defaults to llama3.1). Without --judge-provider, forge only runs an offline stub that writes placeholder answers.

Two caveats: forge reads .txt/.md/.json/.jsonl files in a folder as raw text, so give it plain text (not the page-JSONL), e.g. python -c "from pypdf import PdfReader as R; open('docs/book.txt','w',encoding='utf-8').write('\n\n'.join(p.extract_text() or '' for p in R('book.pdf').pages))". And generate --template qa --context doc.txt does take a document, but only its first 8000 characters per request, so split long documents.

Two honest paths

1. Continued pre-training: closest to "the model should know this document". Use the ingested JSONL directly (a text field is detected as plaintext):

base: HuggingFaceTB/SmolLM2-1.7B
task: pretrain
data:
  train: ./book.jsonl
  format: plaintext
  max_length: 2048
training:
  epochs: 3
  quantization: 4bit
  lora: {r: 64, alpha: 16}
output: ./output_pretrain

Check it with soup train --config soup.yaml --dry-run. Trade-off: it absorbs wording and facts, but does not teach question answering.

2. Q&A or RAFT, then SFT: teaches answering. Trade-off: quality depends on the generator model, and forge rows keep the passage in the prompt, so they teach reading with the text present. Soup has no builder that makes RAFT rows from a PDF.

Docs: continued pre-training, RAFT, forge, generate. No open issue covers PDF ingestion yet.