GenerateJob

GenerateJob

Chat/LLM generation: one prompt per row, built from a template over the row's columns. The output is text, or a validated dict when output_schema is set.

dib.GenerateJob(
    model,                      # spec or URI, see Model Sources
    prompt,                     # "{column}" template, rendered per row
    system=None,                # static system prompt
    output_schema=None,         # pydantic model class or JSON-schema dict
    sampling=dib.Sampling(),    # generation parameters
    engine="auto",              # see Engines
    engine_options={},          # engine-specific settings
)

Prompt templates

Python str.format syntax with column names as fields:

prompt="Product: {title}\nPrice: {price:.2f} EUR\nReview: {review}\n\nAnswer in JSON {{...}}"

• Fields must be plain column names. Positional ({}, {0}) and attribute/index access ({row.x}, {a[0]}) are rejected when the job is created.
• {{ and }} produce literal braces.
• Format specs work ({price:.2f}). Null values (None, NaN, NA) render as empty strings.
• Every referenced column must exist in the input (checked before the run). A row dict that lacks a key gets a PromptRenderError for that row only.
• system is static text: braces in it are literal, no fields are filled in.

Messages sent to the model: an optional system message, then one user message with the rendered prompt. Local engines apply the model's chat template (a plain User:/Assistant: format if the tokenizer has none).

Structured output

from typing import Literal
from pydantic import BaseModel

class Extraction(BaseModel):
    company: str | None
    amount_eur: float | None
    category: Literal["invoice", "receipt", "other"]
    tags: list[str]

job = dib.GenerateJob(model=..., prompt="Extract from:\n{text}", output_schema=Extraction)

• The JSON schema of the model is passed to the engine. openai uses response_format={"type": "json_schema", ...} and falls back to prompting automatically if the server rejects it. vllm uses guided/structured decoding. transformers adds the schema and a "respond with JSON only" instruction to the system prompt.
• The answer is parsed (code fences and surrounding prose are tolerated) and validated with pydantic. Failures become row errors (SchemaValidationError, with the validation problems and the start of the raw output).
• The value is a JSON-native dict (model_dump(mode="json")): dates and datetimes are ISO strings, enums are their values. It is identical on every backend.
• A JSON-schema dict is sent to the engine too, but only checked for being valid JSON, not validated. Use a pydantic model when you need validation.
• On Spark the column becomes a struct. See Output Contract.

Sampling

option	type	default	description
temperature	float	0.0	Sampling temperature; 0 = greedy (deterministic).
max_tokens	int	512	Maximum number of generated tokens per row.
top_p	float | None	None	Nucleus sampling; None = engine default.
stop	tuple[str, ...] | None	None	Stop sequences; generation ends before the first match.
seed	int | None	None	Random seed, where the engine supports it.

Sampling is shared by all rows of a job. With temperature=0 the transformers engine decodes greedily. The seed is passed to engines that support it.

Examples

Classification with a Databricks Foundation Model endpoint:

job = dib.GenerateJob(
    model=dib.models.databricks("databricks-meta-llama-3-3-70b-instruct"),
    system="You are a precise support-ticket router.",
    prompt="Ticket:\n{subject}\n\n{body}\n\nWhich team should handle it?",
    output_schema=Route,
    sampling=dib.Sampling(temperature=0, max_tokens=64),
    engine_options={"max_concurrency": 16, "max_retries": 8},
)
routed = job.run(tickets_df, backend="spark", partitions=16)

A self-hosted model from Unity Catalog on GPUs:

job = dib.GenerateJob(
    model=dib.models.uc("main.llm.qwen2_5_7b_instruct", alias="prod", stage_dir="/Volumes/main/llm/stage"),
    prompt="Summarize:\n{document}",
    engine_options={"max_model_len": 8192, "gpu_memory_utilization": 0.9},
)

Check a few rows before a big run:

print(job.preview(df, n=3))

See also: Running Jobs, Engines.