Skip to content

LLM Integration & Prompt Engineering

Jump to bottom
Chazona Baum edited this page Jun 24, 2026 · 1 revision

Relevant source files

Lodestar leverages Large Language Models (LLMs) via OpenRouter to perform high-fidelity extraction, research, and candidate-job alignment. The system is designed for defensive parsing, cost transparency, and latency optimization through aggressive prompt caching.

The LLM Seam: Llm Trait & OpenRouter Implementation

The codebase abstracts LLM interactions through the Llm trait, allowing for a real implementation (OpenRouterLlm) and a mock implementation (FakeLlm) for unit testing src-tauri/src/llm.rs#4-46

OpenRouterLlm

The OpenRouterLlm communicates with the OpenRouter API using a blocking reqwest client src-tauri/src/llm.rs#137-152 It handles three specialized features:

  1. Web Search Tool: When LlmRequest.web is true, the request body includes the openrouter:web_search tool, enabling the model to perform live research src-tauri/src/llm.rs#122-128
  2. Prompt Caching: Uses a cached_prefix to implement Anthropic-style prompt caching. The request body is constructed as a multipart array where the first part contains the stable prefix (e.g., the candidate's dossier) and an ephemeral cache control breakpoint with a 1-hour TTL src-tauri/src/llm.rs#110-116
  3. Cost & Telemetry Parsing: Extracts precise usage data from the usage object in the response:

Data Flow: Request to Telemetry

The following diagram illustrates how a LlmRequest is transformed into an API call and how the resulting LlmResponse populates the system's telemetry.

LLM Request/Response Lifecycle

flowchart TD
    G["usage.cost -> cost_micro_usd"]
    H["cached_tokens -> cache_read_tokens"]
    I["cache_write_tokens -> cache_write_tokens"]
    subgraph subGraph1 ["Natural Language Space (API)"]
        F["OpenRouter API"]
    end
    subgraph subGraph0 ["Code Entity Space (Rust)"]
        A["LlmRequest"]
        B["build_or_body()"]
        C["OpenRouterLlm::complete()"]
        D["OrResponse (JSON)"]
        E["LlmResponse"]
    end
    A --> B
    B --> C
    C --> D
    D --> E
    C --> F
    F --> D
    E --> G
    E --> H
    E --> I
Loading

Sources: src-tauri/src/llm.rs#8-42src-tauri/src/llm.rs#103-135src-tauri/src/llm.rs#137-152

Prompt Engineering & Defensive Parsing

All prompts are centralized in prompts.rs. The system uses a "DATA-fence" strategy to prevent prompt injection from untrusted scraped HTML src-tauri/src/prompts.rs#38-49

The Four Core Prompt Builders

Builder Tier Input Output
structure-listings Balanced Sanitized careers page Vec<StructuredListing>
structure-jd Frontier Sanitized JD text StructuredJd
research-gaps Frontier Missing fields + Web Search Vec<ResearchedField>
alignment Frontier Dossier + Job + Research FitBreakdown

Sources: src-tauri/src/prompts.rs#41-49docs/product/model-tiers.md#22-29

Injection Defense: sanitize.rs and DATA-fences

Lodestar employs a multi-layered defense against prompt injection from scraped careers pages docs/product/openrouter-guardrails.md#12-21:

  1. HTML Sanitization: sanitize.rs strips <script>, <style>, and hidden elements, while resolving relative URLs to absolute ones src-tauri/src/sanitize.rs#11-59
  2. Explicit Fencing: Sanitized text is wrapped in <<<SCRAPED_DATA>>> markers src-tauri/src/sanitize.rs#8-9
  3. Instruction Framing: The system prompt explicitly instructs the model that everything between markers is DATA and never instructions src-tauri/src/prompts.rs#42-44

Defensive Parsing

LLM responses are parsed using extract_json_array or extract_json_object. These functions are resilient to:

Prompt Caching Strategy

The alignment step is the most computationally expensive and token-heavy stage because it requires the full user profile (accomplishments, experience, positioning). To optimize this, Lodestar uses a Candidate Dossier Caching Strategydocs/product/model-tiers.md#49-58

Dossier Caching Logic

flowchart TD
    B["build_or_body()"]
    subgraph subGraph2 ["OpenRouter / Anthropic"]
        M["Multipart JSON Body"]
        C["Cache Hit?"]
        H["Read from Cache (Discounted)"]
        W["Write to Cache (Full Price)"]
    end
    subgraph subGraph1 ["Alignment Step"]
        DOS["Candidate Dossier (Prefix)"]
        JD["Job Description (Suffix)"]
        REQ["LlmRequest"]
    end
    subgraph subGraph0 ["Vault Storage"]
        P["profile/*.md"]
        A["accomplishments/*.md"]
    end
    P --> DOS
    A --> DOS
    JD --> REQ
    DOS --> REQ
    REQ --> B
    B --> M
    M --> C
    C --> H
    C --> W
Loading

Sources: src-tauri/src/llm.rs#16-24src-tauri/src/llm.rs#110-116docs/product/model-tiers.md#49-58

Research Validation State Machine

When the research-gaps step runs, it uses the OpenRouter web search tool to fill missing information (e.g., compensation, tech stack). Because web search results can be unreliable, the system applies a validation state machine src-tauri/src/llm.rs#12-15

  1. Gap Detection: The pipeline identifies missing RESEARCHABLE_FIELDS in the Job struct.
  2. LLM Research: The model is prompted to search the web and return structured research.
  3. Validation: The parse_and_validate_research logic checks for:
  • Data Quality: Does the research provide a value or a valid Rejection reason?
  • Source Reliability: Are the findings consistent with the known job context?
  1. State Update: Validated fields are merged into the Job entity, while invalid ones trigger a retry or a terminal failure state.

Sources: src-tauri/src/llm.rs#12-15src-tauri/src/prompts.rs#1-3 (Note: specific validation logic details are found in pipeline/steps.rs and research-gaps modules described in section 4.5).

Clone this wiki locally