Proposal: activation capture, probes, and model steering in EDSL, with a Modal-backed service

[johnjosephhorton]

Proposal

Add support for activation capture, probe training, and inference-time steering to EDSL, with an optional Expected Parrot-hosted backend. Modal looks like a plausible infrastructure provider for that backend.

The motivating paper is Interpreting and Steering LLM Agents for Social Simulations (PDF). It compares prompt-based manipulation, sparse autoencoder (SAE) feature steering, and probe-direction steering in social simulation tasks. The paper already uses EDSL, including a custom Goodfire controller bridge for SAE steering; its probe experiments use PyTorch/Hugging Face hooks and scikit-learn logistic regression. Strong prompting can outperform SAE steering on creativity tasks, so strong prompt baselines should be part of the design.

The proposed user workflow is:

Run a survey, learn a behavioral direction, and test that intervention across new surveys—using the same experiment and results workflow.

This is a design proposal, not an implemented feature. The new APIs below are illustrative. No hosted deployment or performance benchmark has been completed.

What probes actually do

A probe is a small model trained to read information from a larger model's internal activations. The base language model stays frozen.

For a lottery experiment:

1. Present a prompt such as choosing between 50 tokens guaranteed and a 50% chance of 120 tokens.
2. Record an activation vector h at a specified intermediate layer and token position, such as the final prompt token before generation.
3. Let the model answer, and label the example according to its observed choice: Safe or Risky.
4. Repeat across varied examples and fit a linear classifier, such as logistic regression: score = wᵀh + b.
5. Evaluate the classifier on held-out scenarios.

Only the probe's weights change during fitting. If the activations and labels have already been saved, fitting the probe does not require loading the original language model.

Steering is a separate operation. During a new generation, intercept the activation and replace it with:

h_new = h + strength × w / ||w||

Then let the remaining layers continue. Positive and negative strengths move the state toward opposite sides of the probe's decision boundary. This modifies the current computation, not the prompt or the model's stored weights.

Reading activations is enough for probe fitting and scoring. Steering additionally requires write access during inference. These are usually intermediate-layer activations; output text and token log probabilities are not substitutes.

Increasing the linear probe's score does not guarantee a corresponding behavioral change. A probe might recognize payoff size, wording, or an answer already present in the context instead of a general behavioral trait. Prediction, intervention effects, and cross-task generalization therefore need separate evaluation.

SAEs offer a related interface: inspect sparse features and intervene on selected features. They require a compatible SAE as well as access to the model runtime. Training new SAEs would be a separate, larger project.

What writing this in EDSL could look like

Ordinary experiment: existing EDSL functionality

from edsl import (
    Agent, Model, Scenario, ScenarioList,
    Survey, QuestionMultipleChoice,
)

question = QuestionMultipleChoice(
    question_name="choice",
    question_text="""
    Choose between:
    Safe: receive 50 tokens for certain.
    Risky: a 50% chance of receiving {{ prize }} tokens,
           otherwise receive nothing.
    """,
    question_options=["Safe", "Risky"],
)

survey = Survey([question])
scenarios = ScenarioList([
    Scenario({"prize": prize})
    for prize in [40, 80, 100, 120, 160]
])
agent = Agent(traits={}, instruction="Answer the question.")

# Requires an already-running server exposing this model name.
model = Model(
    "research-model",
    service_name="openai_compatible",
    base_url="http://localhost:8000/v1",
    temperature=0.7,
)

jobs = survey.by(scenarios).by(agent).by(model)
baseline = jobs.run(
    n=20,
    disable_remote_inference=True,
    disable_remote_cache=True,
)
baseline.select("scenario.prize", "answer.choice")

Here disable_remote_inference=True means EDSL contacts the configured endpoint directly instead of using Expected Parrot inference. It does not mean EDSL loads weights locally. A hosted Expected Parrot integration would use its remote execution path instead.

Prompt manipulation already fits the existing interface: replace the neutral agent with, for example, Agent(traits={"risk_attitude": "Enjoys taking risks"}), and rerun the survey. Few-shot examples and reasoning prompts can also be expressed through ordinary survey/prompt construction.

Steering: proposed functionality

# PROPOSED API: these interfaces do not currently exist.
from edsl.interpretability import Probe

probe = Probe.load("risk_choice.probe")
steered_model = model.with_intervention(
    probe.steer(strength=2.0, scope="generated_tokens")
)

steered = (
    survey.by(scenarios).by(agent).by(steered_model)
    .run(
        n=20,
        disable_remote_inference=True,
        disable_remote_cache=True,
    )
)

The survey and neutral agent stay the same. The model configuration tells a compatible server where and how to intervene. An ordinary chat-completions server would not support this automatically.

with_intervention() should return a separate configuration so the baseline model remains unchanged. Multiple strengths could become a ModelList sweep. The intervention is a model-runtime setting; any higher-level agent interface would ultimately need to resolve to such a setting for each request.

Preparing a probe: proposed functionality

# PROPOSED API: requires a backend supporting activation capture.
from edsl.interpretability import ActivationCapture, Probe

# training_jobs uses separate training/development scenarios.
training_results = training_jobs.run(
    capture=ActivationCapture(
        layers=[12, 16, 20],
        position="last_prompt_token",
    ),
    disable_remote_inference=True,
    disable_remote_cache=True,
)

probe = Probe.fit(
    results=training_results,
    positive_answer={"choice": "Risky"},
    split_by="scenario.prize",
)
probe.save("risk_choice.probe")

Layer selection, regularization, and strength calibration use development data. Final evaluation uses untouched scenarios, ideally including new task formulations. The first artifact should be described as a risky-choice probe until stronger evidence supports interpreting it as a general risk-preference measure.

The .probe suffix is illustrative. Whether probes should become first-class .ep packages is an open design question.

Integration points and missing pieces

EDSL already has an OpenAI-compatible inference service. In the checkout inspected for this proposal, its inherited request builder forwards a fixed parameter set. Simply adding a controller or steering field would not send it to the server. The Goodfire bridge described in the paper was not present in the inspected inference-service implementation.

The additions would be:

• Serializable intervention specifications: model revision, hook location, token scope, vector or SAE identity, strength, normalization, and artifact content hash.
• Backend capability checks: explicitly distinguish generation, activation capture, probe scoring, vector steering, and SAE support. Reject unsupported operations.
• Activation/probe artifacts: store large arrays outside result rows; preserve references, dimensions, labels, preprocessing, training splits, and validation metrics.
• Result metadata: retain applied intervention details, capture timing, artifact references, and probe/feature summaries alongside answers.
• Evaluation workflows: behavioral curves, strength sweeps, blind judge panels where appropriate, response validity, and effects on unrelated tasks.

Could Expected Parrot host this on Modal?

Yes. Modal supplies custom GPU containers, multi-GPU execution, HTTP serving, and autoscaling. Expected Parrot would build and operate the instrumented runtime and experiment integration on top. See GPU support and serving guidance.

EDSL survey / experiment
          ↓
Expected Parrot API and job orchestration
  authentication, quotas, artifact references
          ↓
Modal GPU worker
  frozen model + capture/intervention hooks
          ↓
Answers + activation files + experiment metadata

The minimum backend needs three operations:

Operation	GPU work
Generate	Answer prompts normally
Capture	Save selected activations while processing prompts/generating
Steer	Generate while applying supplied directions at specified locations

Probe fitting can run separately, often on CPU. Workers should load weights once per container and reuse them across requests. Large captures should be written as artifacts, with compact references returned through the API.

Start with one instruction-tuned model around 8B parameters and a straightforward PyTorch/Hugging Face runtime. Its BF16 weights alone occupy approximately 16 GB; a 48 GB GPU is a plausible starting point, subject to actual context, batch, and runtime memory requirements. Larger models and optimized serving engines can follow measured demand.

Accept structured operations initially rather than arbitrary user Python. This keeps the public service's execution and isolation requirements bounded. Users would need neither a Modal account nor their own GPU setup.

Experimental correctness and operational requirements

1. Cache identity. Intervention specifications and artifact hashes must survive serialization and affect cache keys. A cached text response cannot satisfy a capture request whose activation artifact is missing.
2. Request isolation. Mutable hooks, controllers, random-number state, and KV caches must not leak across requests. Start with one active experiment batch per worker, then introduce concurrency with explicit isolation.
3. Comparable controls. Use the same checkpoint, precision, tokenizer, prompts, and runtime for baseline and intervention conditions. Test zero-strength equivalence, hook cleanup after failures, and mixed-condition isolation.
4. Precise capture semantics. Distinguish prompt-only measurements from measurements taken after the answer appears. Replaying a completed transcript must be labeled separately from a trace captured during the original generation.
5. Preprocessing and direction semantics. A classifier trained on standardized activations needs an explicit mapping back to the coordinates used for scoring and steering. Record normalization and intervention position conventions.
6. Activation volume. Default to selected layers and positions, not every token at every layer. Storage and transfer can dominate otherwise.
7. Validation beyond prediction. Include held-out tasks, positive/zero/negative strength sweeps, matched random-direction controls, malformed-response rates, and unrelated behavioral measures.
8. Deployment reproducibility. Pin checkpoint and runtime versions and record hardware/precision details. Do not assume a seed alone guarantees identical behavior across different serving engines.

Modal GPU snapshots may help some cold-start workloads, but they remain an alpha feature with limitations, including multi-GPU compatibility. They should be an optional optimization, not a requirement for the first version. Snapshot documentation

Alternatives and possible partners

These findings come from provider documentation reviewed on September 16, 2026; availability and access conditions should be rechecked before integration.

Service	Relevance
NDIF + NNsight	Remote activation access and interventions without local weights. NDIF offers free research infrastructure. A useful way to prototype an adapter before operating our own fleet.
Goodfire / Silico	Commercial interpretability tooling on their infrastructure or a customer's. Current pricing lists an academic/nonprofit plan at $1,000/month. A public arbitrary-activation/steering API was not verified from current product pages; the older Ember SDK is archived.
ACS Infra	Documents activation capture and user-supplied steering vectors. An application-based beta, currently serving base models.
Neuronpedia	Hosted SAE-feature steering with a documented API; a narrower interface than arbitrary activation access for custom probes.

The intervention specification and artifact formats should be independent of provider so a hosted Expected Parrot backend and external research backends can share the EDSL-facing interface.

Scope, economics, and a first milestone

The recommended first milestone is one model, one lottery survey, prompt baselines, activation capture, a fitted probe, and a held-out steering-strength sweep. Deliver the answers, artifacts, and comparison curves as a reproducible EDSL workflow.

Rough engineering estimates, assuming an experienced developer and available GPU access:

• Reusable prompt-only experiments and evaluation: a few days.
• Narrow capture/train/steer prototype: approximately 1–2 weeks.
• Reliable private beta with isolation, caching, artifacts, quotas, and failure recovery: several additional weeks.
• SAE support using a compatible pretrained SAE: additional backend-dependent work. Training new SAEs is outside the initial scope.

Modal's listed GPU rates at the time of review are approximately $1.95 per L40S GPU-hour and $3.95 per H100 GPU-hour, before CPU, memory, storage, and other charges. Thus 100 L40S GPU-hours would be about $195 in GPU charges. This is not a throughput estimate or a customer price; we need to benchmark model loading, capture, generation, and artifact handling. Pricing

Questions for discussion

• Should the first backend be an NDIF adapter, an Expected Parrot-managed Modal deployment, or both behind one interface?
• Should probes and steering configurations become portable .ep objects?
• Is activation capture plus vector steering the right initial scope, with SAE inspection added later?
• Which small instruction-tuned model and behavioral task would make the most useful first reproducible example?