Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,20 @@ This format follows [Keep a Changelog](https://keepachangelog.com/) and adheres
## [Unreleased]

### Added
- **Auto-bootstrap empty Foundry projects on first deploy.** New optional
`prompt_agent_bootstrap` block in `agentops.yaml` lets the prompt-agent
deploy workflow create the first version of an agent in a dev / qa / prod
Foundry project that does not yet have one. When the stage step looks up
the seed agent and gets a 404, it reads the model deployment (required)
plus optional `description`, `model_parameters`, and `tools` from
`prompt_agent_bootstrap`, combines them with `prompt_file`, and creates
the first version automatically. The deployment artifact records the new
`action: "bootstrapped"` for that first run; subsequent deploys follow
the normal reuse / next-version flow. Eliminates the previous
per-environment manual seeding step. `agentops workflow analyze` now
warns when a prompt-agent workspace is missing this block. Authentication
(401 / 403) and other non-404 errors continue to propagate — the
bootstrap path only triggers on a genuine "agent does not exist" 404.
- **`--doctor-gate` flag on `agentops workflow generate`.** New option
`--doctor-gate critical|warning|none` controls the Doctor severity floor
in the PR workflow template. Default is `critical`, which makes the PR
Expand Down
10 changes: 10 additions & 0 deletions docs/tutorial-end-to-end.md
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,16 @@ agentops workflow generate `
> warnings during hardening sprints. Use `--doctor-gate none` to make Doctor
> advisory-only (the pre-`--doctor-gate` behavior).

> **Promoting prompt agents across multiple Foundry projects?** Add a
> `prompt_agent_bootstrap` block (model deployment plus optional
> description, model_parameters, and tools) to `agentops.yaml`. When the
> deploy workflow runs against a dev / qa / prod Foundry project that does
> not yet contain the agent, it reads that block plus `prompt_file` and
> creates the first version automatically. No per-environment manual
> seeding. See the
> [prompt-agent quickstart](tutorial-prompt-agent-quickstart.md) for the
> full multi-environment journey.

Before running that workflow, make the PR gate runnable in GitHub. Install the
AgentOps workflow skill if needed:

Expand Down
190 changes: 133 additions & 57 deletions docs/tutorial-prompt-agent-quickstart.md

Large diffs are not rendered by default.

14 changes: 14 additions & 0 deletions plugins/agentops/skills/agentops-config/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,20 @@ thresholds:
groundedness: ">=3"
avg_latency_seconds: "<=30"

# Prompt-agent only: auto-bootstrap empty Foundry projects on first deploy.
# When the deploy workflow runs against a Foundry project that does not yet
# contain the agent named in `agent:`, AgentOps reads this block plus
# `prompt_file` and creates the first version automatically. Recommended
# for multi-environment prompt-agent workflows (sandbox → dev → qa → prod)
# so operators do not have to manually recreate the seed agent in every
# Foundry project.
prompt_agent_bootstrap:
model: gpt-4o-mini # required - same deployment name in every env
description: "Helps plan short trips."
# model_parameters: # optional - temperature, top_p, etc.
# temperature: 0.2
# tools: [] # optional - tool definitions

# Publish results to the Foundry Evaluations panel.
# - execution: local + publish: true → Classic Foundry (uploads metrics)
# - execution: cloud → New Foundry (server-side run;
Expand Down
20 changes: 15 additions & 5 deletions plugins/agentops/skills/agentops-workflow/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -405,16 +405,26 @@ Prompt-agent workflows:

1. read `prompt_file` from `agentops.yaml` or
`AGENTOPS_AGENT_PROMPT_FILE`;
2. create or reuse a candidate Foundry prompt-agent version from that file;
3. generate `.agentops/deployments/agentops.candidate.yaml`;
4. run `agentops eval run` against the candidate version;
5. record `.agentops/deployments/foundry-agent.json` as a deployment
2. look up the seed agent in the active environment's Foundry project;
if it does not exist (typical first deploy into dev / qa / prod),
read the optional `prompt_agent_bootstrap` block from
`agentops.yaml` (required `model`, optional `description`,
`model_parameters`, `tools`) plus `prompt_file` and create the
first version automatically (recorded as `action: "bootstrapped"`);
3. otherwise create or reuse a candidate Foundry prompt-agent version
from `prompt_file`;
4. generate `.agentops/deployments/agentops.candidate.yaml`;
5. run `agentops eval run` against the candidate version;
6. record `.agentops/deployments/foundry-agent.json` as a deployment
artifact only when the gate passes.

This avoids the bad pattern of evaluating one agent version and deploying a
different prompt. The invariant is: **evaluated version == deployed version**.
Foundry manages agent versions; AgentOps owns the repo-side gate and
deployment record.
deployment record. For multi-environment prompt-agent workflows
(sandbox → dev → qa → prod), strongly recommend adding the
`prompt_agent_bootstrap` block so operators do not have to manually
recreate the seed agent in every Foundry project.

If this is not a Foundry prompt agent and azd is not ready, generate
`--kinds pr` only or use `--deploy-mode placeholder`. Do not ship
Expand Down
89 changes: 89 additions & 0 deletions src/agentops/core/agentops_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,86 @@ def _version_non_empty(cls, value: str) -> str:
return value


class PromptAgentBootstrap(BaseModel):
"""Bootstrap defaults for prompt-agent CI/CD when the target Foundry
project does not yet contain the seed agent referenced by ``agent``.

AgentOps' Foundry prompt-agent deployment path normally looks up an
existing seed (``name:version``) in the target project, clones its
definition, and replaces the instructions with ``prompt_file``. That
forces every environment (sandbox, dev, qa, prod) to have the agent
pre-created manually.

When ``prompt_agent_bootstrap`` is set, the deployment step instead
bootstraps the agent in any environment whose target Foundry project
is still empty (the seed lookup returns 404) using these values plus
the contents of ``prompt_file``. The action recorded in the
deployment artifact will be ``bootstrapped`` for that first run.

This block is **only** consulted on the not-found code path. Once
the agent exists in the target project, the reuse / next-version
flow takes over and ``prompt_agent_bootstrap`` is ignored — changing
``model`` here will not migrate an existing dev agent to a new
deployment. Treat schema changes beyond ``instructions`` as a
deliberate operations event.

Fields:

``model``
Required. Azure OpenAI / Foundry model deployment name to use
when creating the agent. Must exist with the same name in every
environment that may bootstrap (sandbox, dev, qa, prod).

``description``
Optional human-readable description recorded on the agent.

``model_parameters``
Optional dict of model parameters (e.g. ``{"temperature": 0.2}``)
passed through to the Foundry ``PromptAgentDefinition``.

``tools``
Optional list of tool definitions (JSON-serializable dicts that
match the Foundry tools schema) registered with the agent at
bootstrap time.
"""

model: str = Field(
...,
description=(
"Model deployment name. Must exist with the same name in "
"every Foundry project that may bootstrap from this config."
),
)
description: Optional[str] = Field(
None,
description="Optional human-readable description for the agent.",
)
model_parameters: Optional[Dict[str, Any]] = Field(
None,
description=(
"Optional model parameters dict (e.g. {'temperature': 0.2}) "
"passed through to Foundry PromptAgentDefinition."
),
)
tools: Optional[List[Dict[str, Any]]] = Field(
None,
description=(
"Optional tool definitions (JSON dicts matching Foundry "
"tools schema) registered with the agent at bootstrap."
),
)

model_config = ConfigDict(extra="forbid", protected_namespaces=())

@field_validator("model")
@classmethod
def _model_non_empty(cls, value: str) -> str:
value = value.strip()
if not value:
raise ValueError("prompt_agent_bootstrap.model must be non-empty")
return value


# ---------------------------------------------------------------------------
# Top-level config
# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -339,6 +419,15 @@ class AgentOpsConfig(BaseModel):
default_factory=DatasetSyncConfig,
description="Cloud evaluation dataset submission policy.",
)
prompt_agent_bootstrap: Optional[PromptAgentBootstrap] = Field(
None,
description=(
"Optional bootstrap defaults used when the prompt-agent "
"deployment target is empty (seed lookup returns 404). "
"Lets CI/CD auto-create the agent in dev/qa/prod from "
"sandbox-only authoring. See PromptAgentBootstrap docs."
),
)

model_config = ConfigDict(extra="forbid")

Expand Down
Loading
Loading