Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,9 +221,9 @@ Coverage highlights:
docs/
├── concepts.md # Core concepts and evaluation scenarios
├── how-it-works.md # Architecture and request flow
├── tutorial-prompt-agent-quickstart.md # Quickstart for Foundry Prompt Agents
├── tutorial-hosted-agent-quickstart.md # Quickstart for Foundry hosted / HTTP agents
├── tutorial-end-to-end.md # Full Foundry + AgentOps release-readiness workshop
├── tutorial-prompt-agent-quickstart.md # Tutorial for Foundry Prompt Agents
├── tutorial-hosted-agent-quickstart.md # Tutorial for Foundry hosted / HTTP agents
├── tutorial-end-to-end.md # End-to-end release-readiness tutorial (Foundry + AgentOps)
├── doctor-explained.md # Doctor checks, history, and operations model
├── ci-github-actions.md # CI/CD setup
├── release-process.md # Release and versioning
Expand Down
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,22 @@ This format follows [Keep a Changelog](https://keepachangelog.com/) and adheres
## [Unreleased]

### Changed
- **Tutorial wording: "quickstart" → "tutorial", "workshop" → "tutorial".**
The three documentation entries that were labeled "Prompt Agent quickstart",
"Hosted Agent quickstart", and "End-to-end workshop" now read as "Foundry
Prompt Agent tutorial", "Hosted or HTTP Agent tutorial", and "End-to-end
tutorial" across `README.md`, `plugins/agentops/README.md`, `AGENTS.md`,
`docs/concepts.md`, `docs/doctor-explained.md`, the `agentops-workflow`
skill (both synced copies), and the H1s + cross-references inside each
tutorial doc. The README description for the end-to-end tutorial now also
states explicitly that it **extends** either of the type-specific tutorials
(sandbox → dev → qa → prod plus Foundry red-team scans plus
trace-to-regression promotion) so the difference between the three is
obvious at a glance. The "quickstart" framing no longer fits doc bodies
that grew past 1000 lines covering multi-environment promotion, regression
injection, Doctor evidence, and Cockpit. The tutorial **filenames are
intentionally preserved** (`tutorial-*-quickstart.md`) to keep inbound
links and bookmarks stable.
- **Skill + tutorial guidance now require `Cognitive Services OpenAI User` as a prerequisite RBAC role.**
The `agentops-workflow` skill, `tutorial-prompt-agent-quickstart.md`,
`tutorial-end-to-end.md`, and `docs/ci-github-actions.md` now instruct users
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,9 +210,9 @@ Cockpit sections, in display order:

## Documentation

- [Prompt Agent quickstart](docs/tutorial-prompt-agent-quickstart.md) - use this when the Foundry target is `agent: name:version`.
- [Hosted Agent quickstart](docs/tutorial-hosted-agent-quickstart.md) - use this when the target is a Foundry hosted or HTTP endpoint URL.
- [End-to-end workshop](docs/tutorial-end-to-end.md) - complete Foundry + AgentOps journey: create, debug, evaluate, release, observe, red-team follow-through, and trace regression.
- [Foundry Prompt Agent tutorial](docs/tutorial-prompt-agent-quickstart.md) - use this when the Foundry target is `agent: name:version`. Walks the sandbox → dev journey with a PR gate.
- [Hosted or HTTP Agent tutorial](docs/tutorial-hosted-agent-quickstart.md) - use this when the target is a Foundry hosted or HTTP endpoint URL. Same sandbox → dev journey for endpoint-based agents.
- [End-to-end tutorial](docs/tutorial-end-to-end.md) - extends either of the above with the full sandbox → dev → qa → prod promotion, Foundry red-team scans, and trace-to-regression promotion.
- [Core concepts](docs/concepts.md)
- [How it works](docs/how-it-works.md)
- [Doctor explained](docs/doctor-explained.md)
Expand Down
6 changes: 3 additions & 3 deletions docs/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,11 +180,11 @@ AgentOps auto-selects common evaluation patterns from the dataset:

Use one of the three hands-on tutorials for scenario coverage:

- [Prompt Agent quickstart](tutorial-prompt-agent-quickstart.md) for Foundry
- [Foundry Prompt Agent tutorial](tutorial-prompt-agent-quickstart.md) for Foundry
prompt agents referenced as `name:version`.
- [Hosted Agent quickstart](tutorial-hosted-agent-quickstart.md) for Foundry
- [Hosted or HTTP Agent tutorial](tutorial-hosted-agent-quickstart.md) for Foundry
hosted endpoints, generic HTTP agents, RAG services, and code-based workflows.
- [End-to-end workshop](tutorial-end-to-end.md) for the complete Foundry +
- [End-to-end tutorial](tutorial-end-to-end.md) for the complete Foundry +
AgentOps loop, including CI/CD, observability, red-team follow-through,
Doctor, release evidence, and trace regression.

Expand Down
4 changes: 2 additions & 2 deletions docs/doctor-explained.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
A 10-minute read for a platform, observability, or AI engineer (and
the engineering managers who own those teams) who runs
`agentops doctor` for the first time. For step-by-step setup, see
the [end-to-end workshop](tutorial-end-to-end.md).
the [end-to-end tutorial](tutorial-end-to-end.md).

## 1. What the Doctor is - and isn't

Expand Down Expand Up @@ -411,7 +411,7 @@ members and CI.
## 11. Next steps

- Walk through a full setup with Azure resources:
[end-to-end workshop](tutorial-end-to-end.md).
[end-to-end tutorial](tutorial-end-to-end.md).
- Open the workspace command center: `agentops cockpit` shows eval
history, Doctor findings, CI/CD status, telemetry readiness, and
Foundry/Azure navigation.
Expand Down
6 changes: 3 additions & 3 deletions docs/tutorial-end-to-end.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# End-to-end tutorial: release readiness for Foundry agents

This tutorial is the full path. Use it after one of the quickstarts when you
This tutorial is the full path. Use it after one of the type-specific tutorials when you
want to validate the complete develop -> evaluate -> release -> observe loop
across **sandbox**, **dev**, **qa**, and **prod** environments. The two
quickstarts cover the same loop for a single agent type in a sandbox + dev
type-specific tutorials cover the same loop for a single agent type in a sandbox + dev
arrangement; this tutorial expands the journey through every release stage.

It is inspired by the Azure Samples repo
Expand Down Expand Up @@ -409,7 +409,7 @@ agentops workflow generate `
> not yet contain the agent, it reads that block plus `prompt_file` and
> creates the first version automatically. No per-environment manual
> seeding. See the
> [prompt-agent quickstart](tutorial-prompt-agent-quickstart.md) for the
> [prompt-agent tutorial](tutorial-prompt-agent-quickstart.md) for the
> full multi-environment journey.

Before running that workflow, make the PR gate runnable in GitHub. Install the
Expand Down
16 changes: 8 additions & 8 deletions docs/tutorial-hosted-agent-quickstart.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Quickstart: Foundry Hosted Agent or HTTP Agent (sandbox → dev with PR gate)
# Tutorial: Foundry Hosted Agent or HTTP Agent (sandbox → dev with PR gate)

Use this quickstart when the agent is reachable as an endpoint URL. The
Use this tutorial when the agent is reachable as an endpoint URL. The
example creates a small **Travel Agent** HTTP endpoint locally (your
**sandbox**), then shows how to swap in a deployed Foundry Hosted Agent
or cloud-hosted URL (your **dev** environment) for CI.
Expand All @@ -14,7 +14,7 @@ arrangement:
normalized `results.json`, runs Doctor with `--severity-fail critical`
so regressions block the PR, and produces release evidence.

The toolkit benefit is the same as the prompt-agent quickstart, adapted
The toolkit benefit is the same as the prompt-agent tutorial, adapted
for endpoint-based agents: you author and iterate against a local
sandbox, then let CI verify the deployed dev environment is still
healthy on every PR. Production-readiness gates (eval thresholds plus
Expand Down Expand Up @@ -54,7 +54,7 @@ unexpected permission prompts.
| You can create an Entra app registration with federated credentials, or an admin is ready to provide the client ID, tenant ID, and subscription ID. | The workflow skill can wire OIDC cleanly; without this, CI cannot authenticate to Azure. |
| Copilot or your coding-agent CLI is signed in before you ask it to run AgentOps skills. | The skill handoff assumes an authenticated coding-agent session that can read the repo and propose GitHub/Azure setup steps. |

Unlike the Prompt Agent quickstart, this endpoint tutorial does not point the
Unlike the Prompt Agent tutorial, this endpoint tutorial does not point the
generated PR workflow at `ai-agent-evals`. Hosted and HTTP agents are evaluated
through the AgentOps local runner because CI must invoke your endpoint, extract
the response, apply repo thresholds, and write the normalized `results.json`.
Expand Down Expand Up @@ -132,7 +132,7 @@ green dev → ready for promotion to qa / prod
> evaluate the PR's *unmerged* code unless your CI does a per-PR
> ephemeral deploy. If you need PR-time regression catching for hosted
> agents, the workflow skill can guide you through adding a per-PR
> ephemeral deploy step (out of scope for this quickstart). The
> ephemeral deploy step (out of scope for this tutorial). The
> sandbox loop (local FastAPI + `agentops eval run`) is the
> equivalent author-side gate.

Expand Down Expand Up @@ -674,7 +674,7 @@ explicitly) so it uses dev. The Foundry project endpoint plus the agent URL
environment) together let CI evaluate the deployed dev endpoint and land
results in the dev observability target.

Use the same workflow-skill handoff pattern as the Prompt Agent quickstart, but
Use the same workflow-skill handoff pattern as the Prompt Agent tutorial, but
keep the scope to the hosted endpoint:

```powershell
Expand All @@ -699,7 +699,7 @@ permission.

Open both Doctor outputs. The report explains the findings; the evidence pack
summarizes what a reviewer needs to decide whether the endpoint is releasable.
In a fresh quickstart, warnings about production telemetry, CI history, or trace
In a fresh tutorial workspace, warnings about production telemetry, CI history, or trace
regression history are expected and useful: they show what remains before this
local endpoint becomes an operated service.

Expand Down Expand Up @@ -761,7 +761,7 @@ You are done when:
`.azure/qa/.env` and `.azure/prod/.env`, set GitHub Environments with
the right `AGENTOPS_AGENT_ENDPOINT`, and use `agentops workflow
generate --kinds qa,prod --force`.
- **Walk through the prompt-agent quickstart** at
- **Walk through the prompt-agent tutorial** at
[tutorial-prompt-agent-quickstart.md](tutorial-prompt-agent-quickstart.md)
to see the full prompt-as-code regression journey (stage-then-eval
at PR time, no per-PR deploys required) and contrast the two
Expand Down
6 changes: 3 additions & 3 deletions docs/tutorial-prompt-agent-quickstart.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Quickstart: Foundry Prompt Agent (sandbox → dev with PR gate)
# Tutorial: Foundry Prompt Agent (sandbox → dev with PR gate)

Use this quickstart when you want a Foundry-managed prompt agent referenced as
Use this tutorial when you want a Foundry-managed prompt agent referenced as
`name:version`. The example creates a small **Travel Agent** in Foundry and
then uses AgentOps to add repo-side readiness, a PR gate that catches
regressions before merge, a `dev` deploy workflow, Doctor evidence, and
Expand Down Expand Up @@ -1077,7 +1077,7 @@ evidence. Read the output in this order:
| `Findings: N (M critical ...)` | The severity rollup; critical items are what you discuss first. |
| `Finding summary` | The terminal triage list. |

In a fresh quickstart it is normal to see warnings for scheduled CI
In a fresh tutorial workspace it is normal to see warnings for scheduled CI
(you only generated `pr` and `dev`), continuous evaluation, qa/prod
deploys, explicit thresholds, or red-team scans. Treat those as the
hardening backlog. The eval gates and the dev deploy loop are
Expand Down
6 changes: 3 additions & 3 deletions plugins/agentops/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Run the AgentOps watchdog and summarize production latency findings.
## Links

- [AgentOps Accelerator](https://github.com/Azure/agentops)
- [Prompt Agent quickstart](https://github.com/Azure/agentops/blob/main/docs/tutorial-prompt-agent-quickstart.md)
- [Hosted Agent quickstart](https://github.com/Azure/agentops/blob/main/docs/tutorial-hosted-agent-quickstart.md)
- [End-to-end workshop](https://github.com/Azure/agentops/blob/main/docs/tutorial-end-to-end.md)
- [Foundry Prompt Agent tutorial](https://github.com/Azure/agentops/blob/main/docs/tutorial-prompt-agent-quickstart.md)
- [Hosted or HTTP Agent tutorial](https://github.com/Azure/agentops/blob/main/docs/tutorial-hosted-agent-quickstart.md)
- [End-to-end tutorial](https://github.com/Azure/agentops/blob/main/docs/tutorial-end-to-end.md)
- [How it works](https://github.com/Azure/agentops/blob/main/docs/how-it-works.md)
4 changes: 2 additions & 2 deletions plugins/agentops/skills/agentops-workflow/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ by discovering the whole Azure subscription.
- `azd env get-values` when `azure.yaml` exists and azd is available.
- `.github/workflows/agentops-*.yml`.
2. Read the generated workflows to determine exactly which GitHub environments
and variables are needed. For the prompt-agent quickstart, `pr` normally
and variables are needed. For the prompt-agent tutorial, `pr` normally
means only `environment: dev`.
3. Treat `dev` here as a GitHub Actions environment for OIDC and variables. It
normally points at the Foundry project already configured by `agentops init`;
Expand Down Expand Up @@ -302,7 +302,7 @@ so the PR-comment step can post.
### GitHub Actions (OIDC)

At the GitHub Environment level when the workflow declares an environment
(preferred for the quickstart), or at repository level when intentionally shared
(preferred for the tutorials), or at repository level when intentionally shared
across environments, set:

- `AZURE_CLIENT_ID` - App registration / managed identity used for OIDC.
Expand Down
4 changes: 2 additions & 2 deletions src/agentops/templates/skills/agentops-workflow/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ by discovering the whole Azure subscription.
- `azd env get-values` when `azure.yaml` exists and azd is available.
- `.github/workflows/agentops-*.yml`.
2. Read the generated workflows to determine exactly which GitHub environments
and variables are needed. For the prompt-agent quickstart, `pr` normally
and variables are needed. For the prompt-agent tutorial, `pr` normally
means only `environment: dev`.
3. Treat `dev` here as a GitHub Actions environment for OIDC and variables. It
normally points at the Foundry project already configured by `agentops init`;
Expand Down Expand Up @@ -302,7 +302,7 @@ so the PR-comment step can post.
### GitHub Actions (OIDC)

At the GitHub Environment level when the workflow declares an environment
(preferred for the quickstart), or at repository level when intentionally shared
(preferred for the tutorials), or at repository level when intentionally shared
across environments, set:

- `AZURE_CLIENT_ID` - App registration / managed identity used for OIDC.
Expand Down
Loading