feat(telemetry): emit lifecycle events for coded runs [PC-4888] - #1880
Merged
Conversation
github-actions
Bot
added
test:uipath-langchain
tudormatei1
marked this pull request as ready for review
September 2, 2026 15:01
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The emitted telemetry schema is internally inconsistent for FAULTED results (ErrorType duplicates ErrorCode), which can break downstream querying/alerting expectations.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR improves observability for coded agent/function runs by restoring and enriching lifecycle telemetry so cloud (job-key) executions are visible in Application Insights, and by routing telemetry to the App Insights instance that matches the authenticated UiPath environment.
Changes:
- Route telemetry to per-environment Application Insights connection strings (alpha/staging/prod) derived from
UIPATH_URL, with safe “unconfigured slot” behavior. - Emit
CodedAgentRun.*/CodedFunctionRun.*lifecycle events (Start + terminal) from the CLI run path, including a standalone scope/identity block on every event. - Update CI workflows to bake per-environment connection strings into the wheel via
envsubst, and add targeted unit tests for environment + connection string resolution.
File summaries
| File | Description |
|---|---|
| packages/uipath/uv.lock | Bumps package version in lock metadata. |
| packages/uipath/pyproject.toml | Bumps uipath version to 2.14.11. |
| packages/uipath/src/uipath/telemetry/_constants.py | Replaces single baked connection string constant with per-environment constants. |
| packages/uipath/src/uipath/telemetry/_environment.py | Adds environment resolution from UIPATH_URL host. |
| packages/uipath/src/uipath/telemetry/_track.py | Implements per-environment connection string selection and placeholder/blank handling. |
| packages/uipath/src/uipath/_cli/_run_telemetry.py | Adds coded run lifecycle telemetry emission with scope and error classification. |
| packages/uipath/src/uipath/_cli/cli_run.py | Wires RunTelemetry into the CLI execution path (start/finished/failed). |
| packages/uipath/tests/telemetry/test_track.py | Updates tests to reflect the new prod baked constant name. |
| packages/uipath/tests/telemetry/test_environment.py | Adds tests for environment resolution from base URL. |
| packages/uipath/tests/telemetry/test_connection_string.py | Adds tests for per-environment connection string routing and precedence. |
| packages/uipath/tests/cli/test_run_telemetry.py | Adds tests for coded run lifecycle telemetry and scope/error hygiene. |
| .github/workflows/build-package.yml | Bakes per-environment connection strings into build artifacts via envsubst. |
| .github/workflows/publish-dev.yml | Updates dev publish workflow to bake per-environment connection strings. |
| .github/workflows/cd.yml | Passes new per-environment secrets through to the reusable build workflow. |
Review details
- Files reviewed: 13/14 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
tudormatei1
force-pushed
the
feat/coded-agent-telemetry
branch
from
September 3, 2026 11:27
8804e88 to
aeeb797
Compare
🚨 Heads up:
|
radu-mocanu
reviewed
Sep 3, 2026
radu-mocanu
reviewed
Sep 3, 2026
radu-mocanu
reviewed
Sep 3, 2026
radu-mocanu
reviewed
Sep 3, 2026
Contributor
Author
we dont double emit as the Cli.Run contians less info and also Cli.RUN is not emitted on the cloud uipath run (cause the check with the job key is still present) . double emit only happens on local runs but that should be alrgiht? |
radu-mocanu
reviewed
Sep 3, 2026
tudormatei1
force-pushed
the
feat/coded-agent-telemetry
branch
from
September 3, 2026 13:16
26d0874 to
aeeb797
Compare
Collaborator
we will have duplicate data either way. no reason to emit an additional event with less info |
radu-mocanu
reviewed
Sep 3, 2026
The SDK baked a single connection string into the published wheel, so every run reported to the same instance regardless of which environment the user had authenticated against. Alpha and staging runs were landing in whichever instance that one secret happened to point at. The environment now comes from the host of UIPATH_URL, which uipath auth writes to .env, and selects between three baked constants. An unauthenticated run, an Automation Suite deployment or any unrecognized host resolves to production. Host matching is exact-or-subdomain on the parsed hostname so a lookalike domain cannot select another environment's instance. TELEMETRY_CONNECTION_STRING still overrides everything, since the platform injects it per environment for low-code agents. Three per-environment variants are read before the baked constants, which is what makes this testable from a source checkout where the constants are still placeholders. The old single constant is gone rather than kept as a fallback: a missing secret should leave telemetry unconfigured, not silently report production data to alpha. That means the three repo secrets have to exist before the next build, or the wheel ships with telemetry off.
Coded runs reported nothing once a job key was present. The only SDK telemetry was Cli.*, which is suppressed for Orchestrator jobs, so a coded agent running in the cloud was invisible in App Insights. CodedAgentRun.* and CodedFunctionRun.* now fire from the run path, mirroring the AgentRun.* contract low-code already emits. The two kinds are separated by the factory's agent_framework, since both report agent_type uipath_coded; low-code is skipped so uipath-agents-python stays its only emitter. Every event carries the folder, job, process, project and trace ids, so a run can be attributed without an Orchestrator lookup. Failed events carry the error code, title and category but not the message or traceback, which for a coded run hold customer source and values. The run is keyed on the job key so a suspend and resume pair stays one run, and TraceId goes through resolve_trace_id because UIPATH_TRACE_ID may be a dashed UUID while spans export as 32-char hex.
On a faulted result ErrorType was set to the contract code, so it duplicated ErrorCode and meant something different from the raised path, where it is the Python exception class. Grouping by ErrorType would mix class names with platform codes and undercount both. ErrorType is now the exception class only, and absent when the runtime returned a faulted result rather than raising. ErrorCode, ErrorTitle and ErrorCategory still carry the contract on both paths, so every failure event keeps a classification. Also adds CloudUserId to the scope assertion, which omitted it while asserting the other two cloud ids, and shortens the docstrings.
2.14.11 was published to PyPI while this branch was open, so the version-availability check started failing on it.
AutomationSuiteClusterId and AutomationSuiteClusterVersion were copied from the low-code enrichment without checking whether anything sets or reads them. Only the first is set anywhere, by the autopilot-everyone-service chart rather than a job container, and the second is never set at all. Downstream neither has a column in agents_AC_telemetry_standardized, whose macro hardcodes ActivationType to AutomationCloud, so both would have been null on every row. IMAGE_VERSION moves to the platform constants alongside the other environment variable names, which pulls uipath-platform into the change. DurationMs is now opt-in per event rather than inferred from the event suffix, so a future non-terminal event does not silently acquire one.
Studio Web runs go through uipath debug, so without this a large share of coded runs report nothing. Same shape as the run command: start once the factory has classified the run, terminal event on the result, failed event on the way out.
Per-tenant subdomains do not appear in UIPATH_URL, so suffix matching was guarding against a case that cannot occur. Exact matching is stricter, so a lookalike host still falls back to production.
tudormatei1
force-pushed
the
feat/coded-agent-telemetry
branch
from
September 3, 2026 14:14
aeeb797 to
b5fb3b1
Compare
|
radu-mocanu
approved these changes
Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Coded agent and function runs emitted no telemetry once a job key was present, so a run in Orchestrator was invisible in App Insights. Observer hit exactly this when it tried to add coded agents and found nothing to query (AGVSOL-5022).
Three things change:
UIPATH_URL, so an alpha login reports to alpha. An unauthenticated run, an Automation Suite deployment, or any unrecognised host falls back to production.CodedAgentRun.*andCodedFunctionRun.*now fire from the run path, Start plus a terminal event, so a cloud run is no longer silent. Low-code runs are skipped because uipath-agents-python already emitsAgentRun.*for those.Three repo secrets need to exist before this merges, because
APPLICATIONINSIGHTS_CONNECTION_STRINGis no longer read by the build:APPLICATIONINSIGHTS_CONNECTION_STRING_ALPHAagents-alp-appins-ne-appinsAPPLICATIONINSIGHTS_CONNECTION_STRING_STAGINGagents-stg-appins-ne-appinsAPPLICATIONINSIGHTS_CONNECTION_STRING_PRODagents-prd-appins-ne-appinsThey are read by
build-package.ymlandpublish-dev.yml, and passed through fromcd.yml. A missing slot does not fail the build: the placeholder substitutes to an empty string and the wheel ships with that environment's telemetry switched off, so it goes quiet rather than reporting to the wrong instance. Worth adding all three in one go. The old secret can stay, since the integration tests still use it.The new event names also need speccing in the BOI portal before they carry real volume, otherwise they land in the NotApplicable bucket. Failed events carry the error code, title and category but not the message or traceback, which for a coded run hold customer source and values. Say if you would rather match low-code exactly there.
Verified against staging: Start, End and Failed all observed in
agents-stgwith the full scope block, and a dashedUIPATH_TRACE_IDcoming out normalized to hex so it joins the LLMOps trace.