Skip to content

RPC: continuous profiling for Python/JS/Go subprocesses#7558

Merged
jkschneider merged 1 commit intomainfrom
worktree-pyroscope-rpc-init
May 4, 2026
Merged

RPC: continuous profiling for Python/JS/Go subprocesses#7558
jkschneider merged 1 commit intomainfrom
worktree-pyroscope-rpc-init

Conversation

@jkschneider
Copy link
Copy Markdown
Member

Summary

Each language's RPC server now starts the Pyroscope SDK when PYROSCOPE_SERVER_ADDRESS is set in the environment, with PYROSCOPE_TAGS forwarded verbatim and a runtime=python|node|go tag added so flame graphs in a shared modcli Pyroscope application can be sliced by which subprocess produced them.

The env-var propagation chain — recipe-worker JVM → modw → mod CLI JVM → ProcessBuilder for the RPC subprocess — works without code changes here because RewriteRpcProcess.run() calls pb.environment().putAll(extras) rather than clearing inherited env. So once the saas-side CliProcessEnvContributor (which today wires the JVM Pyroscope agent on the mod CLI) is extended to also set PYROSCOPE_SERVER_ADDRESS / PYROSCOPE_TAGS / PYROSCOPE_APPLICATION_NAME, all four runtimes profile under the same modcli app, separated by the runtime tag.

Per-language notes

  • Python (rewrite-python/rewrite/src/rewrite/rpc/server.py): _init_pyroscope() called from main() after argparse. pyroscope-io is a new optional dep ([profiling] extra) so local dev and CI without the package don't break — the function warns and no-ops on ImportError.
  • JS (rewrite-javascript/rewrite/src/rpc/server.ts): initPyroscope(logger) called from main() right after the logger is constructed. @pyroscope/nodejs is a new optionalDependencies entry so npm install doesn't fail when the binding's native components aren't available; initPyroscope catches the require and warns.
  • Go (rewrite-go/cmd/rpc/main.go): initPyroscope() is the first line of main(). github.com/grafana/pyroscope-go v1.2.8 is a regular dep — Go has no optional-dep concept and the SDK is small and pure-Go (uses stdlib pprof under the hood). The function short-circuits on missing env so non-profiled deployments pay one os.Getenv at startup.
  • C#: no change needed in rewrite-csharp. CoreCLR loads the Pyroscope native profiler purely from CORECLR_ENABLE_PROFILING / CORECLR_PROFILER / CORECLR_PROFILER_PATH, which the saas-side contributor will set alongside PYROSCOPE_*.

Test plan

  • python -c "import ast; ast.parse(open('.../server.py').read())" — parses
  • tomli parse of pyproject.toml — parses; [profiling] extra resolves
  • npx tsc --noEmit -p tsconfig.test.json (rewrite-javascript) — clean
  • go build ./cmd/rpc (rewrite-go) — clean
  • go vet ./cmd/rpc (rewrite-go) — clean
  • End-to-end on dev recipe-worker: with profiling capability + flag enabled, trigger a Python recipe run and confirm application=modcli flame graphs in Grafana split by runtime tag (python, node, go, jvm). Requires the saas-side CliProcessEnvContributor extension to land first.
  • Confirm [profiling] extra installs cleanly in mod-cli-base's Python venv.
  • Confirm optionalDependencies doesn't fail npm install on a clean container without the native binding present.

Each language's RPC server now starts the Pyroscope SDK when
PYROSCOPE_SERVER_ADDRESS is set in the environment, with PYROSCOPE_TAGS
forwarded verbatim and a runtime=python/node/go tag added so flame
graphs in a shared `modcli` Pyroscope application can be sliced by
which subprocess produced them. Env vars propagate from the parent JVM
via ProcessBuilder.environment() inheritance, so the saas-side
CliProcessEnvContributor that already drives the JVM agent transparently
reaches all four runtimes once these inits land.

Python: pyroscope-io is a new optional dep ([profiling] extra) so local
dev and CI without the package don't break — _init_pyroscope() warns
and no-ops on ImportError.

JS: @pyroscope/nodejs is a new optionalDependency so npm install
doesn't fail when the binding's native components aren't available;
initPyroscope() catches the require and warns.

Go: github.com/grafana/pyroscope-go is a regular dep (Go has no optional
deps; the SDK is small and pure-Go via stdlib pprof). initPyroscope()
short-circuits on missing env so non-profiled deployments pay only one
os.Getenv at startup.

C# requires no rewrite-side change: CoreCLR loads the Pyroscope native
profiler purely from CORECLR_ENABLE_PROFILING / CORECLR_PROFILER /
CORECLR_PROFILER_PATH env vars, which the same saas-side contributor
sets alongside PYROSCOPE_*.
@github-project-automation github-project-automation Bot moved this to In Progress in OpenRewrite May 4, 2026
@jkschneider jkschneider merged commit 5da0a22 into main May 4, 2026
1 check failed
@jkschneider jkschneider deleted the worktree-pyroscope-rpc-init branch May 4, 2026 02:37
@github-project-automation github-project-automation Bot moved this from In Progress to Done in OpenRewrite May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant