Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,48 @@ print(result.resolved) # [("app", "1.0.0", None), ("lib", "2.0.0", None)]
`solve()` reports failures and bad input via `result.status`
(`"solved"` / `"failed"` / `"error"`), never as a Python exception.

### Wiring `pyrer` behind `rez`

`pyrer` only does the solve; `rez` still does package discovery, env
construction, and the whole `ResolvedContext` lifecycle. To plug
`pyrer` in behind a normal `rez env` / `ResolvedContext` flow:

1. Walk rez's package paths into the `pyrer` repo shape — once per
process, reusable across many solves:

```python
from rez.packages import iter_package_families

def build_pyrer_repo(package_paths):
repo = {}
for fam in iter_package_families(paths=package_paths):
repo[fam.name] = {
str(pkg.version): {
"requires": [str(r) for r in (pkg.requires or [])],
"variants": [
[str(r) for r in v] for v in (pkg.variants or [])
],
}
for pkg in fam.iter_packages()
}
return repo
```

2. Call `pyrer.solve(requests, json.dumps(repo))` instead of
`rez.solver.Solver.solve()`.

3. Translate `result.resolved` back into rez `Variant` objects with
`rez.packages.get_package(name, version).get_variant(idx)` and feed
them into whatever you'd normally hand to `ResolvedContext`.

The
[Wiring `pyrer` into `rez`](https://doubleailes.github.io/rer/docs/getting-started/rez-integration/)
guide has the full walkthrough — a minimal monkey-patch of
`Resolver._solve`, a fallback for configs `pyrer` doesn't model yet
(`intersection_priority`, `@early` / `@late` requires, custom
orderers / filters), and a sanity-check loop for diffing `pyrer`
against rez on your own repo.

## Building & testing

```bash
Expand Down
3 changes: 3 additions & 0 deletions docs/content/docs/getting-started/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,9 @@ builds the environment, and manages contexts. rer just does the solving.

- **[Quick Start →](../quick-start/)** — install rer and run your first
resolve from Python or Rust.
- **[Wiring `pyrer` into `rez` →](../rez-integration/)** — how to plug
`pyrer` in behind a normal `rez env` / `ResolvedContext` flow, with a
minimal shim and the caveats.
- **[Engineering notes →](../../engineering/)** — design decisions and
measurements behind the port (e.g. why some rez optimisations are
intentionally absent).
Expand Down
262 changes: 262 additions & 0 deletions docs/content/docs/getting-started/rez-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,262 @@
+++
title = "Wiring `pyrer` into `rez`"
description = "How to use pyrer as the solver backend behind a normal rez workflow."
date = 2026-05-15T08:30:00+00:00
updated = 2026-05-15T08:30:00+00:00
draft = false
weight = 30
sort_by = "weight"
template = "docs/page.html"

[extra]
lead = "How to plug pyrer into a normal rez workflow: rez still handles package discovery and environment construction; pyrer just does the solve."
toc = true
top = false
+++

## What `pyrer` is, and what it is not

`pyrer` is **only the solver hotpath** — the rez-faithful phase-based
backtracking algorithm, ported to Rust and called from Python through
PyO3. It is *not* a replacement for `rez`. It does not:

- discover packages on the filesystem,
- parse `package.py` (it takes pre-parsed requirements as strings),
- build the runtime environment (PATH, env vars, shell hooks),
- handle the `rxt` context lifecycle, suites, or context bundling.

`rez` keeps doing all of that. `pyrer` is dropped in at the one step
where the cost lives: solving the version constraints.

The minimum integration looks like:

```
┌────────────────────────────────────────────┐
│ rez: iter_package_families / iter_packages │ ← package discovery
└────────────────────────────────────────────┘
┌────────────────────────────────────────────┐
│ build a pyrer repo dict (name → version │
│ → {requires, variants}) │ ← one-time conversion
└────────────────────────────────────────────┘
┌────────────────────────────────────────────┐
│ pyrer.solve(requests, json.dumps(repo)) │ ← the fast bit
└────────────────────────────────────────────┘
┌────────────────────────────────────────────┐
│ resolve → rez Variant objects → │
│ ResolvedContext / env build │ ← rez again
└────────────────────────────────────────────┘
```

## Building the `pyrer` repo from `rez`

`pyrer.solve()` expects an in-memory repository in the schema
`{name: {version: {"requires": [...], "variants": [[...]]}}}` — the
same shape rer's tests use.

A minimal converter from a rez package path looks like this:

```python
from rez.packages import iter_package_families


def build_pyrer_repo(package_paths):
"""Walk rez's package paths and produce the JSON-shaped repo pyrer wants."""
repo = {}
for family in iter_package_families(paths=package_paths):
versions = {}
for pkg in family.iter_packages():
versions[str(pkg.version)] = {
"requires": [str(r) for r in (pkg.requires or [])],
"variants": [
[str(r) for r in variant]
for variant in (pkg.variants or [])
],
}
repo[family.name] = versions
return repo
```

Two notes on this step:

- It is **eager** — every package on every path is loaded. `rez`
normally loads lazily; the trade-off is one upfront cost vs many
small ones during the solve. On a real repo, eager loading is
typically a few seconds; on the rez 188-case benchmark it is the
dominant pre-solve cost.
- If you're running many resolves against the same repo (CI, batch
validation, a long-lived daemon), build the dict **once** and reuse
it. Better still, share a `pyrer.make_shared_cache()` between
`Solver::new_with_cache(...)` calls so even rer's internal variant
cache lives across solves. See the
[`SharedVariantCache`](https://github.com/doubleailes/rer/blob/main/crates/rer-resolver/src/rez_solver/context.rs)
type.

## Solving

```python
import json
import pyrer

repo = build_pyrer_repo(["/sw/pkg", "/sw/site"])
repo_json = json.dumps(repo)

result = pyrer.solve(["maya-2024", "nuke-14"], repo_json)

print(result.status) # "solved" | "failed" | "error"
print(result.solve_time_ms) # wall-clock of just the Rust solve
for name, version, variant_idx in result.resolved:
print(name, version, variant_idx)
```

`status` distinguishes:

- **`"solved"`** — `result.resolved` is the resolution as a list of
`(name, version, variant_index)` tuples. `variant_index` is `None`
for packages with no `variants` defined.
- **`"failed"`** — a real resolve conflict; `result.failure_description`
has a human-readable reason.
- **`"error"`** — bad input (malformed repo JSON, unparseable
requirement string, missing top-level package).

No Python exception is ever raised from `pyrer.solve()` — even a
panic inside the Rust solver is caught and reported as `"error"`.

## Translating the result back to `rez`

To hand the resolution back to rez (for environment construction,
context bundling, etc.), look the resolved tuples up against rez's own
package iterator:

```python
from rez.packages import get_package, get_variant


def resolve_to_rez_variants(result, package_paths):
"""Turn pyrer's (name, version, variant_index) tuples into rez Variants."""
variants = []
for name, version, idx in result.resolved:
pkg = get_package(name, version, paths=package_paths)
if pkg is None:
raise RuntimeError(f"package vanished after solve: {name}-{version}")
# idx is None for packages with no variants — rez models that
# as a single variant with index 0 internally.
variants.append(pkg.get_variant(idx if idx is not None else 0))
return variants
```

These `Variant` objects can be fed into rez's normal context machinery
(see `rez.resolved_context.ResolvedContext` — you'll want to look at
how its internal solver result is normally consumed and substitute the
list above). For most workflows the most useful thing is to call
`rez.rex.bind` / `Variant.apply_value` on each variant against an
`ActionInterpreter`, which is the same code rez runs after its own
solve.

## A complete monkey-patch shim

If you want pyrer to transparently accelerate `rez env` /
`ResolvedContext` without changing call sites, the smallest sound
patch is to replace `rez.solver.Solver.solve` with a delegating
implementation. This is non-trivial to get right (rez's `Solver`
exposes a rich status surface — `phase_stack`, `failure_reason`,
graph rendering, callback support) so the patch is best kept narrow:
intercept the happy path, fall back to the real rez solver on any
non-default config (custom orderer, `late` binding requires,
`@early` evaluation, etc.).

The minimum viable shim — for studios with default-configured repos —
looks roughly like:

```python
import json
import pyrer
import rez.solver as _rez_solver
import rez.resolver as _rez_resolver

_original_resolve = _rez_resolver.Resolver._solve


def _pyrer_resolve(self):
# Fall back to rez on anything pyrer doesn't support yet.
if self.package_filter or self.package_orderers:
return _original_resolve(self)

repo = build_pyrer_repo(self.package_paths)
requests = [str(r) for r in self.package_requests]
result = pyrer.solve(requests, json.dumps(repo))

if result.status != "solved":
return _original_resolve(self) # let rez produce the canonical failure

self.resolved_packages_ = resolve_to_rez_variants(
result, self.package_paths,
)
self.status_ = _rez_solver.SolverStatus.solved
return self


_rez_resolver.Resolver._solve = _pyrer_resolve
```

Load this once at process start (e.g. via a `rezconfig.py`'s
`plugin_path` entry or a `sitecustomize.py`) and any `rez env`,
`rez build`, `rez-bundle` etc. running in that process will route
through `pyrer` for the solve.

## Caveats and what isn't supported yet

`pyrer.solve` is the solver only. The following are **not** modelled
by it — if your studio depends on any of these, fall back to rez's
solver for those resolves:

- **`VariantSelectMode::intersection_priority`.** `pyrer` implements
rez's default `version_priority` only; see
[issue #63](https://github.com/doubleailes/rer/issues/63).
- **`@early` / `@late` binding requires.** `pyrer` takes already-
parsed strings; if a package's requires depend on the resolve
context, rez has to evaluate them first.
- **Custom package orderers and filters.** Anything that hooks into
`PackageOrder` / `PackageFilter` runs in rez; the integration shim
above falls back when these are configured.
- **Cyclic-failure detail.** Both solvers detect cycles; the human-
readable failure message differs in wording.
- **Variant-index parity.** The differential test currently checks
the resolved `(name, version)` set, not the variant index — variant
selection is rez-faithful by construction but is not enforced by
the test suite. See the [README's *Validated 1:1* note](../../../../#status).

## Sanity-checking against rez

To make sure `pyrer` agrees with `rez`'s solver on your own repo,
generate a small set of representative requests and diff the
resolutions:

```python
from rez.resolved_context import ResolvedContext

for request in your_real_requests:
rer_result = pyrer.solve(request, repo_json)
rez_ctx = ResolvedContext(request, package_paths=["/sw/pkg"])
rer_set = {(n, v) for n, v, _ in rer_result.resolved}
rez_set = {(v.name, str(v.version)) for v in rez_ctx.resolved_packages}
assert rer_set == rez_set, f"diverge on {request}"
```

If any case diverges, open an issue with the request and a minimal
package set that reproduces — the project's correctness bar is "match
rez 1:1" and divergence is a release blocker.

## See also

- [Quick Start →](../quick-start/) — the basic `pyrer.solve` API in
isolation, without any rez integration.
- [Engineering notes →](../../engineering/) — design decisions behind
the port (e.g. why some rez optimisations are intentionally absent).
- [rez integration in the repo README](https://github.com/doubleailes/rer/blob/main/README.md)
— short reference card.
Loading