Add AnacondaProject spec for anaconda-project.yml#96
Conversation
Reads the legacy Anaconda Project format used by Anaconda Enterprise / Workbench. Modelled content: - env_specs with inherit_from flattening, top-level packages/channels/platforms propagation, and an implicit ``default`` env spec when none is declared - pip sub-packages split into a separate PIP-stack Environment - anaconda-project-lock.yml read for LOCK-precision Environments, merging the ``all``/``unix``/<platform> package buckets - commands: unix/bash pass through; notebook/bokeh_app get the conventional shell-string translation (``jupyter notebook <path>`` / ``bokeh serve <path>``), with the original kind preserved - variables: list form and dict form (``default`` → value) - CondaEnv, LockFile, and Process artifacts wired to ``anaconda-project prepare|lock|run`` Fields not yet typed in projspec (name/description/icon/categories, downloads, services, top-level platforms, command_kinds/env_specs/http, locking_enabled) are carried verbatim on DescriptiveMetadata.meta so no information is lost.
martindurant
left a comment
martindurant
left a comment
There was a problem hiding this comment.
Please add to the api.rst file in the docs too
| precision=Precision.SPEC, | ||
| ) | ||
| if pip_packages: | ||
| envs[f"{env_name}.pip"] = Environment( |
There was a problem hiding this comment.
This is an extension of the conda env rather than an independent one? Maybe it's OK like this or maybe we cn come up with something better. I was musing in another channel about a "FROM" (or based-on) field for environments anyway.
There was a problem hiding this comment.
Agreed that the current <env>.pip sibling misrepresents the semantics — pip packages in anaconda-project.yml are installed into the conda env, not alongside it. A based-on/FROM field on Environment would be the right long-term fix, and the same problem applies to pixi ([pypi-dependencies]) and uv.
For this PR, I see three options and I'd like your preference:
- Keep the sibling
<env>.pipEnvironment, but clarify in the class docstring that the relationship is layered rather than alternate. Status quo, honest about the limitation. - Fold pip packages into the conda env's
packageslist as tagged strings (e.g.pip::requests>=2.28). Loses discoverability by stack. - Drop the pip emission entirely in v1 and stash pip packages in
DescriptiveMetadata.meta["pip_packages"]. Data preserved, no misleading model, clean migration path when the based-on field lands.
The conda-first / pip-second convention is common enough that (1) is probably fine as an interim. Happy to do whichever you'd like.
There was a problem hiding this comment.
Yes I think I agree - I should create an issue to track this and request feedback.
- Move the format description into the class docstring so it surfaces
in info() and the UIs; keep the module docstring to one line.
- Parse anaconda-project.yml and anaconda-project-lock.yml with
yaml.safe_load directly. The manifest is not a jinja-templated file,
and the mustache-style HTTP option tokens ({{port}}, {{address}}, ...)
round-trip through safe_load as plain strings, which is the correct
behaviour for downstream consumers.
- Register AnacondaProject in docs/source/api.rst.
2 participants
Summary
Adds a new
AnacondaProjectProjectSpec for the legacyanaconda-project.ymlformat (as used by Anaconda Enterprise / Workbench and theanaconda-projectCLI at https://github.com/anaconda/anaconda-project). Complements the existingCondaProject(conda-incubator'sconda-project.yml), which is a different manifest despite the similar name.Motivation
anaconda-projectitself is a legacy tool — it remains in use chiefly because Anaconda Enterprise / Workbench requires it, and that platform is on its way out. The immediate value of projspec being able to read the format is to support the migration of projects away from platforms that require it and onto more actively maintained ecosystems (pixi, uv, etc.).Looking forward, projspec is a natural home for format-conversion tooling: once it can parse every format that matters, converting between them becomes a small layer on top of the existing
content/artifactmodel. This PR only adds the reader; future work could add an emit/convert side. At that point projspec could serve as the essential middleware in a migration pipeline — a single library that understands the source format, normalises the project model, and emits a target format — rather than every migration tool reimplementing the same readers.What's modelled
env_specs.<n>.packages/channels(+ top-level propagation)Environment(stack=CONDA, precision=SPEC)packages: [..., {pip: [...]}]Environment(stack=PIP)at<env>.pipenv_specs.<n>.inherit_from(string or list)anaconda-project-lock.ymlpresentEnvironment(precision=LOCK), mergingall/unix/<platform>bucketscommands.*.unix/bashCommand(cmd=str)+Processartifactcommands.*.notebookCommand(cmd="jupyter notebook <path>")commands.*.bokeh_appCommand(cmd="bokeh serve <path>")commands.*.windows(only)Commandvariables(list or dict withdefault)EnvironmentVariablesdefaultenv whenenv_specs:absentdefaultenv spec materialisedArtifacts (
CondaEnv/LockFile/Process) are wired toanaconda-project prepare|lock|run.What's carried on
DescriptiveMetadata.metaFollowing the pattern used by
dataworkflows.py,infra.py,backstage.py, etc., fields that don't yet have typed representations in projspec are preserved verbatim so nothing is lost:name,description,icon,categoriesdownloads,servicesplatforms,skip_importscommand_kinds— originalnotebook/bokeh_apppaths before shell-string translationcommand_env_specs— per-commandenv_specbindingcommand_supports_http_optionslocking_enabledOut of scope (future upstream work)
default/description/encrypted)Download/Servicecontent typeskindfield or subclass onCommand(notebook, http-service)platformsas a first-classEnvironmentfieldTest plan
tests/test_anaconda_project.pycovering match, basic parse, inheritance, pip split, command kinds (notebook/bokeh_app/unix/string), lock file reading, downloads/services, variable forms — all inline fixtures, matching the style oftests/test_new_specs.pypytest tests/test_anaconda_project.py— 26 passedpytest tests/— same 5 pre-existing "tool not installed" failures as onmain, no new failuresMarked as draft pending any upstream shaping discussion.