Skip to content

Add Poetry package manager support for Python#13086

Open
Copilot wants to merge 9 commits intomainfrom
copilot/add-poetry-support
Open

Add Poetry package manager support for Python#13086
Copilot wants to merge 9 commits intomainfrom
copilot/add-poetry-support

Conversation

Copy link
Contributor

Copilot AI commented Nov 21, 2025

Description

Adds Poetry as a Python package manager option in .NET Aspire, following existing patterns for pip and uv. This implementation allows developers to use Poetry for managing Python dependencies and virtual environments in Aspire applications.

Important: Aspire does not override Poetry's default configuration for local development. Users must explicitly configure Poetry's virtual environment behavior via the env parameter if they want to control where Poetry creates virtual environments (e.g., in-project vs cache directory). However, when building container images, Aspire sets POETRY_VIRTUALENVS_IN_PROJECT=true to ensure Poetry creates the virtual environment within the container.

Fixes #6742

Changes Made

New Files

  • PoetryInstallationManager.cs - Validates Poetry installation
  • PoetryEnvironmentAnnotation.cs - Stores custom Poetry environment variables using IReadOnlyDictionary<string, string>

Modified Files

  • PythonAppResourceBuilderExtensions.cs - Added WithPoetry extension method, Poetry-specific Dockerfile generation, and registered all installation managers upfront in AddPythonAppCore
  • AddPythonAppTests.cs - Added 8 comprehensive unit tests for Poetry support
  • Aspire.Hosting.Python.cs - Updated public API surface

Key Implementation Details

API Signature:

public static IResourceBuilder<T> WithPoetry<T>(
    this IResourceBuilder<T> builder,
    bool install = true,
    string[]? installArgs = null,
    (string key, string value)[]? env = null)
    where T : PythonAppResource

Virtual Environment Management:

  • Local Development: Aspire does not set POETRY_VIRTUALENVS_IN_PROJECT or POETRY_VIRTUALENVS_PATH by default. Poetry uses its own default configuration (typically creates venvs in a cache directory). Users can customize Poetry's behavior via the env parameter.
  • Container Images: Aspire sets POETRY_VIRTUALENVS_IN_PROJECT=true in the Dockerfile to ensure Poetry creates the virtual environment within the container at /app/.venv. This removes reliance on copying the venv from the host.

Installation:

  • Base command: poetry install --no-interaction
  • Custom arguments can be appended via installArgs parameter
  • Can be disabled with install: false

Dockerfile Generation:

  • Multi-stage build pattern (builder + runtime stages)
  • Sets POETRY_VIRTUALENVS_IN_PROJECT=true to create venv in container
  • Supports poetry.lock for reproducible builds
  • Optimized layer caching
  • Non-root user execution following security best practices
  • Poetry installs dependencies within the container (no venv copying from host)

Service Collection Registration:

  • All installation managers (PythonInstallationManager, UvInstallationManager, PoetryInstallationManager) are now registered upfront in AddPythonAppCore
  • This allows WithPoetry(), WithUv(), and WithPip() to be called safely during BeforeStartEvent for global configuration
  • Prevents crashes when service collection is frozen

Usage Examples

// Basic usage (Poetry uses its default virtual environment location for local dev)
builder.AddPythonApp("api", "../api", "main.py").WithPoetry();

// With in-project virtual environment (recommended for local development)
builder.AddPythonApp("api", "../api", "main.py")
    .WithPoetry(env: [("POETRY_VIRTUALENVS_IN_PROJECT", "true")]);

// With custom install arguments
.WithPoetry(installArgs: ["--no-root", "--sync"])

// With custom virtual environment path
.WithPoetry(env: [("POETRY_VIRTUALENVS_PATH", "/custom/venvs")])

// Skip installation (expects existing venv)
.WithPoetry(install: false)

// Global configuration via BeforeStartEvent (now works correctly)
builder.Eventing.Subscribe<BeforeStartEvent>((e, ct) =>
{
    foreach (var resource in builder.Resources.OfType<PythonAppResource>())
    {
        builder.CreateResourceBuilder(resource)
            .WithPoetry(env: [("POETRY_VIRTUALENVS_IN_PROJECT", "true")]);
    }
    return Task.CompletedTask;
});

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No
  • Does the change require an update in our Aspire docs?
    • Yes
      • Link to aspire-docs issue: TBD (documentation should cover the new WithPoetry extension method, different behavior between local dev and container builds, global configuration pattern, and usage examples)
    • No

Testing Results

  • ✅ All 101 tests passing (including 8 new Poetry-specific tests)
  • ✅ Build successful - 0 warnings, 0 errors
  • ✅ CodeQL security analysis - No vulnerabilities detected
  • ✅ Code review feedback addressed
  • ✅ Global configuration via BeforeStartEvent now works correctly
Original prompt

This section details on the original issue you should resolve

<issue_title>Add support for poetry</issue_title>
<issue_description>Poetry is interesting because it is more like uv than pip, but also has some different defaults around the vdir</issue_description>

<agent_instructions># Aspire: Poetry Package Manager Support (Refined API Spec)

Scope

Add support for Poetry as a Python package manager in Aspire, following existing Pip/Uv patterns.
Poetry:

  • Owns virtual environment creation.
  • Aspire influences where the venv is created via environment variables.
  • No auto-detection; Poetry must be explicitly selected.
  • API favors a single, concise opt-in method with parameters controlling install behavior.

1. Public API

1.1 Enum

public enum PythonPackageManager
{
    Pip,
    Uv,
    Poetry, // NEW
}

1.2 Builder Extension

Single explicit selector; no AddPoetryApp helper.

public static IResourceBuilder<PythonAppResource> WithPoetry(
    this IResourceBuilder<PythonAppResource> builder,
    bool install = true,
    string[]? installArgs = null,
    (string key, string value)[]? env = null)

Parameters

Name Type Default Meaning
install bool true If true, Aspire will invoke poetry install during its restore/build phase. If false, Aspire skips installing and only resolves the venv; failure occurs if no venv exists.
installArgs string[]? null Additional arguments appended to the poetry install command (after Aspire’s default flags). Examples: ["--no-root"], ["--sync"].
env (string key, string value)[]? null Extra environment variables applied to the Poetry restore step (and optionally cached for container generation). These can override defaults like POETRY_VIRTUALENVS_IN_PROJECT.

Behavior

  • Calling WithPoetry sets PackageManager = PythonPackageManager.Poetry.
  • Parameters are stored in resource metadata for later use in restore/container build phases.
  • No new PoetryOptions class; the method’s parameters replace earlier options concepts.

2. Virtual Environment Strategy

2.1 Default Venv Location

  • If WithVirtualEnvironment(path) is NOT specified:
    • Aspire sets POETRY_VIRTUALENVS_IN_PROJECT=true unless the user overrides it via the env parameter.
    • Expect venv at <projectPath>/.venv after poetry install.

2.2 Explicit Virtual Environment Path

  • If WithVirtualEnvironment(customPath) is used:
    • Aspire sets POETRY_VIRTUALENVS_PATH=<customPath>.
    • Does not set POETRY_VIRTUALENVS_IN_PROJECT unless customPath == <projectPath>/.venv.
    • After install (or directly if install=false), Aspire runs poetry env info --path to resolve the canonical venv directory (Poetry may hash subdirectories).

2.3 Resolution Algorithm (Conceptual)

  1. If an explicit virtual environment path is configured:
    • Invoke poetry env info --path; use its result if valid.
    • Error if missing.
  2. Else:
    • Check for <projectPath>/.venv; use if exists.
    • If missing and install=true, re-run resolution after install.
    • If still missing, run poetry env info --path:
      • Use returned path if valid.
      • Error otherwise.

2.4 No Aspire-driven venv creation

  • Aspire does NOT call python -m venv for Poetry projects.
  • Poetry is the authority for creation and lifecycle.

3. Restore / Build Phase

3.1 Command Construction

Base command:

poetry install --no-interaction

Rules:

  • Always include --no-interaction.
  • Append user-supplied installArgs in order.
  • Skip entire invocation if install=false.

3.2 Environment Variables

Aspire-provided defaults (unless overridden in env):

  • In-project (no explicit venv path): POETRY_VIRTUALENVS_IN_PROJECT=true
  • Explicit path: POETRY_VIRTUALENVS_PATH=<customPath>

User-specified env entries:

  • Merged after defaults; if keys collide, user values win.
  • Empty or null env means defaults only.

Example merge order (conceptual):

  1. Internal defaults based on WithVirtualEnvironment.
  2. User-specified env overrides.
  3. System/path variables from existing pipeline (PATH, etc.).

3.3 Container Build

  • Emit the same environment variable(s) (POETRY_VIRTUALENVS_IN_PROJECT or POETRY_VIRTUALENVS_PATH).
  • Run identical poetry install command unless install=false.
  • Capture resulting venv path for runtime specification.

4. Execution Phase

  • After resolution, derive Python executable:
    • POSIX: <venvPath>/bin/python
    • Windows: <venvPath>/Scripts/python.exe
  • Run the application directly via that Python.
  • Never use poetry run.

5. Error Handling

Situation Action
Poetry binary not on PATH Fail: "Poetry was selected but poetry is not on P...

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 3 commits November 21, 2025 07:10
Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for Poetry package manager Add Poetry package manager support for Python Nov 21, 2025
Copilot AI requested a review from davidfowl November 21, 2025 07:28
Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
@github-actions
Copy link
Contributor

github-actions bot commented Nov 21, 2025

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 13086

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 13086"

Comment on lines +687 to +688
.Copy("pyproject.toml", "/app/")
.Copy("poetry.lock", "/app/")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only works if I didn't split my app into multiple packages :(
But, I don't expect you to solve everything :D

@tjwald
Copy link

tjwald commented Nov 22, 2025

Tested locally. Multiple issues:

  • Will override existing poetry.toml files that configure poetry behavior.
  • When trying to configure globally all pythonapps to use poetry, using the BeforeStartEvent will crash aspire apphost since WithPoetry() is trying to mutate the ServiceCollection which is already frozen.

@davidfowl
Copy link
Contributor

When trying to configure globally all pythonapps to use poetry, using the BeforeStartEvent will crash aspire apphost since WithPoetry() is trying to mutate the ServiceCollection which is already frozen.

This isn't unique to poetry. I assume this applies to everything in the python (and more), extension methods.

@davidfowl
Copy link
Contributor

@copilot Implement the PR feedback

…ction

Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 22, 2025

@copilot Implement the PR feedback

Added IsReadOnly check before registering all installation managers (PythonInstallationManager, UvInstallationManager, PoetryInstallationManager) in commit 0e6d100. This prevents crashes when the service collection is frozen, such as when using BeforeStartEvent for global configuration.

@davidfowl
Copy link
Contributor

@copilot We don't want to override POETRY_VIRTUALENVS_PATH or POETRY_VIRTUALENVS_IN_PROJECT by default. Don't set them

… default

Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 22, 2025

@copilot We don't want to override POETRY_VIRTUALENVS_PATH or POETRY_VIRTUALENVS_IN_PROJECT by default. Don't set them

Removed default environment variable settings in commit f981883. Poetry will now use its own defaults. Users can configure Poetry behavior (e.g., POETRY_VIRTUALENVS_IN_PROJECT=true for in-project venvs) via the env parameter.

.Run("groupadd --system --gid 999 appuser && useradd --system --gid 999 --uid 999 --create-home appuser")
.EmptyLine()
.Comment("Copy the application and virtual environment from builder")
.CopyFrom(builderStage.StageName!, "/app", "/app", "appuser:appuser")
Copy link

@tjwald tjwald Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would not contain the .venv if not using the POETRY_VIRTUALENVS_IN_PROJECT=true.
I think that in the docker file turning this flag on is preferred, but again, but other users might not like that.

@davidfowl
Copy link
Contributor

@copilot When using poetry and building a container image, assume the venv is not local and run poetry install to create the venv in the container. This will remove a reliance on the venv being on disk and being copied into the container.

Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 23, 2025

@copilot When using poetry and building a container image, assume the venv is not local and run poetry install to create the venv in the container. This will remove a reliance on the venv being on dis...

Set POETRY_VIRTUALENVS_IN_PROJECT=true in the container builder stage in commit fd7a110. Poetry now creates the venv within the container at /app/.venv during poetry install, removing the need to copy the venv from the host disk.

@davidfowl
Copy link
Contributor

@copilot This pattern is broken:

// Global configuration via BeforeStartEvent
builder.Eventing.Subscribe((e, ct) =>
{
foreach (var resource in builder.Resources.OfType())
{
builder.CreateResourceBuilder(resource)
.WithPoetry(env: [("POETRY_VIRTUALENVS_IN_PROJECT", "true")]);
}
return Task.CompletedTask;
});

We need to add the installtion managers up front in AddPythonAppCore

Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 23, 2025

@copilot This pattern is broken:

// Global configuration via BeforeStartEvent...

Fixed in commit e648bc6. All installation managers (PythonInstallationManager, UvInstallationManager, PoetryInstallationManager) are now registered upfront in AddPythonAppCore, allowing WithPoetry() and WithUv() to be called safely during BeforeStartEvent for global configuration.

@davidfowl davidfowl marked this pull request as ready for review November 24, 2025 10:10
Copilot AI review requested due to automatic review settings November 24, 2025 10:10
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Poetry as a third Python package manager option in .NET Aspire, joining pip and uv. Poetry is a popular Python dependency management tool that handles its own virtual environment creation. The implementation follows established patterns from pip/uv while accommodating Poetry's distinct behavior, particularly around virtual environment management.

Key Changes:

  • Adds WithPoetry() extension method with parameters for install control, custom arguments, and environment variable configuration
  • Implements Poetry-specific Dockerfile generation with multi-stage builds and optimized layer caching
  • Registers all installation managers upfront to prevent service collection freezing issues during global configuration

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
PoetryInstallationManager.cs New installation validator for Poetry CLI tool availability
PoetryEnvironmentAnnotation.cs New annotation to store Poetry-specific environment variables
PythonAppResourceBuilderExtensions.cs Added WithPoetry method, Poetry Dockerfile generation, and upfront service registration
AddPythonAppTests.cs Added 8 comprehensive unit tests covering Poetry functionality
Aspire.Hosting.Python.cs Updated public API surface with WithPoetry signature

/// <returns>A reference to the <see cref="IResourceBuilder{T}"/> for method chaining.</returns>
/// <remarks>
/// <para>
/// This method creates a child resource that runs <c>poetry install --no-interaction</c> in the working directory of the Python application.
Copy link

Copilot AI Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation states this method 'creates a child resource' but should clarify that this only happens when install is true. When install is false, no installer resource is created. Consider rephrasing to: 'When <paramref name="install"/> is true (default), this method creates a child resource that runs...' to match the actual behavior documented in the parameter description.

Suggested change
/// This method creates a child resource that runs <c>poetry install --no-interaction</c> in the working directory of the Python application.
/// When <paramref name="install"/> is true (default), this method creates a child resource that runs <c>poetry install --no-interaction</c> in the working directory of the Python application.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for poetry

4 participants