[Bug]: jupyEnv doesn't respect python version from pyproject.toml

[dmayle]

Current Behavior

When I create a kernel from a pyprojet.toml, the python version is ignore, and 3.10 is used as default

Expected Behavior

I expect the kernel to respect pyproject.toml, or alternatively, document that it's not using that part of pyproject.toml

Steps To Reproduce

Config from kernels.nix:

kernel.python.p39 = {
  enable = true;
  projectDir = ./p39;
};

Config from pyproject.toml:

[tool.poetry]
name = "p39"
version = "0.1.0"
description = ""
authors = []
readme = "README.md"

[tool.poetry.dependencies]
python = ">=3.9,<3.10"
ipykernel = "^6.22.0"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

OS

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.24, NixOS, 23.05 (Stoat), 23.05.20230417.0a4206a`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - nixpkgs: `/etc/nixpath/nixpkgs`

### Version

Latest (HEAD as of this bug report)

### Additional Context

_No response_

### Relevant log output

_No response_

[dmayle]

Flake.nix:

{
  description = "Your jupyenv project";

  nixConfig.extra-trusted-substituters = [
    "https://tweag-jupyter.cachix.org"
  ];
  nixConfig.extra-trusted-public-keys = [
    "tweag-jupyter.cachix.org-1:UtNH4Zs6hVUFpFBTLaA4ejYavPo5EFFqgd7G7FxGW9g="
  ];

  inputs.flake-compat.url = "github:edolstra/flake-compat";
  inputs.flake-compat.flake = false;
  inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  inputs.jupyenv.url = "github:tweag/jupyenv";

  outputs = {
    self,
    flake-compat,
    flake-utils,
    nixpkgs,
    jupyenv,
    ...
  } @ inputs:
    flake-utils.lib.eachSystem
    [
      flake-utils.lib.system.x86_64-linux
    ]
    (
      system: let
        inherit (jupyenv.lib.${system}) mkJupyterlabNew;
        jupyterlab = mkJupyterlabNew ({...}: {
          nixpkgs = inputs.nixpkgs;
          imports = [(import ./kernels.nix)];
        });
      in rec {
        packages = {inherit jupyterlab;};
        packages.default = jupyterlab;
        apps.default.program = "${jupyterlab}/bin/jupyter-lab";
        apps.default.type = "app";
      }
    );
}

[dmayle]

flake.lock in a gist:
https://gist.github.com/dmayle/94220d3e42019dd17e51f17cc004b8fc

[dmayle]

This may be user error... I didn't realize that "^3.9" was equivalent to ">=3.9,<4.0". I'm going to test with ">=3.9,<3.10" to see if I get the correct python version

[dmayle]

I have updated my pyproject.toml to use >=3.9,<3.10 and this problem is still an issue

[jaapterwoerds]

I have struggled with this as well, I think the pinned python version is actually something which is managed outside poetry. The python version bounds are merely there to guide the dependency resolution, possible for multiple supported versions of python. The way I would expect this to work is that in the kernels.nix we actually pin the python version so that dependency resolution is done for that version.

What i noticed from working with Python developers professionally is that the workflow on how to use poetry is not very well understood. How poetry2nix / jupyenv interacts with this is not documented from a user perspective from what I could find. I think the documentation would benefit from defining how to use jupyenv in combination with poetry2nix so you can use only the nix toolchain to start a project, generate a pyproject.toml, generate a lock file and later in the life cycle add / remove / update / dependencies purely with nix and nix develop.