After doing pixi shell, the manifest path is still pointing at the default of pixi.toml

[kszlim]

Problem description

Instead, I'd expect that doing a pixi shell --manifest-path some/path/file.toml would make your manifest path always some/path/file.toml. This would make multiple manifest files much more usable (currently using this as a workaround for the lack of optional dependencies for cuda vs CPU only situations).

I think regardless of whether multiple manifest files makes sense, I think the behavior should always be the default (at least I can't think of any reason not for this to be)?

[baszalmstra]

I didn't understand the issue but now I understand that you want to be able to use a different filename for the pixi.toml if explicitly specified.

I'm also not comfortable adding this as a supported use case for pixi. I think this is confusing behavior because it makes it less clear to a user who is browsing the files of your package (and potential future IDE integrations). I think the name pixi.toml implies what the context of the file is (its a pixi project manifest). Would some/path/pixi.toml not work for your use-case? Having multiple pixi projects in subdirectories (and in the future: workspaces) definitely is but then I think the individual project file should still just be called pixi.toml.

I think instead we should modify our logic so that if the argument to --manifest-path doesn't point to a pixi.toml file it is considered incorrect.

[kszlim]

That's a little orthogonal to my issue, I think once you've pixi shell with an explicit manifest file, within the shell you should continue implicitly using it.

[baszalmstra]

Mm maybe I misunderstand then. Do you maybe have a reproducible case which shows what is happening now and what you would expect?

[kszlim]

If you have two toml files:

pixi.toml
pixi-cuda.toml

The first has a task called pixi run train-cpu and the latter toml has a task pixi run train-gpu

Let's say you do this from the root directory of the package (which has the two toml files)

pixi shell --manifest-path pixi-cuda.toml it will put you in the (pixi-cuda) conda env. However if you invoke pixi run train-cuda after that point, that task won't exist as despite being in the (pixi-cuda) conda env, it still will search the directory for a pixi.toml file instead of using the pixi-cuda.toml that the environment shell was sourced from. Also, the environment variable is pointing towards the pixi.toml in a hard coded manner instead of always reflecting it's correct location.

[baszalmstra]

This was discussed in our discord server.

I think it is confusing that with the suggestions above pixi run starts works differently with regards to which project it loads based on whether or not you are in an active shell. For instance, take this example:

Given this folder structure.

- pixi.toml
- subproject/
  - pixi.toml

$ pixi shell # activate the shell from the project in the root
$ cd subproject
$ pixi run task

Is task taken from the project at the root or in the subproject?

I prefer to always use the same way of doing things (so task from subproject will be used in the sample above).

I added a PR that requires that a --manifest-path always points to a pixi.toml. Since we use the root of the project to store information about the environment this would cause conflicts if you can have multiple project files in the same directory. I also think the name of the file has semantic meaning. It indicates to a user and (future) automation that this file is a pixi project file.

I understand this is far from ideal given that we don't support multiple environments yet and this makes your workaround harder. But I don't want to introduce an API to enable a workaround that people will start relying on.

[baszalmstra]

With #324 you can no longer have multiple pixi projects in the same folder which fixes part of this issue.

The other issue, where a manifest should be preferred that is used in the shell instead of looking at the files at disk, we will not implement.

Im going to close this issue for now. I know we actually made the situation more complicated for this particular use case rather than simpler but it is not our vision with pixi to enable globally activated environments.

The original issue where you want to have multiple dependencies sets can still be achieved by using subdirectories with pixi.tomls inside until we figure out a solution to do this from a single pixi.toml file.

[kszlim]

The original issue where you want to have multiple dependencies sets can still be achieved by using subdirectories with pixi.tomls inside until we figure out a solution to do this from a single pixi.toml file.

Do you know if there's a non-verbose way of doing this? Having tasks associated with certain sets of dependencies seems very painful to use, ie. you have to invoke pixi from the subdirectory instead of globally (within the project).