Support workspace only project #309

ischaojie · 2023-06-11T15:00:12Z

[WIP] #236

For projects that do not need to be published as a package, add a "project-only" field for identification.

[tool.rye]
managed = true
project-only = true

Welcome everyone to express their own opinions.

bluss · 2023-06-11T15:37:29Z

rye/src/pyproject.rs

+        .and_then(|x| x.get("rye"))
+        .and_then(|x| x.get("workspace-only"))
+        .and_then(|x| x.as_bool())
+        .unwrap_or(false)


would this config variable live better inside the existing group called tool.rye.workspace?

What about instead of adding a workspace only value we just expect members = [] where members is empty? So it's implied to be used as a workspace without members to install.

I guess there are different ways to go but I was thinking that a workspace-only can be used both with members and without. And it would always be a workspace, hence the workspace key should be set anyway.

(Edit, I guess to clarify, in my model the "only" does not mean there are no members. It means the project is only a workspace and not a python package. And a workspace can always have members in it.)

The tool.rye.workspace configuration is not normally needed for single project development. The workspace-only here might be more appropriately called no-package? or is-package = true ?

Setting tool.rye.workspace makes sure that rye creates the virtualenv in that (the current) project's directory, which fits well with the "workspace-only" feature.

Do we need an additional orthogonal marker that's independent of workspace? Then you create more scenarios and additional complexity (package or "not package", workspace or not workspace is a 2x2 matrix). We can have three combinations, where a pyproject.toml becomes with rye, a:

python package

python package and a workspace

workspace (only)

To just keep the discussion diverse, I kinda think about this as everything is a workspace. Some workspaces you want to add packages/projects (members) to. Some of those projects/packages (members) are meant to be installed or their environments are managed via a workspace .venv.

So

python package

What keeps us from considering it as

python package and a workspace

and

workspace (only)

Is just a workspace with no added members.

I appreciate that. A workspace is a ceiling, or the top level of a thing. Can't include workspaces in workspaces. Then it would seem anything that is "the top" is a workspace?

bluss · 2023-06-11T16:05:28Z

rye/src/lock.rs

+            let applicable_extras =
+                format_project_extras(features_by_project.as_ref(), &pyproject)?;
+            writeln!(local_req_file, "-e {}{}", rel_url, applicable_extras)?;
+        }


I think this skips installing any workspace member, but it's only the root we want to skip installing (?)

Yes indeed, further consideration is needed here.

fwiw I think your PR did something useful when I tried it, I inserted the skip root (but keep other members) here and a name in the pyproject file, and then rye sync did what was expected I think - pulling in members and dev-dependencies. Since this is all close to the pip-tools backend (which might be replaced), I think it's tricky to know how to approach this change. (I mean: whatever we implement might need to be changed later?)

ischaojie · 2023-06-14T03:43:30Z

Using project-only instead of workspace-only might be a better expression. Suppose I have a workspace called o that contains projects o1 and o2, and the file structure looks like this:

/o
  /pyproject.toml
  /src/o
  /o1
    /pyproject.toml
    /src/o1
  /o2
    /pyproject.toml
    /src/o2

Each project can declare whether it is project-only or not (identify project-only=true in [tool.rye]). For example, I'm assuming that o1 is a package, o2 is just a normal project, and root o is also a package, so that when I sync or build, I generate -e file:. -e file:o1 and o-0.1.0-py3-none-any.whl o1-0.1.0-py3-none-any.whl.

bluss · 2023-06-14T18:19:45Z

@ischaojie it does seem to fit together well using project only. When you say for example o2 is a project only, what is that used for? Wondering about the usecase for having a leaf that's not a python lib.

0xDing · 2023-06-22T12:32:08Z

@ischaojie it does seem to fit together well using project only. When you say for example o2 is a project only, what is that used for? Wondering about the usecase for having a leaf that's not a python lib.

some team use monorepo

T-256 · 2023-07-16T09:19:24Z

Thanks to all of discussions, here is what's I take:

Packages are buildable to be install on an environment.
Environments include installed packages.
Workspaces are managing packages (if available) and environments (if available).
Projects are for creating different workspace kinds.

My expectation from Rye:

As Project Manager:
1. Create a packable workspace.
2. Create a non-packable workspace.
As Packable Workspace Manager:
- build, install, publish, ... current and workspace members packages.
As Non-Packable Workspace Manager:
- build, install, publish, ... only workspace members packages (if any available).

Currently rye init xyz will create Packable Workspace:

.
├── .git
├── .gitignore
├── .python-version
├── README.md
├── pyproject.toml
└── src
    └── xyz
        └── __init__.py

pyproject.toml

[project]
name = "xyz"
version = "0.1.0"
description = "Add a short description here"
authors = [
    { name = "Tester", email = "Tester@test.com" }
]
dependencies = []
readme = "README.md"
requires-python = ">= 3.7"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.rye]
managed = true

[tool.hatch.metadata]
allow-direct-references = true

This is a workspace that manages one package.

IIUC what's this PR tries to do is prevent create a workspace that limited to any packages.
rye init xyz --workspace-only:

.
├── .git
├── .gitignore
├── .python-version
├── README.md
├── pyproject.toml

pyproject.toml

[project]
name = "xyz"
version = "0.1.0"
description = "Add a short description here"
authors = [
    { name = "Tester", email = "Tester@test.com" }
]
dependencies = []
readme = "README.md"
requires-python = ">= 3.7"

[tool.rye]
managed = true

By this, no package created initially. (build-system is not in pyproject.toml)
You can still create sub projects here and add them to tool.rye.workspace.members.
IMO non-packable workspace should be default behavior and use packable workspace only when explicitly called (e.g. --private or --build-system xxxxx).

mitsuhiko · 2024-01-21T18:16:57Z

I think I will add a version of this. I do find the "project only" moniker confusing. Currently thinking of just calling it "virtual" and not be too descriptive about what this means.

mitsuhiko · 2024-01-21T19:41:05Z

Closing in favor of updated #551

bluss reviewed Jun 11, 2023

View reviewed changes

Support workspace only project

fe8ca16

ischaojie force-pushed the workspace-only branch from e36a33c to 901bea3 Compare June 14, 2023 03:46

Change to project-only

e9c5722

ischaojie force-pushed the workspace-only branch from 901bea3 to e9c5722 Compare June 14, 2023 04:02

T-256 mentioned this pull request Nov 17, 2023

Using rye without project name and version in pyproject.toml #498

Closed

mitsuhiko mentioned this pull request Jan 21, 2024

Added support for virtual projects #551

Merged

mitsuhiko closed this Jan 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support workspace only project #309

Support workspace only project #309

ischaojie commented Jun 11, 2023 •

edited

Loading

bluss Jun 11, 2023

cnpryer Jun 12, 2023

bluss Jun 12, 2023 •

edited

Loading

ischaojie Jun 12, 2023

bluss Jun 12, 2023

cnpryer Jun 12, 2023

bluss Jun 12, 2023 •

edited

Loading

bluss Jun 11, 2023

ischaojie Jun 12, 2023

bluss Jun 12, 2023 •

edited

Loading

ischaojie commented Jun 14, 2023 •

edited

Loading

bluss commented Jun 14, 2023 •

edited

Loading

0xDing commented Jun 22, 2023

T-256 commented Jul 16, 2023

mitsuhiko commented Jan 21, 2024

mitsuhiko commented Jan 21, 2024

Support workspace only project #309

Support workspace only project #309

Conversation

ischaojie commented Jun 11, 2023 • edited Loading

bluss Jun 11, 2023

Choose a reason for hiding this comment

cnpryer Jun 12, 2023

Choose a reason for hiding this comment

bluss Jun 12, 2023 • edited Loading

Choose a reason for hiding this comment

ischaojie Jun 12, 2023

Choose a reason for hiding this comment

bluss Jun 12, 2023

Choose a reason for hiding this comment

cnpryer Jun 12, 2023

Choose a reason for hiding this comment

bluss Jun 12, 2023 • edited Loading

Choose a reason for hiding this comment

bluss Jun 11, 2023

Choose a reason for hiding this comment

ischaojie Jun 12, 2023

Choose a reason for hiding this comment

bluss Jun 12, 2023 • edited Loading

Choose a reason for hiding this comment

ischaojie commented Jun 14, 2023 • edited Loading

bluss commented Jun 14, 2023 • edited Loading

0xDing commented Jun 22, 2023

T-256 commented Jul 16, 2023

mitsuhiko commented Jan 21, 2024

mitsuhiko commented Jan 21, 2024

ischaojie commented Jun 11, 2023 •

edited

Loading

bluss Jun 12, 2023 •

edited

Loading

bluss Jun 12, 2023 •

edited

Loading

bluss Jun 12, 2023 •

edited

Loading

ischaojie commented Jun 14, 2023 •

edited

Loading

bluss commented Jun 14, 2023 •

edited

Loading