Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for publishing of crates in subdirectories of repositories #939

Merged
merged 15 commits into from
Mar 4, 2022

Conversation

mosteo
Copy link
Member

@mosteo mosteo commented Feb 12, 2022

These are regular crates for all purposes but for the detail that they're not at the root of a repository, so they don't introduce complex issues like in previous proposals for child crates.

The only fancy thing is that to reuse a repository checkout, instead of using the name_version_hash naming in the dependencies, we use a repoBasename_hash naming for such releases. This is out of sight of the user anyway, as it happens inside alire/cache/dependencies.

This is an empty file in the dependencies folder that helps curious users to
understand where a particular release is coming from. For example:

dependencies
 +-- monorepo_adef1234
      +-- mycrate
 +-- mycrate_1.0.0_in_monorepo_adef1234
@mosteo mosteo force-pushed the feat/monorepo branch 10 times, most recently from f648e82 to fa7487a Compare February 13, 2022 02:00
Windows may return paths with either long or short names which complicate a bit
comparing two directories as we do to detect nested crates. To us, the problem
esteems from git returning different names than the Ada std lib.
@mosteo mosteo marked this pull request as ready for review February 13, 2022 11:46
Co-authored-by: Jesper Quorning <jesper.quorning@gmail.com>
Copy link
Member

@Fabien-Chouteau Fabien-Chouteau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest I am bit worried about this monorepo support, to me it sounds like the old ways of doing things 🤷‍♂️

I have a couple questions:

  • Can two releases from a monorepo with the same version come on different commits?
  • Can I release a patch fix for a create in a mono repo without making a release for the other crates?

-- Verify that subdir is not used with an archive (it is only for VCSs)

if Subdir /= "" and then Commit = "" then
Raise_Checked_Error ("Cannot publish a nested crate from an archive");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have this limitation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to tackle the presumed more wanted feature first. In principle it should be doable too, but I'm not sure it makes sense to publish as a source archive and include folders over the one you want to publish. Maybe to share resources at some point?

@mosteo
Copy link
Member Author

mosteo commented Feb 14, 2022

  • Can two releases from a monorepo with the same version come on different commits?

This PR is just selecting a subdir inside the checkout and ignoring the rest of it, there is nothing fancy going on with versions. It's point 3 in #737 (that I should have linked in the initial submission but forgot).

So, not sure if this is what you ask, but each crate still has its own version, even if several come from the same commit, and the version needs not to match. Conversely you can have releases for different crates coming from different commits too. Or have different crates with the same version in different commits.

Or, if you mean: can the code for the same version of the same release be found in two different checkouts of the same repo? Then yes, but their subdirs can't appear simultaneously in the environment, as only one of the two will end selected by the solver for use.

Still, I should add a test for that, if you approve this PR ;-)

  • Can I release a patch fix for a crate in a mono repo without making a release for the other crates?

Yes, every crate is published independently just like before. I just realized that cargo has something related: https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html (that would be point 5 in #737); not sure how that interacts with publishing though, I see no mention of it in cargo publish. So I expect it's also orthogonal there.

To be honest I am bit worried about this monorepo support, to me it sounds like the old ways of doing things man_shrugging

Well, yes, we've seen the arguments before :-) I think it is orthogonal how people want to organize their crates in repositories with the publishing proper; our particular problem is that it's more trouble publishing right now if you share the repo as that forces to upload tarballs manually.

At the end of the day, for us currently is a matter of automation. If we could easily upload tars without having to mess with the public releases of the user, we could dispense with this particular PR (point 4 in #737). But that for some reason makes me even more wary. Create Alire-specific releases in their repo? How to distinguish our artifacts from theirs in the same GitHub release? It would be different if we had dedicated backend infrastructure.

Although on second though maybe indeed having a single alire releases release in their repo, with all our tarballs inside, is practical. GitHub says that the only limit is on the individual files size, not on the release proper.

@onox
Copy link
Contributor

onox commented Feb 15, 2022

To be honest I am bit worried about this monorepo support, to me it sounds like the old ways of doing things man_shrugging

Managing code spread over more than a dozen repos instead of one is just very cumbersome.

Well, yes, we've seen the arguments before :-) I think it is orthogonal how people want to organize their crates in repositories with the publishing proper; our particular problem is that it's more trouble publishing right now if you share the repo as that forces to upload tarballs manually.

Uploaded tarballs are also quite opaque and it's not easy to verify you get the exact same code as some particular commit (unlike GitHub's .zip download feature).

@onox
Copy link
Contributor

onox commented Feb 25, 2022

@mosteo Is something blocking this PR? I would like to start publishing the wayland-ada crates to the index 🙂

@mosteo
Copy link
Member Author

mosteo commented Feb 26, 2022

@mosteo Is something blocking this PR? I would like to start publishing the wayland-ada crates to the index slightly_smiling_face

@onox I want to include an extra test for a case that raised Fabien. Besides, if you start using this feature your releases will go into the devel index so they won't be generally available... Is that OK for you?

@onox
Copy link
Contributor

onox commented Feb 27, 2022

Is that OK for you?

Sure, there's no rush.

@mosteo mosteo merged commit 26c5c28 into alire-project:master Mar 4, 2022
@mosteo mosteo deleted the feat/monorepo branch March 4, 2022 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants