Publish unpublished crates to crates.io

[corneliusroemer]

Problem description

I noticed that in the "install from source" guide you mention that one can't cargo install pixi from crates.io because of reliance on unpublished crates.

It might be nice to publish those unpublished crates to crates.io

Are these crates you developed or dependencies? Which ones are they?

The reason for this is that we depend on some unpublished crates which disallows us to publish to crates.io.

[corneliusroemer]

It looks like it's possibly only the astral-sh UV crates that aren't on cargo.

Stuff like that:

distribution-filename = { git = "https://github.com/astral-sh/uv", tag = "0.4.0" }
distribution-types = { git = "https://github.com/astral-sh/uv", tag = "0.4.0" }

[baszalmstra]

Yep there are quite a few of them. We did split pixi into several crates. Some of them do not depend on any of the astral crates and we do plan on publishing those to crates.

Unfortunately for the pixi binary itself we cant until astral decides to publish some of those crates.

[corneliusroemer]

You might be able to fork and publish the forks but yeah that'd be annoying to maintain. Have you reached out to astral and asked them to consider publishing themselves? They might just not think that anyone is using their crates.

…

On Sat, Sep 7, 2024, 13:45 Bas Zalmstra ***@***.***> wrote: Yep there are quite a few of them. We did split pixi into several crates. Some of them do not depend on any of the astral crates and we do plan on publishing those to crates. Unfortunately for the pixi binary itself we cant until astral decides to publish some of those crates. — Reply to this email directly, view it on GitHub <#2002 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF77AQIBRZUUF4B2SOW7MU3ZVLRM7AVCNFSM6AAAAABNZ562P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZVGE3DCNRYGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[baszalmstra]

We sure have but unfortunately no luck.

Forking and publishing their work also doesnt seem appropriate.

If I may ask, why would you prefer to get pixi from crates.ion? You can still install pixi with cargo by using a git reference.

[corneliusroemer]

I couldn't find a very explicit request in the uv issue list to publish unpublished crates to crates.io

Here are some related issues though:

• Paths to using published versions of vendored, forked crates? astral-sh/uv#5588
• Feature request: Publishing crate docs astral-sh/uv#4392
• Reconcile published and vendored pep508_rs crates astral-sh/uv#3538

Did I miss something?

Reasons for using crates.io for everything instead of relying on git for some:

Using Git dependencies in Cargo instead of crates.io introduces several risks:

1. Build Reproducibility: Using crates.io guarantees the use of a specific, immutable version of a crate, ensuring consistent builds. With Git, there's no such guarantee unless you pin to a specific commit, which can be cumbersome.
2. Crates.io auditing: crates.io crates undergo some level of community review and can be more easily audited for security. Using arbitrary Git repositories can expose your project to unvetted or potentially insecure code (don't think there's a bug risk for this with astral but still)
3. Availability Issues: Git repositories may become unavailable if the repo is deleted or goes offline, while crates.io provides a stable hosting service for crates. (again no big risk in this case)

Plus easier source installs, reusability etc

[ruben-arts]

He @corneliusroemer,

We completely agree, and if there was a way to handle it with a level of maintenance that we found acceptable, we would have done it already. We've had offline conversations with the uv team, which is where we got this information.

A short counter-argument for why we’re not too worried about using git dependencies at the moment:

1. The Cargo.lock contains the commit hashes, so from a reproducibility perspective, we’re pretty solid. We update uv dependencies manually, primarily handled by @tdejager, which ensures careful, non-automated updates.
2. I’m not sure how putting the tool on crates.io would be any different, as we already get significant input via our GitHub repo.
3. uv has 600 forks, which should make it robust in terms of availability. GitHub going down seems less likely than crates.io, given how much more depends on it.

The biggest downside right now is that you can't cargo install pixi, and it’s harder to start distributing our own crates—but we're actually starting to address that now for the pixi build feature.