Allow crates to be published with cyclic dev-dependencies #4242
Comments
|
Yeah in general I think we should basically ignore dev-dependencies when we publish. We shouldn't attempt to resolve them, verify they're already published, etc. |
|
Maybe dev-dependencies don't have to be shown in the crate's page too. |
|
I bumped into this issue as well. From what I can tell the workaround above gets us by this for now. |
|
+1 to this-- it now affects the futures crate. |
While preparing for publication, I realized I cannot publish crates with cyclic dev-dependencies. That's an interesting issue, and a work-around is currently used: rust-lang/cargo#4242 Another issue: it seems "imgui-glium-renderer" requires the "glutin" feature of "glium". However it was not enabled. So I enabled it.
While preparing for publication, I realized I cannot publish crates with cyclic dev-dependencies. That's an interesting issue, and a work-around is currently used: rust-lang/cargo#4242 Another issue: it seems "imgui-glium-renderer" requires the "glutin" feature of "glium". However it was not enabled. So I enabled it.
While preparing for publication, I realized I cannot publish crates with cyclic dev-dependencies. That's an interesting issue, and a work-around is currently used: rust-lang/cargo#4242 Another issue: it seems "imgui-glium-renderer" requires the "glutin" feature of "glium". However it was not enabled. So I enabled it.
While preparing for publication, I realized I cannot publish crates with cyclic dev-dependencies. That's an interesting issue, and a work-around is currently used: rust-lang/cargo#4242
While preparing for publication, I realized I cannot publish crates with cyclic dev-dependencies. That's an interesting issue, and a work-around is currently used: rust-lang/cargo#4242
While preparing for publication, I realized I cannot publish crates with cyclic dev-dependencies. That's an interesting issue, and a work-around is currently used: rust-lang/cargo#4242
While preparing for publication, I realized I cannot publish crates with cyclic dev-dependencies. That's an interesting issue, and a work-around is currently used: rust-lang/cargo#4242
|
This affects a lot of crates, what do we need to do to fix this? Isn't there a check somewhere that checks for the dependencies before publishing, where we can just filter dev-dependencies away and call it a day? [0] I mean, this would even affect core and std, where std could depend on core to build, but core would depend on std for tests. The check is spurious anyways, because this sort of dependencies are not circular: build core -> build std -> build core tests is ok. Tests and "dev-only" artifacts are their own crates (core+tests is a different crate than core, and so are every test/foo.rs in core, and core/examples, etc.). I'll admit that the core/std analogy is imperfect because these crates are too magical, but when something wouldn't be correct for the std library, chances are it won't be correct for users either. FWIW I just tried to prepare coresimd/stdsimd for a quick crates.io release, and ran into this, where coresimd builds standalone and stdsimd depends on coresimd, but coresimd depends on stdsimd for testing. The only solution I came up with is the same that gluon uses which is insane: publish coresimd prunning dev-dependencies, publish stdsimd, re-publish coresimd with a minor version update to use the newly publish stdsimd as dev-dependencies. Which is a lot of effort, for.. what exactly? Is it possible for users to execute [0] cc @matklad, are you willing to mentor this? It might be enough to just skip dev dependencies here: cargo/src/cargo/ops/registry.rs Line 110 in af3f1cd and also in packaging: https://github.com/rust-lang/cargo/blob/0e7a46e3276f56822b6ef48e341c522be27db17e/src/cargo/ops/cargo_package.rs |
Hm, I think a simpler work-around is possible? Just comment-out For mentoring instructions, I believe you've basically nailed it :) The only thing I have to add is that the test should go to this file, and it probably makes sense to start with a test and work through all errors. For the tests, I think we should cover at least two scenarios:
|
|
Hm I'm actually not sure that the fix would go in that location nor that we could easily write a test for this. AFAIK the check for "does this dependency exist" is done on crates.io, not locally in Cargo. In that sense we'd need to test with crates.io, not with just local Cargo test suite business. I also think the "fix" for this is to basically stop informing crates.io about dev-dependencies. Although we did that initially I'm not really sure why we need to do that, as they're never used and are otherwise just bloating the index. The fix for that would go around here by filtering out dev-dependencies. Cargo would then also need to be updated to not verify that dev-dependencies have versions (allowing This would be a relatively large change though (and somewhat of a policy shift) so it would likely need some more scrutiny and discussion before happening. |
Hm, probably a crazy idea, but, given that we already modify Cargo.toml on publishing to rewrite path dependencies (source), could we, at the same step, just drop dev-deps altogether as well? |
|
Yes I think such a fix would work, but it alone wouldn't be 100% sufficient because crates.io doesn't actually parse the manifest for dependency information (it only uses what was sent in JSON) |
While preparing for publication, I realized I cannot publish crates with cyclic dev-dependencies. That's an interesting issue, and a work-around is currently used: rust-lang/cargo#4242
|
I guess the proposed change would also now allow |
* Release 0.4.0-alpha * dependencies should use the exact version * Update CHANGELOG to reflect the new version * Remove the patch for ark-std from Cargo.toml * Temporary workaround to cyclic-dependency resolution while publishing rust-lang/cargo#4242 * Bump alpha version of ff after dev-dependencies were published as alpha * Bump alpha version of ec after dev-dependencies were published as alpha * chore: Release * add a release job to makefile make comments * Add back the missing bump for alpha.3 for serialize-derive * workflow comment * Bug fixes -> Bugfixes * Add a release-pr workflow * 2-step release workflow * test only publishable crates * Add the custom PR template * move release template to correct location * fix dependent version * fix all crates to have the same version * chore: Release * chore: Release * Switch to release-pr@v2 action * tag prefix not needed, it's only for publishing * Make the version into a choice * bump version of unreleased poly-benches * chore: Release Co-authored-by: Weikeng Chen <w.k@berkeley.edu> Co-authored-by: Pratyush Mishra <pratyushmishra@berkeley.edu>
This is used for doc-tests, and would use a cyclic dependency if an explicit version is specified. See also: rust-lang/cargo#4242 Also add some output to publishing CI.
To work around a known issue with `cargo publish` ([1}), update all self-dependencies under dev-dependencies to use `path = "."` instead of a versioned crate, as `publish` insists on fetching from crates.io, despite a correctly versioned crate being available locally. [1]: rust-lang/cargo#4242
To work around a known issue with `cargo publish` ([1}), update all self-dependencies under dev-dependencies to use `path = "."` instead of a versioned crate, as `publish` insists on fetching from crates.io, despite a correctly versioned crate being available locally. [1]: rust-lang/cargo#4242
To work around a known issue with `cargo publish` ([1}), update all self-dependencies under dev-dependencies to use `path = "."` instead of a versioned crate, as `publish` insists on fetching from crates.io, despite a correctly versioned crate being available locally. [1]: rust-lang/cargo#4242
To work around a known issue with `cargo publish` ([1}), update all self-dependencies under dev-dependencies to use `path = "."` instead of a versioned crate, as `publish` insists on fetching from crates.io, despite a correctly versioned crate being available locally. [1]: rust-lang/cargo#4242
To work around a known issue with `cargo publish` ([1}), update all self-dependencies under dev-dependencies to use `path = "."` instead of a versioned crate, as `publish` insists on fetching from crates.io, despite a correctly versioned crate being available locally. [1]: rust-lang/cargo#4242
* bump crate versions to 0.3.1 * workaround for cargo dev-dependencies problem To work around a known issue with `cargo publish` ([1}), update all self-dependencies under dev-dependencies to use `path = "."` instead of a versioned crate, as `publish` insists on fetching from crates.io, despite a correctly versioned crate being available locally. [1]: rust-lang/cargo#4242
This is used for doc-tests, and would use a cyclic dependency if an explicit version is specified. See also: rust-lang/cargo#4242 Also add some output to publishing CI.
This is used for doc-tests, and would use a cyclic dependency if an explicit version is specified. See also: rust-lang/cargo#4242 Also add some output to publishing CI.
* bump crate versions to 0.3.1 * workaround for cargo dev-dependencies problem To work around a known issue with `cargo publish` ([1]), update all self-dependencies under dev-dependencies to use `path = "."` instead of a versioned crate, as `publish` insists on fetching from crates.io, despite a correctly versioned crate being available locally. [1]: rust-lang/cargo#4242
* bump to 0.3.1 and work around cargo publish issue (#992) * bump crate versions to 0.3.1 * workaround for cargo dev-dependencies problem To work around a known issue with `cargo publish` ([1]), update all self-dependencies under dev-dependencies to use `path = "."` instead of a versioned crate, as `publish` insists on fetching from crates.io, despite a correctly versioned crate being available locally. [1]: rust-lang/cargo#4242 * Bump Janus crate versions to 0.4.0 * Fix more self-dev-dependencies As in PR#992, we work around a `cargo publish` issue.
This is used for doc-tests, and would use a cyclic dependency if an explicit version is specified. See also: rust-lang/cargo#4242 Also add some output to publishing CI.
## Description - `valence` and `valence_protocol` have been divided into smaller crates in order to parallelize the build and improve IDE responsiveness. In the process, code architecture has been made clearer by removing circular dependencies between modules. `valence` is now just a shell around the other crates. - `workspace.packages` and `workspace.dependencies` are now used. This makes dependency managements and crate configuration much easier. - `valence_protocol` is no more. Most things from `valence_protocol` ended up in `valence_core`. We won't advertise `valence_core` as a general-purpose protocol library since it contains too much valence-specific stuff. Closes #308. - Networking code (login, initial TCP connection handling, etc.) has been extracted into the `valence_network` crate. The API has been expanded and improved with better defaults. Player counts and initial connections to the server are now tracked separately. Player counts function by default without any user configuration. - Some crates like `valence_anvil`, `valence_network`, `valence_player_list`, `valence_inventory`, etc. are now optional. They can be enabled/disabled with feature flags and `DefaultPlugins` just like bevy. - Whole-server unit tests have been moved to `valence/src/tests` in order to avoid [cyclic dev-dependencies](rust-lang/cargo#4242). - Tools like `valence_stresser` and `packet_inspector` have been moved to a new `tools` directory. Renamed `valence_stresser` to `stresser`. Closes #241. - Moved all benches to `valence/benches/` to make them easier to run and organize. Ignoring transitive dependencies and `valence_core`, here's what the dependency graph looks like now: ```mermaid graph TD network --> client client --> instance biome --> registry dimension --> registry instance --> biome instance --> dimension instance --> entity player_list --> client inventory --> client anvil --> instance entity --> block ``` ### Issues - Inventory tests inspect many private implementation details of the inventory module, forcing us to mark things as `pub` and `#[doc(hidden)]`. It would be ideal if the tests only looked at observable behavior. - Consider moving packets in `valence_core` elsewhere. `Particle` wants to use `BlockState`, but that's defined in `valence_block`, so we can't use it without causing cycles. - Unsure what exactly should go in `valence::prelude`. - This could use some more tests of course, but I'm holding off on that until I'm confident this is the direction we want to take things. ## TODOs - [x] Update examples. - [x] Update benches. - [x] Update main README. - [x] Add short READMEs to crates. - [x] Test new schedule to ensure behavior is the same. - [x] Update tools. - [x] Copy lints to all crates. - [x] Fix docs, clippy, etc.
It was removed in rust-lang/futures-rs@73ade1e1 It was replaced with https://github.com/taiki-e/cargo-hack#--no-dev-deps , which works for publishing. |
|
Eventually, |
and others
In gluon I have the project split up into multiple crates with the intent that users can include only the crates they need if they desire a smaller binary footprint. For convenience I provide a main
gluoncrate which provides an easy to use interface when one uses all the crates.While this split works fine for mostly everything I want to use the easy interface when writing documentation tests so I added the
gluoncrate as a dev-dependency in gluon_vm and this compiles without issue since a little bit back. It does however break when trying to publishgluon_vmas thegluondev-dependency has not been published yet!Currently I managed to work around it by specifying a version range but it would be nice if there was a way to publish a group of crates as a whole or to ignore the version check if a dev-dependency does not exist on crates.io (yet).
The text was updated successfully, but these errors were encountered: