workspace: when setting post-release, dependents not updated #206

antifuchs · 2020-03-20T22:42:58Z

With your help in the previous issue, I could cut a 0.5.0 release of chars, thanks for that! Unfortunately, the process failed when it was bumping the dev versions for the two crates:

:;    cargo release minor --verbose
[2020-03-20T13:38:50Z DEBUG] Files changed in chars_data since chars_data-v0.4.1: [
        "/Users/asf/Hacks/chars/chars_data/chars_data/Cargo.toml",
        "/Users/asf/Hacks/chars/chars_data/chars_data/data/entities/LICENSE",
        "/Users/asf/Hacks/chars/chars_data/chars_data/data/entities/README.md",
        "/Users/asf/Hacks/chars/chars_data/chars_data/data/entities/entities.json",
        "/Users/asf/Hacks/chars/chars_data/chars_data/data/entities/retrieve.sh",
        "/Users/asf/Hacks/chars/chars_data/chars_data/data/unicode/NameAliases.txt",
        "/Users/asf/Hacks/chars/chars_data/chars_data/data/unicode/UnicodeData.txt",
        "/Users/asf/Hacks/chars/chars_data/chars_data/src/ascii.rs",
        "/Users/asf/Hacks/chars/chars_data/chars_data/src/fst_generator.rs",
        "/Users/asf/Hacks/chars/chars_data/chars_data/src/unicode.rs",
    ]
[2020-03-20T13:38:50Z DEBUG] Files changed in chars since v0.4.1: [
        "/Users/asf/Hacks/chars/chars/chars/Cargo.toml",
        "/Users/asf/Hacks/chars/chars/chars/release.toml",
        "/Users/asf/Hacks/chars/chars/chars/src/display.rs",
        "/Users/asf/Hacks/chars/chars/chars/src/unicode/mod.rs",
        "/Users/asf/Hacks/chars/chars/chars/tests/human_names.rs",
    ]
    ]
Release
Release
  chars_data 0.5.0
  chars 0.5.0
? [y/N]
y
[2020-03-20T13:38:53Z INFO ] Update chars_data to version 0.5.0
[2020-03-20T13:38:53Z INFO ] Fixing chars's dependency on chars_data to `^0.5.0` (from `^0.4.1-dev`)
[2020-03-20T13:38:53Z INFO ] Update chars to version 0.5.0
[2020-03-20T13:38:54Z DEBUG] Substituting values for /Users/asf/Hacks/chars/chars/../CHANGELOG.md
[2020-03-20T13:38:54Z DEBUG] Substituting values for /Users/asf/Hacks/chars/chars/../CHANGELOG.md
[2020-03-20T13:38:54Z DEBUG] Substituting values for /Users/asf/Hacks/chars/chars/../CHANGELOG.md
[master 08543b9] Release {{version}} 🎉🎉
 4 files changed, 8 insertions(+), 6 deletions(-)
[2020-03-20T13:38:54Z INFO ] Running cargo publish on chars_data
    Updating crates.io index
   Packaging chars_data v0.5.0 (/Users/asf/Hacks/chars/chars_data)
   Verifying chars_data v0.5.0 (/Users/asf/Hacks/chars/chars_data)
   Compiling memchr v2.3.3
   Compiling fst v0.4.0
   Compiling lazy_static v1.4.0
   Compiling regex-syntax v0.6.17
   Compiling unicode-width v0.1.7
   Compiling quick-error v1.2.3
   Compiling unicode_names2 v0.4.0
   Compiling thread_local v1.0.1
   Compiling getopts v0.2.21
   Compiling aho-corasick v0.7.10
   Compiling regex v1.3.5
   Compiling regex v1.3.5
   Compiling chars_data v0.5.0 (/Users/asf/Hacks/chars/target/package/chars_data-0.5.0)
    Finished dev [unoptimized + debuginfo] target(s) in 24.07s
   Uploading chars_data v0.5.0 (/Users/asf/Hacks/chars/chars_data)
[2020-03-20T13:39:47Z INFO ] Waiting for publish to complete...
[2020-03-20T13:39:50Z INFO ] Running cargo publish on chars
    Updating crates.io index
   Packaging chars v0.5.0 (/Users/asf/Hacks/chars/chars)
   Verifying chars v0.5.0 (/Users/asf/Hacks/chars/chars)
  Downloaded chars_data v0.5.0
   Compiling memchr v2.3.3
   Compiling fst v0.4.0
   Compiling lazy_static v1.4.0
   Compiling unicode-width v0.1.7
   Compiling regex-syntax v0.6.17
   Compiling unicode_names2 v0.4.0
   Compiling quick-error v1.2.3
   Compiling byteorder v1.3.4
   Compiling thread_local v1.0.1
   Compiling getopts v0.2.21
   Compiling aho-corasick v0.7.10
   Compiling regex v1.3.5
   Compiling chars_data v0.5.0
   Compiling chars v0.5.0 (/Users/asf/Hacks/chars/target/package/chars-0.5.0)
    Finished dev [unoptimized + debuginfo] target(s) in 45.72s
   Uploading chars v0.5.0 (/Users/asf/Hacks/chars/chars)
[2020-03-20T13:40:38Z INFO ] Waiting for publish to complete...
[2020-03-20T13:40:44Z DEBUG] Creating git tag chars_data-v0.5.0
[2020-03-20T13:40:44Z DEBUG] Creating git tag v0.5.0
[2020-03-20T13:40:44Z INFO ] Starting chars_data's next development iteration 0.5.1-dev
[2020-03-20T13:40:44Z WARN ] Fatal: Invalid TOML file format: Error during execution of `cargo metadata`: error: no matching package named `chars_data` found
    location searched: /Users/asf/Hacks/chars/chars_data
    prerelease package needs to be specified explicitly
    chars_data = { version = "0.5.1-dev" }
    required by package `chars v0.5.0 (/Users/asf/Hacks/chars/chars)

So, looks like what's happening in that stage is:

It bumps the version of chars_data to 0.5.1-dev
It tries to bump the dependency in chars on chars_data to 0.5.1-dev
at that point, cargo refuses to find dev release version dependencies with no exact match -> error

I think I saw a cargo bug to that effect before, but I can't find it at the moment. I wonder if you can even do that bump with any in-flight versions of files. I imagine that the process might have to be:

Bump the version of chars_data to 0.5.1-dev but keep the previous Cargo.toml in place (save the update to a temp file, probably?)
Bump the dependency version in all dependent crates while cargo metadata still agrees, but also keep those old Cargo.toml files in place
Move all the temp files into place

The text was updated successfully, but these errors were encountered:

epage · 2020-03-23T18:20:09Z

This is a wild guess because I feel like the cargo book doesn't do a good job talking about prerelease

Currently, chars has

dev-version-ext = "dev"

I think that should be

dev-version-ext = "dev.0"

We append dev-version-ext as-is and do not parse it or add the number to it. I suspect the number is required.

Note the default is

dev-version-exit = "alpha.0"

FYI we only support bumping RC, Beta, and Alpha. If Cargo supports Dev, we could add support for it.

antifuchs · 2020-03-24T02:18:16Z

I don't think that's what is happening. I have constructed a cargo workspace in the in-between state that cargo release leaves chars's workspace in https://github.com/antifuchs/cargo-dev-version-test-example (in the dev-version-half-done branch):

Two crates lib-foo and lib-bar.
lib-foo depends on lib-bar
lib-bar is at version 0.1.0-dev.0
lib-foo still declares its dependency on lib-bar at version 0.1.0.

Running cargo metadata in the lib-foo directory of that workspace errors out, because cargo refuses to resolve a dependency that's declared as having version 0.1.0 but the actual version in the path dependency is 0.1.0-dev.0 (or 0.1.0-dev, it doesn't matter).

I believe that's what's happening in the cargo release process for chars above: After cutting the releases, it bumps the version of chars_data to 0.5.0-dev, then cargo in chars can't find the original dependency (declared to be 0.5.0) anymore, because it's one with alphabetic parts in the version numbers.

I'm pretty sure this is cargo's pre-release prevention kicking in here... If you only-numeric components like 0.1.1, everything in those in-between states keeps fitting together perfectly.

One thing I can think of that would probably help is to reverse the order: Bump the versions (and their dependency versions) of the last-released crate (I assume that's the "outermost" dependent crate) first. You can see an example of this in the dev-version-working branch of the repo above.

epage · 2020-03-24T06:11:54Z

Oh, we aren't updating dependents after bumping the post-release version. I overlooked that since I don't do post-release version bumps.

The logic for updating dependents is here
https://github.com/sunng87/cargo-release/blob/master/src/main.rs#L514

while that is missing from here
https://github.com/sunng87/cargo-release/blob/master/src/main.rs#L728

ckaran · 2020-05-08T14:23:41Z

I think I just got bitten by this exact bug. Dry run output (after cleaning up the failure):

cfkaran2@Hammer:~/Documents/repositories/bit_network$ cargo release --dry-run --workspace -vvvvvv
[2020-05-08T14:11:21Z DEBUG] Creating git tag bit_network_library-v0.1.0
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network/bit_network_library
[2020-05-08T14:11:21Z TRACE] git tag -a bit_network_library-v0.1.0 -m (cargo-release) bit_network_library version 0.1.0 
[2020-05-08T14:11:21Z DEBUG] Creating git tag bit_network_driver-v0.1.0
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network/bit_network_driver
[2020-05-08T14:11:21Z TRACE] git tag -a bit_network_driver-v0.1.0 -m (cargo-release) bit_network_driver version 0.1.0 
[2020-05-08T14:11:21Z DEBUG] Creating git tag bit_network_visualizer-v0.1.0
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network/bit_network_visualizer
[2020-05-08T14:11:21Z TRACE] git tag -a bit_network_visualizer-v0.1.0 -m (cargo-release) bit_network_visualizer version 0.1.0 
[2020-05-08T14:11:21Z INFO ] Starting bit_network_library's next development iteration 0.1.1-alpha.0
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network/bit_network_library
[2020-05-08T14:11:21Z TRACE] git commit  -am (cargo-release) start next development iteration 0.1.1-alpha.0
[2020-05-08T14:11:21Z INFO ] Starting bit_network_driver's next development iteration 0.1.1-alpha.0
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network/bit_network_driver
[2020-05-08T14:11:21Z TRACE] git commit  -am (cargo-release) start next development iteration 0.1.1-alpha.0
[2020-05-08T14:11:21Z INFO ] Starting bit_network_visualizer's next development iteration 0.1.1-alpha.0
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network/bit_network_visualizer
[2020-05-08T14:11:21Z TRACE] git commit  -am (cargo-release) start next development iteration 0.1.1-alpha.0
[2020-05-08T14:11:21Z INFO ] Pushing HEAD to origin
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network
[2020-05-08T14:11:21Z TRACE] git push origin
[2020-05-08T14:11:21Z INFO ] Pushing bit_network_library-v0.1.0 to origin
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network
[2020-05-08T14:11:21Z TRACE] git push origin bit_network_library-v0.1.0
[2020-05-08T14:11:21Z INFO ] Pushing bit_network_driver-v0.1.0 to origin
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network
[2020-05-08T14:11:21Z TRACE] git push origin bit_network_driver-v0.1.0
[2020-05-08T14:11:21Z INFO ] Pushing bit_network_visualizer-v0.1.0 to origin
[2020-05-08T14:11:21Z TRACE] cd /home/cfkaran2/Documents/repositories/bit_network
[2020-05-08T14:11:21Z TRACE] git push origin bit_network_visualizer-v0.1.0

And doing the actual run:

cfkaran2@Hammer:~/Documents/repositories/bit_network$ cargo release --workspace -vvvvvv
Release
  bit_network_library 0.1.0
  bit_network_driver 0.1.0
  bit_network_visualizer 0.1.0
? [y/N] 
y
[2020-05-08T14:18:40Z DEBUG] Creating git tag bit_network_library-v0.1.0
[2020-05-08T14:18:40Z DEBUG] Creating git tag bit_network_driver-v0.1.0
[2020-05-08T14:18:40Z DEBUG] Creating git tag bit_network_visualizer-v0.1.0
[2020-05-08T14:18:40Z INFO ] Starting bit_network_library's next development iteration 0.1.1-alpha.0
[2020-05-08T14:18:40Z WARN ] Fatal: Invalid TOML file format: Error during execution of `cargo metadata`: error: no matching package named `bit_network_library` found
    location searched: /home/cfkaran2/Documents/repositories/bit_network/bit_network_library
    prerelease package needs to be specified explicitly
    bit_network_library = { version = "0.1.1-alpha.0" }
    required by package `bit_network_driver v0.1.0 (/home/cfkaran2/Documents/repositories/bit_network/bit_network_driver)`
cfkaran2@Hammer:~/Documents/repositories/bit_network$ git tag -l
bit_network_driver-v0.1.0
bit_network_library-v0.1.0
bit_network_visualizer-v0.1.0
cfkaran2@Hammer:~/Documents/repositories/bit_network$ cargo metadata --format-version 1
error: no matching package named `bit_network_library` found
location searched: /home/cfkaran2/Documents/repositories/bit_network/bit_network_library
prerelease package needs to be specified explicitly
bit_network_library = { version = "0.1.1-alpha.0" }
required by package `bit_network_driver v0.1.0 (/home/cfkaran2/Documents/repositories/bit_network/bit_network_driver)`

It looks like bit_network_library gets version bumped, and then cargo metadata gets grumpy, both leaving my repository is weird inconsistent state (properly tagged, but only one of the Cargo.toml files being version bumped).

Any idea on if/when this will be resolved?

antifuchs · 2020-05-08T14:33:16Z

@ckaran, I managed to cleanly release my crate with a cargo-release version that had the fix from #209 applied. I'd love some independent verification, so if you're willing to try it - I'd love to see if it works for you (:

ckaran · 2020-05-11T13:52:17Z

@antifuchs I'd be glad to! Is there a recommended method for testing it out that won't replace my current install of cargo release?

antifuchs · 2020-05-11T14:21:00Z

@ckaran I'm pretty sure you can cargo run --manifest-path /path/to/cargo-release/Cargo.toml -- [cargo release options] your way to victory! But, note: The PR just got merged to HEAD, you can probably just install the git version of cargo release now (:

ckaran · 2020-05-11T15:26:42Z

@antifuchs I just cloned the latest version of the repository (14ecbe336ae1ed7205b4a7f5f4f9783b76b132d5), and it didn't work for me.

$ cargo run --manifest-path /home/cfkaran2/Documents/repositories/cargo-release/Cargo.toml -- release --workspace
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/home/cfkaran2/Documents/repositories/cargo-release/target/debug/cargo-release release --workspace`
Release
  bit_network_library 0.1.0
  bit_network_driver 0.1.0
  bit_network_visualizer 0.1.0
? [y/N] 
y
[2020-05-11T15:18:27Z INFO ] Starting bit_network_library's next development iteration 0.1.1-alpha.0
[2020-05-11T15:18:27Z WARN ] Fatal: Invalid TOML file format: Error during execution of `cargo metadata`: error: no matching package named `bit_network_library` found
    location searched: /home/cfkaran2/Documents/repositories/bit_network/bit_network_library
    prerelease package needs to be specified explicitly
    bit_network_library = { version = "0.1.1-alpha.0" }
    required by package `bit_network_driver v0.1.0 (/home/cfkaran2/Documents/repositories/bit_network/bit_network_driver)`

The tags were correctly set in the repository, but only the top-level Cargo.toml file was picked up, and only one of the Cargo.toml files was updated (bit_network_library).

davepacheco · 2021-02-02T00:56:53Z

I seem to have run into the same problem with cargo-release v0.13.10. In the root of my repo, I've got "dropshot" and "dropshot_endpoint". "dropshot" depends on "dropshot_endpoint" -- always the same version. (The latter is a proc macro used in the former -- that's the only reason it's a separate crate.) It seems that when I try to do a release of v0.4.0, it updates "dropshot_endpoint" to v0.4.1-alpha.0, then tries to update "dropshot", but cargo bails:

[2021-02-02T00:42:49Z WARN ] Fatal: Invalid TOML file format: Error during execution of `cargo metadata`: error: no matching package named `dropshot_endpoint` found
    location searched: /Users/dap/oxide/dropshot/dropshot_endpoint
    prerelease package needs to be specified explicitly
    dropshot_endpoint = { version = "0.4.1-alpha.0" }
    required by package `dropshot v0.4.0 (/Users/dap/oxide/dropshot/dropshot)`

This seems to go back to @antifuchs's original comment: there doesn't seem to be an order in which this will always work without saving the Cargo.toml updates to temp files first?

FYI: in my case, I didn't have a clean cargo release because I had chosen the same tag for both crates and it bailed out. I fixed this by hand and reran it as cargo release --skip-tag --skip-publish -vvv 0.4.0 and got the error above. I don't think this matters but I wanted to mention it just in case!

epage · 2021-02-02T01:26:33Z

@ckaran and @davepacheco I created a test repo to try to reproduce the problems you are having but I didn't get failures. Could you create a PR that gets it into the failure state to help clarify what the difference is?

https://github.com/epage/cargo-release-test

davepacheco · 2021-02-02T18:23:46Z

Thanks @epage! I went ahead and did that: epage/cargo-release-test#1

This was helpful. My problem was a result of trying to use cargo release with the same version after a previous cargo release had failed. It would be nice if the tool handled this better, but it may be different than what's described here.

ckaran · 2021-02-04T14:37:05Z

@epage My apologies, but I'm not likely to get the time to do this for quite a while; I'm in the middle of several other projects and can't really pause. Is what @davepacheco sufficient, or do you really need me to create a minimized workspace that illustrates the issue?

epage · 2021-08-12T20:06:32Z

Finally got a chance to dig into some of these problems. One problem is we relied on cargo to resolve dependencies for us but for some reason in one case its not, so we are getting confused

epage · 2021-08-12T20:50:29Z

I believe #295 resolves the remaining work for closing this out. Feel free to open an issue if you run into another corner case.

epage added the bug Not as expected label Mar 24, 2020

antifuchs mentioned this issue Mar 29, 2020

Update dependents to dev versions #209

Merged

epage changed the title ~~workspace release: failed during bump to dev version~~ workspace: when setting post-release, dependents not updated Jul 22, 2020

epage mentioned this issue Jul 22, 2020

Releasing a workspace does not update local path dependencies #223

Closed

davepacheco mentioned this issue Feb 2, 2021

prepare for releases via cargo release oxidecomputer/dropshot#82

Merged

davepacheco mentioned this issue Feb 2, 2021

repro dropshot issue epage/cargo-release-test#1

Open

epage closed this as completed Aug 12, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

workspace: when setting post-release, dependents not updated #206

workspace: when setting post-release, dependents not updated #206

antifuchs commented Mar 20, 2020

epage commented Mar 23, 2020

antifuchs commented Mar 24, 2020

epage commented Mar 24, 2020

ckaran commented May 8, 2020

antifuchs commented May 8, 2020

ckaran commented May 11, 2020

antifuchs commented May 11, 2020

ckaran commented May 11, 2020

davepacheco commented Feb 2, 2021

epage commented Feb 2, 2021

davepacheco commented Feb 2, 2021

ckaran commented Feb 4, 2021

epage commented Aug 12, 2021

epage commented Aug 12, 2021

workspace: when setting post-release, dependents not updated #206

workspace: when setting post-release, dependents not updated #206

Comments

antifuchs commented Mar 20, 2020

epage commented Mar 23, 2020

antifuchs commented Mar 24, 2020

epage commented Mar 24, 2020

ckaran commented May 8, 2020

antifuchs commented May 8, 2020

ckaran commented May 11, 2020

antifuchs commented May 11, 2020

ckaran commented May 11, 2020

davepacheco commented Feb 2, 2021

epage commented Feb 2, 2021

davepacheco commented Feb 2, 2021

ckaran commented Feb 4, 2021

epage commented Aug 12, 2021

epage commented Aug 12, 2021