Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement support for optional crates enabled with dep: features #1885

Merged
merged 15 commits into from
Mar 23, 2023

Conversation

gferon
Copy link
Contributor

@gferon gferon commented Mar 17, 2023

Skip crates that are marked as optional = true and not enabled with a dep: feature.

For example, the matrix-sdk uses this extensively (see: https://github.com/matrix-org/matrix-rust-sdk/blob/main/crates/matrix-sdk/Cargo.toml) and this patch makes it work with cargo-bazel.

Copy link
Collaborator

@UebelAndre UebelAndre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting up this PR! Would you be able to add some testing for this? 🙏 😄

@gferon
Copy link
Contributor Author

gferon commented Mar 20, 2023

Thanks for putting up this PR! Would you be able to add some testing for this? pray smile

For sure, I first wanted to get #1884 merged, which I'm fixing right now.

jhchabran added a commit to sourcegraph/sourcegraph that referenced this pull request Mar 21, 2023
…9698)

This PR updates `rules_rust` to `v0.19.0`, as we're using Rust `1.68.0`
which is only supported in that version. I suspect that's what caused
the strange errors when repinning.

Quoting @tjdevries explanation on the fix: 

> Pinned explicitly with features that match the features
> required by rocket. Once bazel rules correctly roll up all the
features,
> we can remove this, but until then, this works just fine for building
> with bazel (and we rarely update rocket, so this is fine).

The whole issue are coming from:
bazelbuild/rules_rust#1884 and
bazelbuild/rules_rust#1885 which are addressing
the problem upstream.

---

Still, this now circles back to a compilation error what we've
previously seen. To reproduce it

```
bazel build //docker-images/syntax-highlighter/...
```

Which in turn leads to the following errors: 

```
ERROR: /private/var/tmp/_bazel_tech/3eea80c6015362974b7d423d1f30cb62/external/crate_index__futures-util-0.3.21/BUILD.bazel:22:13: Compiling Rust rlib futures_util v0.3.21 (180 files) [for tool] failed: (Exit 1): process_wrapper failed: error executing command (from target @crate_index__futures-util-0.3.21//:futures_util) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --env-file ... (remaining 82 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
error[E0432]: unresolved imports `futures_task::waker`, `futures_task::ArcWake`
  --> external/crate_index__futures-util-0.3.21/src/stream/stream/flatten_unordered.rs:21:20
   |
21 | use futures_task::{waker, ArcWake};
   |                    ^^^^^  ^^^^^^^ no `ArcWake` in the root
   |                    |
   |                    no `waker` in the root
   |                    help: a similar name exists in the module (notice the capitalization): `Waker`

error[E0432]: unresolved import `futures_task::ArcWake`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:23:9
   |
23 | pub use futures_task::ArcWake;
   |         ^^^^^^^^^^^^^^^^^^^^^ no `ArcWake` in the root

error[E0432]: unresolved import `futures_task::waker`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:27:9
   |
27 | pub use futures_task::waker;
   |         ^^^^^^^^^^^^^^-----
   |         |             |
   |         |             help: a similar name exists in the module (notice the capitalization): `Waker`
   |         no `waker` in the root

error[E0432]: unresolved imports `futures_task::waker_ref`, `futures_task::WakerRef`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:31:24
   |
31 | pub use futures_task::{waker_ref, WakerRef};
   |                        ^^^^^^^^^  ^^^^^^^^ no `WakerRef` in the root
   |                        |
   |                        no `waker_ref` in the root

error: aborting due to 4 previous errors

For more information about this error, try `rustc --explain E0432`.
ERROR: /private/var/tmp/_bazel_tech/3eea80c6015362974b7d423d1f30cb62/external/crate_index__futures-util-0.3.21/BUILD.bazel:22:13: Compiling Rust rlib futures_util v0.3.21 (180 files) failed: (Exit 1): process_wrapper failed: error executing command (from target @crate_index__futures-util-0.3.21//:futures_util) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --env-file ... (remaining 82 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
error[E0432]: unresolved imports `futures_task::waker`, `futures_task::ArcWake`
  --> external/crate_index__futures-util-0.3.21/src/stream/stream/flatten_unordered.rs:21:20
   |
21 | use futures_task::{waker, ArcWake};
   |                    ^^^^^  ^^^^^^^ no `ArcWake` in the root
   |                    |
   |                    no `waker` in the root
   |                    help: a similar name exists in the module (notice the capitalization): `Waker`

error[E0432]: unresolved import `futures_task::ArcWake`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:23:9
   |
23 | pub use futures_task::ArcWake;
   |         ^^^^^^^^^^^^^^^^^^^^^ no `ArcWake` in the root

error[E0432]: unresolved import `futures_task::waker`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:27:9
   |
27 | pub use futures_task::waker;
   |         ^^^^^^^^^^^^^^-----
   |         |             |
   |         |             help: a similar name exists in the module (notice the capitalization): `Waker`
   |         no `waker` in the root

error[E0432]: unresolved imports `futures_task::waker_ref`, `futures_task::WakerRef`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:31:24
   |
31 | pub use futures_task::{waker_ref, WakerRef};
   |                        ^^^^^^^^^  ^^^^^^^^ no `WakerRef` in the root
   |                        |
   |                        no `waker_ref` in the root

error: aborting due to 4 previous errors
```

## Test plan

<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->

Green CI. 

## App preview:

- [Web](https://sg-web-jh-bzl-fix-rust.onrender.com/search)

Check out the [client app preview
documentation](https://docs.sourcegraph.com/dev/how-to/client_pr_previews)
to learn more.

---------

Co-authored-by: TJ DeVries <devries.timothyj@gmail.com>
Co-authored-by: Jason Bedard <jason@aspect.dev>
@gferon
Copy link
Contributor Author

gferon commented Mar 21, 2023

@UebelAndre I added an integration test using clap as an example, but unfortunately this will required CI to run with Rust 1.60 since this is when the new dep: cargo feature syntax was stabilized.

@gferon gferon requested a review from UebelAndre March 21, 2023 16:18
@gferon gferon mentioned this pull request Mar 22, 2023
@gferon
Copy link
Contributor Author

gferon commented Mar 22, 2023

CI is finally green after fixing conflicts and merging #1893 🎉

@gferon
Copy link
Contributor Author

gferon commented Mar 22, 2023

@illicitonion JFYI this was the feature I asked you about on Slack last week if you also wanna give it a look.

@gferon gferon requested a review from UebelAndre March 22, 2023 21:53
Copy link
Collaborator

@UebelAndre UebelAndre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks great! I had one suggestion but not absolutely necessary. Can merge tomorrow morning 😄

crate_universe/src/metadata/dependency.rs Show resolved Hide resolved
@UebelAndre UebelAndre merged commit 9d6741f into bazelbuild:main Mar 23, 2023
@gferon gferon deleted the select-deps branch March 23, 2023 08:21
nyurik pushed a commit to nyurik/rules_rust that referenced this pull request Mar 28, 2023
…elbuild#1885)

* Implement support for optional crates enabled with dep: features

* Fix triple constraints for iOS and watchOS (bazelbuild#1888)

* Clippy

* Use Rust 1.60 in CI

* Fix clippy

* Use unit test instead of integration test

* Reset CI config

* Simplify tests

* Update crate_universe/src/metadata/dependency.rs

---------

Co-authored-by: UebelAndre <github@uebelandre.com>
burmudar pushed a commit to sourcegraph/pr-auditor that referenced this pull request Jul 18, 2023
…9698)

This PR updates `rules_rust` to `v0.19.0`, as we're using Rust `1.68.0`
which is only supported in that version. I suspect that's what caused
the strange errors when repinning.

Quoting @tjdevries explanation on the fix: 

> Pinned explicitly with features that match the features
> required by rocket. Once bazel rules correctly roll up all the
features,
> we can remove this, but until then, this works just fine for building
> with bazel (and we rarely update rocket, so this is fine).

The whole issue are coming from:
bazelbuild/rules_rust#1884 and
bazelbuild/rules_rust#1885 which are addressing
the problem upstream.

---

Still, this now circles back to a compilation error what we've
previously seen. To reproduce it

```
bazel build //docker-images/syntax-highlighter/...
```

Which in turn leads to the following errors: 

```
ERROR: /private/var/tmp/_bazel_tech/3eea80c6015362974b7d423d1f30cb62/external/crate_index__futures-util-0.3.21/BUILD.bazel:22:13: Compiling Rust rlib futures_util v0.3.21 (180 files) [for tool] failed: (Exit 1): process_wrapper failed: error executing command (from target @crate_index__futures-util-0.3.21//:futures_util) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --env-file ... (remaining 82 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
error[E0432]: unresolved imports `futures_task::waker`, `futures_task::ArcWake`
  --> external/crate_index__futures-util-0.3.21/src/stream/stream/flatten_unordered.rs:21:20
   |
21 | use futures_task::{waker, ArcWake};
   |                    ^^^^^  ^^^^^^^ no `ArcWake` in the root
   |                    |
   |                    no `waker` in the root
   |                    help: a similar name exists in the module (notice the capitalization): `Waker`

error[E0432]: unresolved import `futures_task::ArcWake`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:23:9
   |
23 | pub use futures_task::ArcWake;
   |         ^^^^^^^^^^^^^^^^^^^^^ no `ArcWake` in the root

error[E0432]: unresolved import `futures_task::waker`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:27:9
   |
27 | pub use futures_task::waker;
   |         ^^^^^^^^^^^^^^-----
   |         |             |
   |         |             help: a similar name exists in the module (notice the capitalization): `Waker`
   |         no `waker` in the root

error[E0432]: unresolved imports `futures_task::waker_ref`, `futures_task::WakerRef`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:31:24
   |
31 | pub use futures_task::{waker_ref, WakerRef};
   |                        ^^^^^^^^^  ^^^^^^^^ no `WakerRef` in the root
   |                        |
   |                        no `waker_ref` in the root

error: aborting due to 4 previous errors

For more information about this error, try `rustc --explain E0432`.
ERROR: /private/var/tmp/_bazel_tech/3eea80c6015362974b7d423d1f30cb62/external/crate_index__futures-util-0.3.21/BUILD.bazel:22:13: Compiling Rust rlib futures_util v0.3.21 (180 files) failed: (Exit 1): process_wrapper failed: error executing command (from target @crate_index__futures-util-0.3.21//:futures_util) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --env-file ... (remaining 82 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
error[E0432]: unresolved imports `futures_task::waker`, `futures_task::ArcWake`
  --> external/crate_index__futures-util-0.3.21/src/stream/stream/flatten_unordered.rs:21:20
   |
21 | use futures_task::{waker, ArcWake};
   |                    ^^^^^  ^^^^^^^ no `ArcWake` in the root
   |                    |
   |                    no `waker` in the root
   |                    help: a similar name exists in the module (notice the capitalization): `Waker`

error[E0432]: unresolved import `futures_task::ArcWake`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:23:9
   |
23 | pub use futures_task::ArcWake;
   |         ^^^^^^^^^^^^^^^^^^^^^ no `ArcWake` in the root

error[E0432]: unresolved import `futures_task::waker`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:27:9
   |
27 | pub use futures_task::waker;
   |         ^^^^^^^^^^^^^^-----
   |         |             |
   |         |             help: a similar name exists in the module (notice the capitalization): `Waker`
   |         no `waker` in the root

error[E0432]: unresolved imports `futures_task::waker_ref`, `futures_task::WakerRef`
  --> external/crate_index__futures-util-0.3.21/src/task/mod.rs:31:24
   |
31 | pub use futures_task::{waker_ref, WakerRef};
   |                        ^^^^^^^^^  ^^^^^^^^ no `WakerRef` in the root
   |                        |
   |                        no `waker_ref` in the root

error: aborting due to 4 previous errors
```

## Test plan

<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->

Green CI. 

## App preview:

- [Web](https://sg-web-jh-bzl-fix-rust.onrender.com/search)

Check out the [client app preview
documentation](https://docs.sourcegraph.com/dev/how-to/client_pr_previews)
to learn more.

---------

Co-authored-by: TJ DeVries <devries.timothyj@gmail.com>
Co-authored-by: Jason Bedard <jason@aspect.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants