Bazel CI: rules_rust is failing in downstream

[meteorcloudy]

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/813#96514f68-0f66-4b73-9e5b-c53f2eb23ec8

Bisecting result points to 62c7aa4
https://buildkite.com/bazel/culprit-finder/builds/99#88fe4bc6-d36d-4f03-9196-90d0b084f0d2

@cushon

FYI @hlopko

[meteorcloudy]

@cushon Should we rollback this or fix from rules_rust?

[cushon]

@meteorcloudy how do I reproduce the failure?

[meteorcloudy]

I think you can build Bazel from HEAD, they try to build rules_rust by
bazel_head build @docs//:rust-docs

More detailed command is here: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/813#96514f68-0f66-4b73-9e5b-c53f2eb23ec8

[cushon]

@c-parsons

It looks like Bazel at head is being used with the //tools/jdk package from a previous release loaded as the io_bazel workspace by skydoc: https://github.com/bazelbuild/skydoc/blob/0a008878383c51c41db24b313980de19996a6438/setup.bzl#L66

If that's accurate, why is skydoc doing that? Shouldn't it be using the version from the current release?

[c-parsons]

I'm confused by your question, sorry.

You've linked to skydoc's binding of @io_bazel at current HEAD, but rust depends on skydoc at an older version. (Specifically, skydoc at that commit seems to depend on io_bazel at e7ebb7e)

If you need a newer version of a repository that's a transitive dependency of skydoc, you should be able to either resolve the diamond dependency explicitly yourself, or depend on a version of skydoc that happens to have its dependency versions compatible with whatever else you need.

[cushon]

Right, but I think that still explains the issue.

Bazel at head removes the native java_host_runtime_alias and atomically replaced the canonical use of that target in tools/jdk with a starlark implementation, so loading old version of tools/jdk with Bazel at head no longer works.

I'm not sure the thing that stardoc is doing here should be supported.

If you need a newer version of a repository that's a transitive dependency of skydoc, you should be able to either resolve the diamond dependency explicitly yourself

How can rules_rust override the version of io_bazel to match the version of bazel in the current build?

[c-parsons]

What is specifically loading the old version of tools/jdk with Bazel at HEAD?
I still don't understand fully what's going on here, so if you could explain in detail what you think is occurring, that would be helpful. So far, it sounds like you have a diamond dependency, specifically two ways to depend on io_bazel, and they have recently become incompatible. The way this is generally resolved is to select a specific version of the conflicting dependency (in this case io_bazel) which is compatible with both of its dependers.

I'm not sure the thing that stardoc is doing here should be supported.

What is "the thing" in this sentence, specifically?

How can rules_rust override the version of io_bazel to match the version of bazel in the current build?

By defining io_bazel themselves in their workspace file prior to the io_bazel_skydoc definition.

[cushon]

What is specifically loading the old version of tools/jdk with Bazel at HEAD?

bazel query -k 'somepath(@docs//:rust-docs, @io_bazel//tools/jdk:*)'
...
@docs//:rust-docs
@io_bazel//src/main/java/com/google/devtools/build/skydoc:skydoc
@io_bazel//src/main/java/com/google/devtools/build/skydoc:skydoc_lib
@io_bazel//src/main/java/com/google/devtools/build/skydoc/fakebuildapi/cpp:cpp
@io_bazel//src/main/java/com/google/devtools/build/lib/actions:actions
@io_bazel//src/main/java/com/google/devtools/build/lib:unix
@io_bazel//src/main/native:libunix.so
@io_bazel//src/main/native:jni_md.h
@io_bazel//src/main/native:copy_link_jni_md_header
@io_bazel//tools/jdk:jni_md_header-windows
@io_bazel//tools/jdk:include/win32/jni_md.h

So far, it sounds like you have a diamond dependency, specifically two ways to depend on io_bazel, and they have recently become incompatible.

It's not really a diamond dependency, just that Bazel at head isn't compatible with old versions of //tools/jdk.

What is "the thing" in this sentence, specifically?

Accessing stuff that's in @bazel_tools by create a separate workspace containing the entire Bazel tree seems like a bad idea. The canonical supported instance of the java_host_runtime_alias rule is in @bazel_tools.

[cushon]

By defining io_bazel themselves in their workspace file prior to the io_bazel_skydoc definition.

Thanks! I opened a PR with a work-around: bazelbuild/rules_rust#195