bazel query can lie about external dependency version

[dmivankov]

Description of the problem / feature request:

bazel query --output build //external:something may print wrong version of something

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

WORKSPACE

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
load("//:a.bzl", "add_skylib")

add_skylib()
http_archive(
    name = "bazel_skylib",
    urls = [
        "https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
    ],
    sha256 = "1c531376ac7e5a180e0237938a2536de0c54d93f5c278634818e0efc952dd56c",
)

a.bzl

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

def add_skylib():
    http_archive(
        name = "bazel_skylib",
        urls = [
            "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
            "https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
        ],
        sha256 = "97e70364e9249702246c0e9444bccdc4b847bed1eb03c5a3ece4f83dfe6abc44",
    )

BUILD: empty file

$ bazel query --output build //external:bazel_skylib
http_archive(
  name = "bazel_skylib",
  urls = ["https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz", "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz"],
  sha256 = "1c531376ac7e5a180e0237938a2536de0c54d93f5c278634818e0efc952dd56c",
)

$ cat "`bazel info output_base`/external/bazel_skylib/version.bzl"
...
version = "1.0.2"

Interestingly adding

load("@bazel_skylib//:version.bzl", "version")

to WORKSPACE makes bazel use 1.0.3. Though there also are cases when having

http_archive(name = "foo" ...)
load("@foo//:something.bzl", "something')

fail because some other dependency manages to override/pre-load foo and bazel fails on not finding something.bzl in non-workspace version (can try to create a bigger repro where bazel_skylib is used in WORKSPACE but version is still mismatching between bazel query and actual external dependency)

What operating system are you running Bazel on?

NixOS

What's the output of bazel info release?

Tried on 3.3.1 and 4.0.0

[Wyverald]

I couldn't repro on macOS with Bazel 4.0.0. This line:

cat "`bazel info output_base`/external/bazel_skylib/version.bzl"

simply reports that the directory does not exist. I suspect this is WAI because just doing a bazel query //external:bazel_skylib doesn't cause bazel_skylib to be fetched, and you're seeing 1.0.2 maybe from an earlier fetch. Could you clean and try again?

[dmivankov]

Hm, so this repro is too small, after clean it goes away, will try to repeat the bigger one then 👍
It was around using gazelle, rules_go, rules_docker and there bazel query was matching http_archive from WORKSPACE, but that wasn't what was used for builds

[dmivankov]

Will close for now, could be that there was some version overriding I was debugging, but looking into .../external/something/ files wasn't robust (needed to force refetch). Will reopen if manage to repro with forced fetch

[Wyverald]

Ah. I think I've got the repro. If you put the following in your WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
	name = "bazel_skylib",
	urls = [
	    "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
	    "https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
	],
	sha256 = "97e70364e9249702246c0e9444bccdc4b847bed1eb03c5a3ece4f83dfe6abc44",
)
load("@bazel_skylib//:version.bzl", "version")
print(version)
http_archive(
    name = "bazel_skylib",
    urls = [
        "https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
    ],
    sha256 = "1c531376ac7e5a180e0237938a2536de0c54d93f5c278634818e0efc952dd56c",
)

Then query would give you 1.0.3, whereas the actual fetched contents are 1.0.2.

This is weirdly WAI (at least the latter part). Later repo definitions override earlier ones. But the repo definition is "solidified" once a fetch is requested. It's definitely not perfect that query gives you the latest repo definition, even when it was "solidified" before that by a fetch.

EDIT: The repo definition is "solidified" by any subsequent load statement, not just a fetch. See comment below.

[dmivankov]

Ow, so commenting out

#load("@bazel_skylib//:version.bzl", "version")
#print(version)

or not commenting it out would change the solidified version..

and here

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
	name = "bazel_skylib",
	urls = [
	    "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
	    "https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
	],
	sha256 = "97e70364e9249702246c0e9444bccdc4b847bed1eb03c5a3ece4f83dfe6abc44",
)
#load("@bazel_skylib//:version.bzl", "version")
http_archive(
    name = "bazel_skylib",
    urls = [
        "https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
    ],
    sha256 = "1c531376ac7e5a180e0237938a2536de0c54d93f5c278634818e0efc952dd56c",
)
load("@bazel_skylib//:version.bzl", "version")
print(version)

printed version depends on whether first load is commented-out or not

I'll reopen because it would be nice if bazel query matched the solidified version (or if bazel cquery or some other flavor would match it)

[dmivankov]

def dbg():
   print(native.existing_rule("bazel_skylib"))

is also printing a wrong version - the last one seen rather the solidified or last seen

[brandjon]

For what it's worth, I was digging through this portion of the codebase the other week and it's implemented roughly as follows:

1. Break the WORKSPACE file into chunks based on where the load() statements appear.
2. Evaluate each chunk as its own entity, first populating it with definitions from the previous chunk, and then adding new ones introduced by this chunk. Any time a repo is defined, it replaces any existing repo of the same name in the current chunk, so that the last definition wins within each chunk.
3. To find the definition for repo foo, scan through all the chunks in order, and take the definition from the first chunk to define foo. This is how you get the "last definition within the first chunk wins" semantics.

Based on that, my guess for the cause of this bug would be that query is using the final state from the last chunk, and not reproducing the behavior of step 3.

These semantics are of course terrible, and I'm not thrilled about the idea of fixing query to match, vs just replacing the whole machinery with something cleaner. (I also had no idea you could do a query over //external, and am unclear on how it's even able to resolve repos using a separate code path from the normal one.)