Definition of @six conflicts between google/protobuf and rules_docker in same bazel workspace

[kellycampbell]

I'm trying to use rules_docker with a project that also uses python protobufs rules from github.com/google/protobuf.

In order to have correct //external:six target used by https://github.com/google/protobuf/blob/master/BUILD#L715 I have this in my WORKSPACE:

new_http_archive(
    name = "six_archive",
    build_file = "@com_google_protobuf//:six.BUILD",
    sha256 = "105f8d68616f8248e24bf0e9372ef04d3cc10104f1980f54d57b2ce73a5ad56a",
    url = "https://pypi.python.org/packages/source/s/six/six-1.10.0.tar.gz#md5=34eed507548117b2ab523ab14b2f8b55",
)

bind(
    name = "six",
    actual = "@six_archive//:six",
)

However, this is not compatible with containerregistry and fails with this error:

external/containerregistry/BUILD.bazel:16:1: no such package '@six//': The repository could not be resolved and referenced by '@containerregistry//:containerregistry'

I haven't been able to find a way for these two to coexist. Some related issues:

• Allow Ignoring of pre-loaded packages. #55
• No such target '//external:six': target 'six' not declared in package 'external' protocolbuffers/protobuf#3677
• Consider removing bind() bazel#1952

[kellycampbell]

I figured out a workaround:

Make a local copy of the google/protobuf/BUILD file from the same commit as you're importing, and modify the //external:six to @six//:six, then create it as a new_http_archive to override the BUILD file:

new_http_archive(
    name = "com_google_protobuf",
    url = "https://github.com/google/protobuf/archive/v3.5.2.zip",
    strip_prefix = "protobuf-3.5.2",
    build_file = "tools/googleprotobuf.bzl" # Path to BUILD file copy with @six//:six
)

[promiseofcake]

Same issue when attempting to build containers in a repository that has a tensorflow dependency (which is also providing that bind for com_google_protobuf).

[erain]

I think this issue is more on bazel's dependency management side rather than this repo.

Maybe open / reference an issue with bazel?

[mikedanese]

This prevents container_image from working with repos that depend on envoy's build as well. cc @mattmoor

[mattmoor]

@erain Yes, but fixes to Bazel flow like molasses. @dekkagaijin can you work with @erain to change the name of the six dependency used by google/containerregistry?

[erain]

Will look into this after fixing the CIs.

[dekkagaijin]

@mattmoor yeah I can help manage it. We're migrating google/containerregistry to py3 using six, so this might just be part of me directing the overall work. I'm also on vacation till the 18th, so we'll probably still be rate limited by the molasses.

[nlopezgi]

Hi @dekkagaijin , do you want to try to send a PR to google/containerregistry to fix your issue and let us know if you need help on our side?

[wmatthews-google]

This is a really awful hack "workaround", but it works...
It simply replaces "six" with "six_hacked" in both containerregistry and rules_docker using the skylark http_archive "patch_cmds" magic.

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

# depending on your version of sed, you need either -i '' or just -i.
SED_CMD = "sed -i '' " + " ".join(["-e '%s'" % e for e in [
    's~name = "six"~name = "six_hacked"~',
    's~"@six//:six"~"@six_hacked//:six_hacked"~',
    's~\"@six\"~\"@six_hacked\"~',
    's~if "six" not in excludes~if "six_hacked" not in excludes~',]])

http_archive(
    name = "containerregistry",
    url = "https://github.com/google/containerregistry/archive/v0.0.25.tar.gz",
    sha256 = "64886684c60bb6f19f73b0e802cdbbf051d10c3803cd11e9ab06d7da8d011ce4",
    strip_prefix = "containerregistry-0.0.25",
    patch_cmds = [SED_CMD + " def.bzl BUILD.bazel"],
)

http_archive(
    name = "io_bazel_rules_docker",
    sha256 = "6dede2c65ce86289969b907f343a1382d33c14fbce5e30dd17bb59bb55bb6593",
    strip_prefix = "rules_docker-0.4.0",
    urls = ["https://github.com/bazelbuild/rules_docker/archive/v0.4.0.tar.gz"],
    patch_cmds = [SED_CMD+ " container/container.bzl"],
)

# ....
# in my case:

load(
    "@io_bazel_rules_docker//container:container.bzl",
    "container_pull",
    _container_repositories = "repositories",
)

_container_repositories()

load(
    "@io_bazel_rules_docker//cc:image.bzl",
    _cc_image_repos = "repositories",
)

_cc_image_repos()

[nlopezgi]

is this still an issue? please reopen if so.

[rodrigoaliste]

I still see this when using protobuf and other google packages such as a google-cloud.

[nlopezgi]

Do these issues occur with rules_docker, or just protobuf with google-cloud. If the latter, there's probably little we can do in this repo to fix, if the former, can you provide repro instructions?

[emranbm]

Any issue for fixing this on Bazel? I'd like to follow that.
@erain
@dekkagaijin

[nlopezgi]

The reference to @ six in this repo has been moved out of repositories.bzl and into WORKSPACE (meaning its currently a test only dependency). You should not be seeing this error coming from an incompatibility with rules_docker anymore. Not sure if there are any upstream issues related to fixing this issue for protobuf or other repos that might also be failing similarly, but not something we are keeping an eye out for in this repo.

[emranbm]

@nlopezgi You're right. I just updated to v0.12.1 and the problem got hidden. Thanks!