incompatible_always_include_files_in_data doesn't work well with cc_library

[exoson]

Description of the bug:

When a target using a native binary rule, for example py_binary, depends on a cc_library target as a data dependency, excessive amounts of files get included in the runfiles of the binary target depending on the flag --incompatible_always_include_files_in_data. Without that flag on, only the .so file in the _solib_* directory gets included. With that flag on, in addition to the .so file in the _solib_* directory, the same .so file will get included outside the _solib_* directory and the .a version of the library will get included in the runfiles.

This is especially problematic when packaging the runfiles inside some other target, for example when using the py_image rule from rules_docker. This will at least double the size of the library in the tar file generated by the py_image target.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Setup repository by running the following:

cat > BUILD <<EOF
load("@io_bazel_rules_docker//python:image.bzl", "py_image")

py_image(
  name = "py_image",
  srcs = ["py_image.py"],
  deps = [":py_binary"],
)
py_binary(
  name = "py_binary",
  srcs = ["py_binary.py"],
  data = [":cc_library"],
)
cc_library(
  name = "cc_library",
  srcs = ["cc_library.cc"],
)
EOF
touch py_image.py py_binary.py cc_library.cc
cat > WORKSPACE <<EOF
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "io_bazel_rules_docker",
    sha256 = "b1e80761a8a8243d03ebca8845e9cc1ba6c82ce7c5179ce2b295cd36f7e394bf",
    urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.25.0/rules_docker-v0.25.0.tar.gz"],
)

load(
    "@io_bazel_rules_docker//repositories:repositories.bzl",
    container_repositories = "repositories",
)
container_repositories()

load("@io_bazel_rules_docker//repositories:deps.bzl", container_deps = "deps")

container_deps()

load(
    "@io_bazel_rules_docker//python:image.bzl",
    _py_image_repos = "repositories",
)

_py_image_repos()
EOF

Build the image with and without the flag on and check the contents of the layer. (Removed the non relevant parts)

$ bazel build //:py_image --noincompatible_always_include_files_in_data                                                                                          
...
INFO: Build completed successfully, 1 total action
$ cat bazel-bin/py_image-layer.manifest | jq
{                                                                            
...                             
  "files": [
...
    {
      "dst": "/app//py_image.binary.runfiles/__main__/_solib_k8/libST-4a519fd6d3e4_libcc_Ulibrary.so",
      "src": "bazel-out/k8-fastbuild-ST-4a519fd6d3e4/bin/_solib_k8/libST-4a519fd6d3e4_libcc_Ulibrary.so"
    },                                    
...                                                             
  ],
...

}
$ bazel build //:py_image --incompatible_always_include_files_in_data 
...
INFO: Build completed successfully, 9 total actions
$ cat bazel-bin/py_image-layer.manifest | jq
{
...
  "files": [
...
    {
      "dst": "/app//py_image.binary.runfiles/__main__/libcc_library.a",
      "src": "bazel-out/k8-fastbuild-ST-4a519fd6d3e4/bin/libcc_library.a"
    },
    {
      "dst": "/app//py_image.binary.runfiles/__main__/libcc_library.so",
      "src": "bazel-out/k8-fastbuild-ST-4a519fd6d3e4/bin/libcc_library.so"
    },
    {
      "dst": "/app//py_image.binary.runfiles/__main__/_solib_k8/libST-4a519fd6d3e4_libcc_Ulibrary.so",
      "src": "bazel-out/k8-fastbuild-ST-4a519fd6d3e4/bin/_solib_k8/libST-4a519fd6d3e4_libcc_Ulibrary.so"
    },
...
  ],
...
}

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 6.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

Issue on the flag #16654

Any other information, logs, or outputs that you want to share?

No response

[fmeum]

CC @comius who reviewed the flag flip.

I understand the use case, but I think that *_library targets aren't really meant to be data dependencies of anything. Could you try to use cc_shared_library instead which is specifically meant to produce shared libraries as final build outputs? Alternatively, cc_binary with linkshared = True should also work.

[oquenchil]

I agree with Fabian, please use a cc_shared_library instead.