incompatible_merge_genfiles_directory: unify bazel-bin and bazel-genfiles directories #6761

laurentlb · 2018-11-26T13:09:37Z

We plan to unify the bazel-bin and bazel-genfiles directories.

This means that bazel-genfiles will point to the same location as bazel-bin, since the distinction was not meaningful. This will allow us to make simplifications in Bazel and its API.

See discussion: https://groups.google.com/d/msg/bazel-dev/5iq7n1soenA/xGM_M1NOCAAJ

The text was updated successfully, but these errors were encountered:

…atible_merge_genfiles_directory` #6761 RELNOTES: New incompatible flag --incompatible_merge_genfiles_directory PiperOrigin-RevId: 222826123

laurentlb · 2019-01-08T16:29:42Z

One downstream project fails with--incompatible_merge_genfiles_directory: protobuf

https://buildkite.com/bazel/bazel-at-release-plus-incompatible-flags/builds/56#70fd5567-2b59-4012-be62-1b082da936b0

laurentlb · 2019-02-21T19:14:57Z

Still blocked by Protobuf (protocolbuffers/protobuf#5691).

meteorcloudy · 2019-03-01T14:29:45Z

@laurentlb FYI, Bazelisk + Incompatible flags couldn't catch this breakage. Because building with this flag seems have an impact on subsequent builds.
See https://buildkite.com/bazel/bazelisk-plus-incompatible-flags/builds/18#d6d93bbf-ca8a-432f-bd99-23d93577ede5
Unless we do a bazel clean between the incompatible flags and no incompatible flags build, we still get the same error message.

For historical reasons, Bazel has two directories for generated files, although a future version of Bazel will remove the genfiles directory: bazelbuild/bazel#6761 Currently, rosmsg() copies all the source files (*.msg, *.srv) to the output directory, so that the directory structure matches the needs of ROS's codegen tools. It uses a hacky way of finding the paths to the transitive dependencies within the output directory, which includes a direct reference to genfiles. By using output_to_bindir=True, we can avoid the direct reference. Fixing the hack would involve rewriting the rosmsg() macro to use a rule, which wouldn't be too hard but might break compatibility. See #2. Change-Id: Ibae3451e1329e397329089aa6796c5477789c788 GitOrigin-RevId: e38ef5d

jin · 2019-03-27T20:14:49Z

This will cause all of these scripts / code that assume existence of bazel-genfiles to stop working, right? https://github.com/search?q=bazel-genfiles&type=Code

We can catch possible breakages due to this change using Buildkite iff the bazel-genfiles path is referenced within the build. However, many of the references are from configuration files and external scripts. How we can improve our imcompatible change infra to work with this kind of changes?

I'm just wondering about the user experience of getting this in the next release, if some part of the bazel-external infrastructure breaks because the genfiles dir no longer exists, and I had no warning about it when using previous Bazel versions interactively.

laurentlb · 2019-03-27T20:28:15Z

When testing downstream projects, we noticed very few issues. But it's true that it tests only within Bazel. Scripts that call Bazel and then look at the specific directory might break.

Not sure what we can do, apart from advertising the change more loudly.
(at least, it shows why this should be done before Bazel 1.0)

laurentlb · 2019-03-27T20:32:17Z

Projects that rely on bazel-genfiles can:

Either specify the flag --incompatible_merge_genfiles_directory=false. This flag will be available for a long time (even after Bazel 1.0), to give users more time to migrate.
Or create a symlink: ln -s bazel-bin bazel-genfiles

jin · 2019-03-27T20:41:32Z

When testing downstream projects, we noticed very few issues. But it's true that it tests only within Bazel. Scripts that call Bazel and then look at the specific directory might break.

The symlinks are, in a way, the only user facing filesystem level API that Bazel exposes. I also have no idea what else we can do. The Downstream pipeline has no visibility into the bazel-external build infra of our users :(

advertising the change more loudly. (at least, it shows why this should be done before Bazel 1.0)

Completely agree.

…ed by ClangImporter. genfiles_dir is already searched but is deprecated (bazelbuild/bazel#6761) and the C rules have been adding bin_dir to their quote include paths since June 2018, so we should catch up. RELNOTES: Swift modules that import C modules can now find headers located in the bazel-bin directory. PiperOrigin-RevId: 264958475

…ed by ClangImporter. genfiles_dir is already searched but is deprecated (bazelbuild/bazel#6761) and the C rules have been adding bin_dir to their quote include paths since June 2018, so we should catch up. RELNOTES: Swift modules that import C modules can now find headers located in the bazel-bin directory. PiperOrigin-RevId: 265051482

Check both "bazel-genfiles" and "bazel-bin" due bazelbuild/bazel#6761

…zel-bin/' instead of 'bazel-genfiles/' These directories are being unified (bazelbuild/bazel#6761), so the distinction is not meaningful, except that in earlier versions of Bazel (e.g. 0.24ish) only bazel-bin is written to. PiperOrigin-RevId: 282721127 Change-Id: I4028f0bb7ed93314c15fb3761762a30c5d647b42

bazel-bin and bazel-genfiles directories are unified. ref1) bazelbuild/bazel#6761 ref2) https://www.lijiaocn.com/%E9%A1%B9%E7%9B%AE/2018/12/17/envoy-02-ide.html

This was flipped over 3 years ago bazelbuild#6761

…ed by ClangImporter. genfiles_dir is already searched but is deprecated (bazelbuild/bazel#6761) and the C rules have been adding bin_dir to their quote include paths since June 2018, so we should catch up. RELNOTES: Swift modules that import C modules can now find headers located in the bazel-bin directory. PiperOrigin-RevId: 265051482

This makes path computation code shorter and more robust. In particular, it makes things work under Bazel flag --incompatible_merge_genfiles_directory (bazelbuild/bazel#6761).

laurentlb added P1 I'll work on this now. (Assignee required) team-Starlark incompatible-change Incompatible/breaking change labels Nov 26, 2018

bazel-io pushed a commit that referenced this issue Nov 26, 2018

Rename flag --experimental_separate_genfiles_directory to `--incomp…

e52ea66

…atible_merge_genfiles_directory` #6761 RELNOTES: New incompatible flag --incompatible_merge_genfiles_directory PiperOrigin-RevId: 222826123

kastiglione mentioned this issue Dec 6, 2018

Transition from bazel-genfiles to bazel-bin bazel-xcode/xchammer#136

Open

laurentlb added the migration-0.22 label Jan 7, 2019

laurentlb changed the title ~~--incompatible_merge_genfiles_directory~~ incompatible_merge_genfiles_directory: unify bazel-bin and bazel-genfiles directories Jan 7, 2019

laurentlb added the migration-0.21 label Jan 8, 2019

philwo added the migration-0.23 label Feb 6, 2019

bazel-io closed this as completed in cce2ab1 Mar 25, 2019

laurentlb added the breaking-change-0.25 label Mar 28, 2019

laurentlb mentioned this issue Apr 2, 2019

not possible to declare_file to genfiles_dir #4858

Closed

benjaminp mentioned this issue Apr 9, 2019

Explicitly pass the genfiles directory to cc_op_gen. tensorflow/tensorflow#27666

Closed

lizan mentioned this issue Jun 14, 2019

ci: configure CircleCI for Envoy-WASM. envoyproxy/envoy-wasm#53

Merged

laurentlb mentioned this issue Jun 17, 2019

incompatible_skip_genfiles_symlink: Remove bazel-genfiles symlink #8651

Closed

benjaminp mentioned this issue Jun 22, 2019

bazel0.26.0: genrule outputs files in both gendir and bindir irrespective of the value of output_to_bindir #8698

Closed

swiple-rules-gardener mentioned this issue Aug 23, 2019

Add Bazel's -bin output directory to the quote include paths searched by ClangImporter. bazelbuild/rules_swift#292

Merged

meisterT mentioned this issue Oct 2, 2019

add declare_file with sibling as ArtifactRoot #4859

Closed

serathius added a commit to serathius/kubernetes that referenced this issue Oct 25, 2019

Handle bazel >= 0.25 in stable metric static analysis

4c7c581

Check both "bazel-genfiles" and "bazel-bin" due bazelbuild/bazel#6761

serathius mentioned this issue Oct 25, 2019

Handle bazel >= 0.25 in stable metric static analysis kubernetes/kubernetes#84378

Merged

crorvick mentioned this issue Nov 18, 2019

genrule outputs in bazel-bin instead of bazel-genfiles #10256

Closed

meteorcloudy mentioned this issue Mar 9, 2021

Prevent --repo_env from triggering unnecessary fetches #13126

Closed

regisd mentioned this issue May 31, 2021

bazel-genfiles location cjweeks/tensorflow-cmake#52

Open

keith added a commit to keith/bazel that referenced this issue Jul 19, 2022

Remove --incompatible_merge_genfiles_directory

ed34ef3

This was flipped over 3 years ago bazelbuild#6761

keith mentioned this issue Jul 19, 2022

Remove --incompatible_merge_genfiles_directory #15918

Closed

keith added a commit to keith/bazel that referenced this issue Jul 20, 2022

Remove --incompatible_merge_genfiles_directory

0ce1da5

This was flipped over 3 years ago bazelbuild#6761

keith added a commit to keith/bazel that referenced this issue Oct 11, 2022

Remove --incompatible_merge_genfiles_directory

3e952a5

This was flipped over 3 years ago bazelbuild#6761

keith added a commit to keith/bazel that referenced this issue Oct 11, 2022

Remove --incompatible_merge_genfiles_directory

5a76818

This was flipped over 3 years ago bazelbuild#6761

keith added a commit to keith/bazel that referenced this issue Oct 11, 2022

Remove --incompatible_merge_genfiles_directory

67055a2

This was flipped over 3 years ago bazelbuild#6761

keith added a commit to keith/bazel that referenced this issue Oct 11, 2022

Remove --incompatible_merge_genfiles_directory

e5eae38

This was flipped over 3 years ago bazelbuild#6761

NaveenBoggarapu mentioned this issue Aug 9, 2023

Documentation issue: Output directory layout #19206

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

incompatible_merge_genfiles_directory: unify bazel-bin and bazel-genfiles directories #6761

incompatible_merge_genfiles_directory: unify bazel-bin and bazel-genfiles directories #6761

laurentlb commented Nov 26, 2018

laurentlb commented Jan 8, 2019

laurentlb commented Feb 21, 2019

meteorcloudy commented Mar 1, 2019

jin commented Mar 27, 2019 •

edited

Loading

laurentlb commented Mar 27, 2019

laurentlb commented Mar 27, 2019 •

edited

Loading

jin commented Mar 27, 2019

incompatible_merge_genfiles_directory: unify bazel-bin and bazel-genfiles directories #6761

incompatible_merge_genfiles_directory: unify bazel-bin and bazel-genfiles directories #6761

Comments

laurentlb commented Nov 26, 2018

laurentlb commented Jan 8, 2019

laurentlb commented Feb 21, 2019

meteorcloudy commented Mar 1, 2019

jin commented Mar 27, 2019 • edited Loading

laurentlb commented Mar 27, 2019

laurentlb commented Mar 27, 2019 • edited Loading

jin commented Mar 27, 2019

jin commented Mar 27, 2019 •

edited

Loading

laurentlb commented Mar 27, 2019 •

edited

Loading