Change where bazelrcs are read from, and make it easier to control which ones are read. #4502
Comments
iirina
added
type: feature request
P2
|
Kubernetes is another project which would benefit from something like this. We currently have a We don't have a |
helenalt
added
P1
|
(assigning to @cvcal since she worked with option parsing more recently than both @lfpino and @iirina) I think this is a good idea; Migration will also need to be given some thought; we could either just change in a Bazel release since projects can simply symlink one location to the other to support building with Bazels both before and after the flag flip or have a release that supports both, then remove the old location in the subsequent release. I think the first plan is simpler and the migration costs are not too onerous. WDY'allT? |
Seems like the reasoning is that a lot of projects already have the Is it possible to specify in the |
|
Specifying it in the |
Yes, and specifically that |
|
I actually was just talking to Luis and Klaus about this (context b/70843852 & b/36168162, for Bazel team members). On the question of which files should be in that list, Klaus had written a proposal for what the bazelrc world should look like a few years ago, hidden in https://www.bazel.build/designs/2016/06/21/environment.html. It doesn't contain exactly this suggestion, but might be a good starting point. I think it would be great if we can work something that is similar to the standards in other open source projects, but I don't know much about those standards so I can't speak to the WORKSPACE/bazel directory in particular. |
|
The current workaround: If this is the best place for you to keep workspace-wide options, the rc file next to the bazel binary could import %workspace%/bazel/bazel.rc. |
|
Excellent points, Chloe. After collecting the info below, I have realized it's even more complicated than I thought. I've listed in decreasing precedence and only summarized the unix behaviour; some of the source links describe behaviour on other platforms. (any mistakes my own) maven (source)
(note: someone patched the maven script to support per-project settings) gradle (sources 1 and 2)
npm (source)
pip (source)
gem (source)
cargo (source)
(this seems too clever for its own good...) current Bazel(I'm not sure about order of precedence here)
Bazel: Klaus' proposal
|
|
Thanks for the investigation! It's a complex world out there... My understanding of Klaus's proposal, which might very well be tainted with my own opinions, so might not mirror the document exactly, is closer to the following:
|
|
@aehlig, since we're talking about your design doc 😄 |
Yep. Having seen that we currently have
We have found it useful for using |
bazel.rc is the release candidate, it's a binary (or at least, it is internally, I'm pretty sure for Bazel too). I agree the naming's a mess :) but I think this particular thing isn't going to change. For config files, I think the file extension should be enough, .bazelrc is as descriptive as bazel.bazelrc, but I think the full filename is only checked in the next-to-binary case, and that's largely due to the way we do things internally, so no opinion there. Also agreed on the bikeshedding, though if we're going to make a change in this realm, file & directory naming is a big part of that, so some amount of care is justified.
Above you say you'd be fine with $WORKSPACE/.bazelrc instead of using the intermediate directory "bazel" or "tools", is this a contradiction of that? More specific rc files can easily have imports to common files that are shared between projects/users/clones, whichever. It's the other direction that's hard, so I'd rather err on the side of making these locations too specific, within reason. |
|
I think this is the list we're working from:
We could then have different trigger flags for these. imperfectly, For an example of the type of configuration that's possible with the above without necessarily needing a file that configures for the intersection of user and project. The user's directory can easily have <in |
|
@sergiocampama brings up the ordering problem, since more specific options should override competing options from a more general source. The three "master" rcs above, binary, workspace and global, don't have an obvious ranking. Maybe global < binary < workspace, but there are some cases where that wouldn't work. |
|
IMO the next-to-binary rc should be provided by the bazel team in the distinct packages for release, like homebrew (macOS), chocolatey (Windows) and apt-get (linux). Given that this is the set of options coming from the bazel developers, it should be the most general one. /etc/bazel.bazelrc could be set by system admins for specific group of developers, so it could be regarded as a bit more specific than the binary one. Don't know what the windows version could like like for that. The $WORKSPACE one could be set by team leads for a project, so it's more specific that the /etc/bazel.bazelrc Developers sets their own in $HOME which is more specific. Finally the --blazerc gets loaded since it's per invocation, the most specific one. WDYT? |
|
You could also divide this into 2 groups, master (binary, /etc) and user (workspace, home), and have 2 flags to disable these --nomaster_bazelrc and --nouser_bazelrc. Or similar |
|
Hardcoding /etc/bazel.bazelrc is unfortunate for testing reasons. We currently have no test coverage that it works, because any such test would require writing to /etc/bazel.bazelrc which is outside of the test sandbox. It would be nice if this location were configurable. |
|
To avoid having too many issues discussing related things, I wanted to make you
all aware of #4629. Apparently, not in all cases, all rc-files are read. This causes
problems if on the one hand, the project has to specify project-specific flags,
on the other hands, the layout of the user's home directory has to be specified;
see, e.g., #3516 (in particular, #3516 (comment)).
This lead to all kind of work-arounds being suggested, e.g., #2054.
…--
Klaus Aehlig
Google Germany GmbH, Erika-Mann-Str. 33, 80636 Muenchen
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschaeftsfuehrer: Paul Terence Manicle, Halimah DeLaine Prado
|
cvcal
changed the title
Since: [1] Bazel changed the list of resource file locations. Adapt to that list and move `%workspace%/tools/bazelrc` to `%workspace%/.bazelrc`. See: [2], [3] for more details. [1] bazelbuild/bazel@ec83598 [2] bazelbuild/bazel#4502 [3] bazelbuild/bazel#5756 Change-Id: Ia13c71bcff7036e2f5541618b28713d9b0d77040
*** Reason for rollback *** Will add try-import before rolling this out, it won't make the 0.17 cut The rc list is once again: - %workspace%/tools/bazel.rc (unless --nomaster_bazelrc) - %binary_dir%/bazel.bazelrc (unless --nomaster_bazelrc) - system rc, /etc/bazel.bazelrc or in %ProgramData% for Windows (unless --nomaster_bazelrc) - the first of the following gets called the "user" bazelrc - path passed by flag --bazelrc - %workspace%/.bazelrc - $HOME/.bazelrc Reopen #4502 *** Original change description *** Change the list of rc files accepted. The new list is hopefully a bit more consistent, as: - system rc (unless --nosystem_rc) - workspace, %workspace%/.bazelrc (unless --noworkspace_rc) - user, $HOME/.bazelrc (unless --nohome_rc) - command-line provided, passed as --bazelrc or nothing if the flag is absent. This list removes two less than useful locations, duplication in the Workspace directory, and the rc next to the bazel binary. This location made sense at Google but is generally nonsensical elsewhere so we are removing it. It also stops the user local rc file from being overriden by passing in a custom file in --bazelrc. In both old and new, --ignore_all_rc_files disables all of the above. For a transition period, any file that you would have loaded but was not read will cause a WARNING to be printed. If you want the old file to still be read without moving its location, you can always import it into one of the new standard locations, or create a symlink. Closes #4502, except for cleanup to remove the warning after a transition period of 1 Bazel version has passed. RELNOTES[INC]: New bazelrc file list. PiperOrigin-RevId: 209797467
|
Won't be making it in to 0.17, will roll forward to 0.18 once the release is cut |
Bazel 0.17.0 will contain a change for which rc files it accepts. bazelbuild/bazel@ec83598 bazelbuild/bazel#4502 Old bazel used to read %workspace%/tools/bazel.rc. New bazel will not read that and instead will only read %workspace%/.bazelrc. Signed-off-by: Jason Zaman <jason@perfinion.com>
Bazel 0.17.0 will contain a change for which rc files it accepts. bazelbuild/bazel@ec83598 bazelbuild/bazel#4502 Old bazel used to read %workspace%/tools/bazel.rc. New bazel will not read that and instead will only read %workspace%/.bazelrc. Signed-off-by: Jason Zaman <jason@perfinion.com>
Bazel 0.17.0 will contain a change for which rc files it accepts. bazelbuild/bazel@ec83598 bazelbuild/bazel#4502 Old bazel used to read %workspace%/tools/bazel.rc. New bazel will not read that and instead will only read %workspace%/.bazelrc. Signed-off-by: Jason Zaman <jason@perfinion.com>
Bazel 0.18.0 will contain a change for which rc files it accepts. bazelbuild/bazel@ec83598 bazelbuild/bazel#4502 Old bazel used to read %workspace%/tools/bazel.rc. New bazel will not read that and instead will only read %workspace%/.bazelrc. Signed-off-by: Jason Zaman <jason@perfinion.com>
|
@cvcal shouldn't this commit with Also- the ignore_all_rc_files flag is equivalent to putting |
|
@cvcal so apparently nohome_rc is also only available from 0.17.1 :( |
|
Yes, |
|
ended up bumping to 0.15.2 as minimal version and using ignore_all_rc_files |
|
15 is the first version with the flag? That sounds right |
|
Thanks for confirming.
Still I think that if the flag exists in the docs it should have release
notes. My 2c
…On Thu, 20 Sep 2018 at 20:16 Chloe Calvarin ***@***.***> wrote:
15 is the first version with the flag? That sounds right
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#4502 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABUIF-4kIPLgQXXTtmGXD5fNeM7xL_soks5uc81-gaJpZM4Rn6ig>
.
|
Bazel 0.18.0 will contain a change for which rc files it accepts. bazelbuild/bazel@ec83598 bazelbuild/bazel#4502 Old bazel used to read %workspace%/tools/bazel.rc. New bazel will not read that and instead will only read %workspace%/.bazelrc. Signed-off-by: Jason Zaman <jason@perfinion.com>
|
Using `$HOME/.bazelrc' will clutter the home-dir. Shouldn't this be $HOME/.config/.bazelrc ? Is this supported? For details check out https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html |
|
This is not supported - we need a place that can make sense in different platforms and setups, and I'm not sure that .config is a standard location. If you care about backups in .config/, you should be able to create a thin bazelrc in HOME that imports the one in .config, which gets backed up, but that doesn't solve the clutter issue. |
|
He seriously, you are compliant with the basedir spec for other files. Eg the cached resource are under ~/.cache/bazel/bazel_$USER/. The standard I pointed to is a quite established solution. |
|
If it really is the standard as you say, then I don't know why it didn't come up earlier in the discussion about this change, we looked for standard config file locations earlier in this thread (see #4502 (comment)). Maybe we were wrong, but no one pointed it out, and so we stuck with the file location that a number of our users were already used to using. I don't believe there's much appetite to change this list a second time, makes the release break a lot of people. |
|
I definitely agree with no more changes to the list at this point. Also basically everything I've seen that supports |
Leave functions that make file accesses in the file library, and general blaze utilities in the blaze_util file, but move the functions that boil down to string manipulation and path formatting to their own file. (With the exception of getCWD, since absolute path syntax is relevant here.) Doing this largely to consolidate all Windows path control into a single place, so that it's easier to notice inconsistencies. For instance, ConvertPath currently makes Windows paths absolute, but not Posix paths, and MakeAbsolute relies on this behavior. In addition, JoinPath assumes Posix path syntax, which leads to some odd looking paths. These will be fixed in a followup change. (Found these issues while working on bazelbuild#4502, trying to fix the windows-specific system bazelrc.) RELNOTES: None. PiperOrigin-RevId: 199368226
For paths passed to Bazel on the command line, the shell expands these variables, but for hardcoded defaults, we must make the library call ourselves. We do not add similar support in Posix systems, where it is less common to rely on standard path-related variables. Prerequisit for issue bazelbuild#4502. RELNOTES: None. PiperOrigin-RevId: 201183214
Bazel 0.18.0 will contain a change for which rc files it accepts. bazelbuild/bazel@ec83598 bazelbuild/bazel#4502 Old bazel used to read %workspace%/tools/bazel.rc. New bazel will not read that and instead will only read %workspace%/.bazelrc. Signed-off-by: Jason Zaman <jason@perfinion.com>
FWIW the above tools mostly allow the user's bazel config to easily be moved to # Cargo, mixes data and global config
export CARGO_HOME="$XDG_DATA_HOME/cargo"
# Npm allows customising
export npm_config_userconfig="$XDG_CONFIG_HOME/npm/config"
# Gradle puts the cache and config together.
export GRADLE_USER_HOME="$XDG_CACHE_HOME/gradle"
# Pip supports $XDG_CONFIG_HOME by default: ~/.config/pip/pip.conf
# maven and gem can't use the XDG spec.Lots of other tools like git also have support for this, see the Arch Wiki Page for a list.
That's reasonable, but is there a way to change this in a semver-minor fashion? Ideally by adding a couple of options at the top of the per-user list, e.g.
Alternatively by keeping the current defaults but also reading an env var, allowing users to do Right now the best workaround I've found for this is |
Description of the feature request:
Check for
bazel/bazel.rcin addition totools/bazel.rc. This could also be done fortools/bazeland any other special files that can be intools/.Feature requests: what underlying problem are you trying to solve with this feature?
Projects that support multiple build systems prefer to keep the files separated as far as possible. However, Bazel forces you to add a
tools/top-level directory to use certain functionality. See this comment from cartographer-project/cartographer#834 which prompted this issue.The
bazel/subdirectory is fairly common for builds that support multiple systems, eg gRPC, glog, Google Cartographer, Ceres Solver. Exceptions (ie projects with nobazel/subdir) include TensorFlow and gRPC Java, although IMHO the latter is good example of where it would help clean things up.What operating system are you running Bazel on?
Ubuntu 14.04
What's the output of
bazel info release?release 0.9.0
The text was updated successfully, but these errors were encountered: