Builds running with systemd wipe out cache

[jordan-bonecutter]

Description of the bug:

I'm testing running a bazel build in a systemd service. This all runs fine and the build completes as it should. However, the build running as the systemd service seems to have a cache completely separate from the build when I run in a terminal. Not only is it separate but it also seems to wipe out the cache for builds outside of systemd. Vice-versa is the same, building without systemd wipes out systemd caches.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

On a server with a bazel build rule, let's say foo for example, create the following systemd service file:

[Unit]
Description=BazelBuildTest
After=network.target

[Service]
WorkingDirectory=/path/to/bazel-repo
ExecStart=/usr/bin/bazel build //foo
Restart=always
User=<my-user>

[Install]
WantedBy=multi-user.target

Replacing the proper values with your own. Now start this service with sudo systemctl start builder.service. Check the journalctl logs with sudo journalctl -u builder -f to confirm that it's running properly. Notice that it doesn't utilize any cached objects during the build. While it's building run bazel build //foo in another terminal. Once the bazel server is available it will begin building but again without using any cached objects.

Which operating system are you running Bazel on?

NAME="Ubuntu" VERSION="20.04.3 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.3 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal

What is the output of bazel info release?

release 5.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

N/A

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

No

[sgowroji]

Hello @jordan-bonecutter, It would be helpful if you can provide a sample code repo and complete steps to reproduce the above request and verify at our end. Thanks!

[jordan-bonecutter]

As mentioned this will be reproducible in any bazel repo with any build rule, so let's try bazelbuild/bazel itself.

1. Clone the repository with git clone git@github.com:bazelbuild/bazel --depth=1
2. cd bazel
3. Let's build a rule with bazel build bootstrap-jars
4. Build it again, ensure that it's fast this time as it can take advantage of cached objects
5. Now, create a systemd service file like so (call it builder.service):

[Unit]
Description=BazelBuildTest
After=network.target

[Service]
WorkingDirectory=/path/to/bazel
ExecStart=/usr/bin/bazel build //bootstrap-jars
Restart=always
User=<my-user>

[Install]
WantedBy=multi-user.target

7. Copy the builder.service file to /etc/systemd/system/builder.service
8. Run the service and tee its stdout to yours with sudo systemctl start builder && sudo journalctl -u builder -f
9. Observe a fresh build, not reusing any of the cached objects
10. Now stop the service with sudo systemctl stop builder
11. Run the build from your terminal with bazel build bootstrap-jars, confirm that it also doesn't use any cached objects

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[jordan-bonecutter]

@bazelbuild/triage this is still an issue with the newest version of bazel (6.4.0). Can this issue remain open?