Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix remote tests to work on RBE #8033

Open
19 of 26 tasks
buchgr opened this issue Apr 15, 2019 · 1 comment
Open
19 of 26 tasks

Fix remote tests to work on RBE #8033

buchgr opened this issue Apr 15, 2019 · 1 comment
Labels
not stale Issues or PRs that are inactive but not considered stale P2 We'll consider working on this in future. (Assignee optional) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: feature request

Comments

@buchgr
Copy link
Contributor

buchgr commented Apr 15, 2019 •
edited by Wyverald
Loading

Fixed tests

Yet to fix

  • //src/tools/singlejar:output_jar_simple_test
    Fix this to work with absolute paths
    copts = select({
        "//src/conditions:windows": ["-DJAR_TOOL_PATH=\\\"io_bazel/$(JAVABASE)/bin/jar.exe\\\""],
        "//conditions:default": ["-DJAR_TOOL_PATH=\\\"io_bazel/$(JAVABASE)/bin/jar\\\""],
    }),
  • //src/test/shell/bazel:external_integration_test
  • //src/test/shell/bazel:maven_test
  • //src/test/shell/bazel:bazel_repository_cache_test
  • //src/test/shell/integration:java_integration_test
  • //src/test/java/com/google/devtools/build/lib:sandbox-tests
  • //src/test/shell/bazel/android/...
    @jin is working on a fix //src/test/shell/bazel/android/... is incompatible with RBE #8235
@buchgr buchgr self-assigned this Apr 15, 2019
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
fix zip64_test to work with absolute $(JAVABASE)
88c4e7b 
Tested with an absolute $(JAVABASE) and --(no)legacy_external_runfiles.

Progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 15, 2019
Closed
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
fix discard_analysis_cachetest.sh to properly depend on $(JAVABASE)
318f57e 
- no longer uses $bazel_javabase from testenv.sh
- works with absolute $(JAVABASE) paths and JDK's other than local_jdk

Progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 15, 2019
Closed
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
fix discard_graph_edges_test to properly depend on $(JAVABASE)
9b2432e 
- no longer uses $bazel_javabase from testenv.sh
- works with absolute $(JAVABASE) paths and JDK's other than loc

Progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 15, 2019
Closed
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
fix nonincremental_builds_test to properly depend on $(JAVABASE)
e8cac68 
- no longer uses $bazel_javabase from testenv.sh
- works with absolute $(JAVABASE) paths and JDK's other than loc

Progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 15, 2019
Closed
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
have bazel_example_test.sh properly depend on
076c98f 
- no longer uses $bazel_javabase from testenv.sh
- works with absolute $(JAVABASE) paths and JDK's other than
  local_jdk.

Progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 15, 2019
Closed
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
fix bazel_with_jdk_test to properly depend on JAVABASE
69b07d8 
Progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 15, 2019
Closed
bazel-io pushed a commit that referenced this issue Apr 15, 2019
@buchgr @copybara-github
fix zip64_test to work with absolute $(JAVABASE)
8fe5479 
Tested with an absolute $(JAVABASE) and --(no)legacy_external_runfiles.

Progress towards #8033

Closes #8034.

PiperOrigin-RevId: 243604034
bazel-io pushed a commit that referenced this issue Apr 15, 2019
@buchgr @copybara-github
fix bazel_with_jdk_test to properly depend on JAVABASE
6b1f66c 
Progress towards #8033

Closes #8042.

PiperOrigin-RevId: 243606106
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
make bazel_sandboxing_cpp_test work with bazel-toolchains
9032138 
the generated error message due to the missing headers file is
dependent on the compiler. relax the pattern matching a bit to
also work with the clang ubuntu container
https://console.cloud.google.com/marketplace/details/google/clang-ubuntu

progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 15, 2019
Closed
@aiuto aiuto added team-Remote-Exec Issues and PRs for the Execution (Remote) team untriaged labels Apr 15, 2019
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
fix zip64_test to work with absolute $(JAVABASE)
e2f5a68 
Tested with an absolute $(JAVABASE) and --(no)legacy_external_runfiles.

Progress towards bazelbuild#8033

Closes bazelbuild#8034.

PiperOrigin-RevId: 243604034
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
fix bazel_with_jdk_test to properly depend on JAVABASE
14eb1df 
Progress towards bazelbuild#8033

Closes bazelbuild#8042.

PiperOrigin-RevId: 243606106
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
correctly access javabase in java_integration_test
2af3d74 
java_integration_test assumes that $(JAVABASE) is always a runfiles
relative path. However, that's not correct as it purely depends on
the usage of java_runtime.

For example,

java_runtime(
name = "abs_path_jdk",
java_home = "/usr/local/lib/jdk",
)

when running with --javabase=//:abs_path_jdk then all $(JAVABASE)
make variables will contain "/usr/local/lib/jdk".

Progress towards bazelbuild#8033
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
correctly access javabase in java_integration_test
34419ed 
java_integration_test assumes that $(JAVABASE) is always a runfiles
relative path. However, that's not correct as it purely depends on
the usage of java_runtime.

For example,

java_runtime(
name = "abs_path_jdk",
java_home = "/usr/local/lib/jdk",
)

when running with --javabase=//:abs_path_jdk then all $(JAVABASE)
make variables will contain "/usr/local/lib/jdk".

Progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 15, 2019
Closed
buchgr added a commit to buchgr/bazel that referenced this issue Apr 15, 2019
@buchgr
enable some tests to run remotely
f4eb7a7 
these tests were tagged "local" in order to not use a sandbox. The
"local" tag unfortunately also applies to remote execution. Mark
them "no-sandbox" so that they can run with remote execution.

Progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 15, 2019
Closed
bazel-io pushed a commit that referenced this issue Apr 15, 2019
@buchgr @copybara-github
enable some tests to run remotely
460843f 
these tests were tagged "local" in order to not use a sandbox. The
"local" tag unfortunately also applies to remote execution. Mark
them "no-sandbox" so that they can run with remote execution.

Progress towards #8033

Closes #8048.

PiperOrigin-RevId: 243626963
bazel-io pushed a commit that referenced this issue Apr 15, 2019
@buchgr @copybara-github
fix nonincremental_builds_test to properly depend on $(JAVABASE)
8817025 
- no longer uses $bazel_javabase from testenv.sh
- works with absolute $(JAVABASE) paths and JDK's other than loc

Progress towards #8033

Closes #8038.

PiperOrigin-RevId: 243670387
buchgr added a commit to buchgr/bazel that referenced this issue Apr 16, 2019
@buchgr
have bazel_example_test.sh properly depend on
7bb734b 
- no longer uses $bazel_javabase from testenv.sh
- works with absolute $(JAVABASE) paths and JDK's other than
  local_jdk.

Progress towards bazelbuild#8033
bazel-io pushed a commit that referenced this issue Apr 16, 2019
@buchgr @copybara-github
fix discard_analysis_cachetest.sh to properly depend on $(JAVABASE)
37beb42 
- no longer uses $bazel_javabase from testenv.sh
- works with absolute $(JAVABASE) paths and JDK's other than local_jdk

Progress towards #8033

Closes #8036.

PiperOrigin-RevId: 243781602
buchgr added a commit to buchgr/bazel that referenced this issue Apr 16, 2019
@buchgr
Update RBE platform
bdab706 
 - Use the docker container from gcr.io/bazel-untrusted/ubuntu1604_bazel_rbe
   which was built specifically for Bazel's tests on RBE.
 - Introduce a highcpu_machine constraint in order to map heavy tests to
 beefy RBE workers.
 - Add the highcpu_machine constraints to heavy test targets.

Progress towards bazelbuild#8033
@buchgr buchgr mentioned this issue Apr 16, 2019
Closed
bazel-io pushed a commit that referenced this issue Apr 16, 2019
@buchgr @copybara-github
fix discard_graph_edges_test to properly depend on $(JAVABASE)
aadf364 
- no longer uses $bazel_javabase from testenv.sh
- works with absolute $(JAVABASE) paths and JDK's other than loc

Progress towards #8033

Closes #8037.

PiperOrigin-RevId: 243797243
bazel-io pushed a commit that referenced this issue Apr 16, 2019
@buchgr @copybara-github
have bazel_example_test.sh properly depend on
61bf65a 
- no longer uses $bazel_javabase from testenv.sh
- works with absolute $(JAVABASE) paths and JDK's other than
  local_jdk.

Progress towards #8033

Closes #8041.

PiperOrigin-RevId: 243798469
bazel-io pushed a commit that referenced this issue Apr 16, 2019
@buchgr @copybara-github
Update RBE platform
593c530 
 - Use the docker container from gcr.io/bazel-untrusted/ubuntu1604_bazel_rbe
   which was built specifically for Bazel's tests on RBE.
 - Introduce a highcpu_machine constraint in order to map heavy tests to
 beefy RBE workers.
 - Add the highcpu_machine constraints to heavy test targets.

Progress towards #8033

Closes #8062.

PiperOrigin-RevId: 243829775
buchgr added a commit to buchgr/bazel that referenced this issue Apr 24, 2019
@buchgr
add .bazelrc with remote config
91cb2f3 
This allows authenticated users to build and test
bazel on RBE. Most Bazel tests already work on RBE,
for a list (and progress update) on which don't yet
see bazelbuild#8033

$ bazel build --config-remote src:bazel
@buchgr buchgr mentioned this issue Apr 24, 2019
Closed
buchgr added a commit to buchgr/bazel that referenced this issue Apr 24, 2019
@buchgr
add .bazelrc with remote config
fcc4e9c 
This allows authenticated users to build and test
bazel on RBE. Most Bazel tests already work on RBE,
for a list (and progress update) on which don't yet
see bazelbuild#8033

$ bazel build --config-remote src:bazel
bazel-io pushed a commit that referenced this issue Apr 24, 2019
@buchgr @copybara-github
add .bazelrc with remote config
8da42b6 
This allows authenticated users to build and test
bazel on RBE. Most Bazel tests already work on RBE,
for a list (and progress update) on which don't yet
see #8033

$ bazel build --config-remote src:bazel

Closes #8128.

PiperOrigin-RevId: 245041747
buchgr added a commit to buchgr/bazel that referenced this issue Apr 30, 2019
@buchgr
add new rbe platforms
6e4138a 
use a custom built docker container that contains all Bazel test
dependencies as well as toolchains generated for this container.
See https://github.com/buchgr/bazel_rbe_toolchains

Progress towards bazelbuild#8033
buchgr added a commit to buchgr/bazel that referenced this issue Apr 30, 2019
@buchgr
add new rbe platforms
5fab9d5 
use a custom built docker container that contains all Bazel test
dependencies as well as toolchains generated for this container.
See https://github.com/buchgr/bazel_rbe_toolchains

Progress towards bazelbuild#8033
@buchgr buchgr added P2 We'll consider working on this in future. (Assignee optional) and removed untriaged labels May 2, 2019
@jin jin mentioned this issue May 3, 2019
Closed
@buchgr buchgr removed their assignment Jan 9, 2020
@sgowroji sgowroji added the stale Issues or PRs that are stale (no activity for 30 days) label Feb 16, 2023
@sgowroji
Copy link
Member

Hi there! We're doing a clean up of old issues and will be closing this one. Please reopen if you’d like to discuss anything further. We’ll respond as soon as we have the bandwidth/resources to do so.

@sgowroji sgowroji closed this as not planned Won't fix, can't repro, duplicate, stale Feb 16, 2023
@tjgq tjgq reopened this Feb 16, 2023
@tjgq tjgq removed the stale Issues or PRs that are stale (no activity for 30 days) label Feb 16, 2023
@sgowroji sgowroji added the not stale Issues or PRs that are inactive but not considered stale label Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
not stale Issues or PRs that are inactive but not considered stale P2 We'll consider working on this in future. (Assignee optional) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@buchgr @aiuto @tjgq @coeuvre @sgowroji