add rbe-debian9 and rbe-ubuntu16_04 container #18

erain · 2018-03-26T19:42:25Z

rename debian8-clang-fully-loaded to its official name: rbe-debian8
rbe-debian9 with almost identical packages with rbe-debian8, except for libc++6-dev and ca-certificates-java
rbe-ubuntu16_04 with almost identical packages with rbe-debian8, except for ca-certificates-java

Tested:

container/build.sh -p my-project -c rbe-debian8 -d debian8 -t latest -b my-debs -a
container/build.sh -p my-project -c rbe-debian9 -d debian9 -t latest -b my-debs -a
container/build.sh -p my-project -c rbe-ubuntu16_04 -d ubuntu16_04 -t latest -b my-debs -a
container/build.sh -d debian8 -l
container/build.sh -d debian9 -l
container/build.sh -d ubuntu16_04 -l
tested produced rbe-debian9 and rbe-ubuntu16_04 image for building gRPC and Bazel

* we also rename debian8-clang-fully-loaded to rbe-debian8: https://pantheon.corp.google.com/launcher/details/google/rbe-debian8 Tested: * container/build.sh -p my-project -c rbe-debian8 -d debian8 -t latest -b my-debs -a * container/build.sh -p my-project -c rbe-debian9 -d debian9 -t latest -b my-debs -a * container/build.sh -d debian8 -l * container/build.sh -d debian9 -l * tested produced rbe-debian9 image for building gRPC and Bazel Change-Id: I233193c2bfaa4f9bf4f759de9c8f62f9c9179ae5

Change-Id: I65c8e4f38301afc8e1a5509012aaf10a839da421

Change-Id: Iaf0381722afbb4bd61d1ccac941e5b1b6c4ab5cc

Change-Id: I6919105867e3f564d277864b13c63d754ee8fe3c

nlopezgi · 2018-03-28T17:13:35Z

container/build.sh

@@ -23,9 +23,10 @@ Usage: build.sh [options]
 Builds the fully-loaded container, with Google Cloud Container Builder or locally.

 Required parameters (when build with Google Cloud Container Builder):
+    -d|--distro             Distro of the base image, debian 8, debian 9


add full list of supported base images here with the same names as expected below (debian8, debian9, ubuntu16_04)

nlopezgi · 2018-03-28T17:25:59Z

container/experimental/rbe-ubuntu16_04/BUILD

+
+container_test(
+    name = "fl-toolchain-test",
+    configs = ["//container/test:rbe-ubuntu16_04.yaml"],


Lets try to see if we can pass several files here and split off everything that is common from the verifications below to a single common file.

Yes this is do-able and I have extracted the common part into a common.yaml file to share among 3 containers.

- consolidate the common part of container_test into one common.yaml config file - fix other minor formatting problems. Change-Id: I2e1fd01682471c0888b8414b3b4716250bce3c2e

nlopezgi

couple of minor nits, maybe replace fl- for rbe- in the two names i mentioned?

nlopezgi · 2018-03-29T13:29:13Z

container/experimental/rbe-debian9/BUILD

+)
+
+container_test(
+    name = "fl-toolchain-test",


call this just "toolchain-test"

nlopezgi · 2018-03-29T13:29:58Z

container/experimental/rbe-ubuntu16_04/BUILD

+load("@io_bazel_rules_docker//contrib:test.bzl", "container_test")
+
+toolchain_container(
+    name = "fl-toolchain",


just "toolchain"? (or rbe-toolchain or something like that?)

Change-Id: I63dea016b728f8cce42f491bb553960585002c63

erain · 2018-03-29T14:40:22Z

I have replaced "fl-toolchain" to "toolchain" everywhere and tested the changes.

erain · 2018-04-03T14:56:26Z

The test failed because the infra itself, and not because the test. I ran the tests locally and they all succeeded.

Merging the change.

erain · 2018-04-03T17:02:14Z

After (many) retries the tests finally passed. We should bring it up to bazel testing infra team, but not the priority ATM.

I will merge this change and started testing the new images.

vladmos · 2018-04-04T13:42:57Z

As mentioned above, this change breaks the tests on the CI: https://buildkite.com/bazel/bazel-toolchains/builds/120
Even if the problem is with the CI itself, the failing test should be fixed somehow or switched off, otherwise it'll be easy to miss other failures in the future.

erain · 2018-04-04T20:51:06Z

@vladmos I have been tested the failing tests and they all passed locally

bazel test //container/experimental/rbe-debian9:toolchain-test
bazel test //container/experimental/rbe-ubuntu16_04

I am sending a PR to disable these two tests temporarily. However, on the other hand, is there a way for me to debug the Bazel CI system to see what might be wrong?

- due to Bazel CI problem, see comments in bazelbuild#18 Change-Id: I819d4f424f2ab0092730fa5eeb9ad3060c78008a

vladmos · 2018-04-05T12:45:46Z

Here's an example of the test logs, looks like the OS version string doesn't match the expected string: https://storage.googleapis.com/bazel-buildkite-artifacts/41fcd03a-3a4c-4ddd-937a-95ddf5a0df3f/container/experimental/rbe-debian9/toolchain-test/attempt_1.log

* temporarily disable container tests - due to Bazel CI problem, see comments in #18

erain · 2018-04-05T20:47:34Z

Hi Vladimir,

If you take a closer look, you'll notice things in the tests output were messed up.

The test itself complain the following target failed: //container/experimental/rbe-debian9:toolchain-test, with wrong OS version. However, if you clone the repo and run bazel test //container/experimental/rbe-debian9:toolchain-test, it would just pass.

So it looks to me that the stdout of another test, //container/experimental/rbe-ubuntu16_04:toolchain-test, affected this test and causeed one OR both tests failed.

Manjaro is based on the Arch linux distribution, so should have the same sysroot composition.

erain requested review from nlopezgi and xingao267 March 26, 2018 19:42

erain changed the title ~~add rbe-debian9 container~~ add rbe-debian9 and rbe-ubuntu16_04 container Mar 26, 2018

erain changed the title ~~add rbe-debian9 and rbe-ubuntu16_04 container~~ add rbe-debian9 and rbe-ubuntu16_04 container Mar 26, 2018

erain added 3 commits March 26, 2018 17:06

add rbe-ubuntu16_04, similarly rbe-debian9

d2261da

Change-Id: I65c8e4f38301afc8e1a5509012aaf10a839da421

add presubmit test for rbe-ubuntu16_04

a229455

Change-Id: Iaf0381722afbb4bd61d1ccac941e5b1b6c4ab5cc

erain force-pushed the rbe-debian9 branch from c5add90 to a229455 Compare March 26, 2018 21:08

move rbe-{debian8, ubuntu16_04} to experimental

fbc45c9

Change-Id: I6919105867e3f564d277864b13c63d754ee8fe3c

nlopezgi reviewed Mar 28, 2018

View reviewed changes

address reviewer's comment

44941d5

- consolidate the common part of container_test into one common.yaml config file - fix other minor formatting problems. Change-Id: I2e1fd01682471c0888b8414b3b4716250bce3c2e

nlopezgi suggested changes Mar 29, 2018

View reviewed changes

address comment: fl-toolchain -> toolchain

4a771e6

Change-Id: I63dea016b728f8cce42f491bb553960585002c63

nlopezgi approved these changes Mar 29, 2018

View reviewed changes

erain merged commit 30422f9 into bazelbuild:master Apr 3, 2018

erain deleted the rbe-debian9 branch April 3, 2018 17:04

erain added a commit to erain/bazel-toolchains that referenced this pull request Apr 4, 2018

temporarily disable container tests

c91bf22

- due to Bazel CI problem, see comments in bazelbuild#18 Change-Id: I819d4f424f2ab0092730fa5eeb9ad3060c78008a

erain mentioned this pull request Apr 4, 2018

temporarily disable container tests #19

Merged

erain added a commit that referenced this pull request Apr 5, 2018

temporarily disable container tests (#19)

5d7a79f

* temporarily disable container tests - due to Bazel CI problem, see comments in #18

freeformstu pushed a commit to freeformstu/bazel-toolchains that referenced this pull request Nov 11, 2023

llvm_toolchain: adds manjaro support (bazelbuild#18)

6649146

Manjaro is based on the Arch linux distribution, so should have the same sysroot composition.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add rbe-debian9 and rbe-ubuntu16_04 container #18

add rbe-debian9 and rbe-ubuntu16_04 container #18

erain commented Mar 26, 2018 •

edited

nlopezgi Mar 28, 2018

erain Mar 28, 2018

nlopezgi Mar 28, 2018

erain Mar 28, 2018

nlopezgi left a comment

nlopezgi Mar 29, 2018

nlopezgi Mar 29, 2018

erain commented Mar 29, 2018

erain commented Apr 3, 2018

erain commented Apr 3, 2018

vladmos commented Apr 4, 2018

erain commented Apr 4, 2018

vladmos commented Apr 5, 2018

erain commented Apr 5, 2018

add rbe-debian9 and rbe-ubuntu16_04 container #18

add rbe-debian9 and rbe-ubuntu16_04 container #18

Conversation

erain commented Mar 26, 2018 • edited

nlopezgi Mar 28, 2018

Choose a reason for hiding this comment

erain Mar 28, 2018

Choose a reason for hiding this comment

nlopezgi Mar 28, 2018

Choose a reason for hiding this comment

erain Mar 28, 2018

Choose a reason for hiding this comment

nlopezgi left a comment

Choose a reason for hiding this comment

nlopezgi Mar 29, 2018

Choose a reason for hiding this comment

nlopezgi Mar 29, 2018

Choose a reason for hiding this comment

erain commented Mar 29, 2018

erain commented Apr 3, 2018

erain commented Apr 3, 2018

vladmos commented Apr 4, 2018

erain commented Apr 4, 2018

vladmos commented Apr 5, 2018

erain commented Apr 5, 2018

erain commented Mar 26, 2018 •

edited