Release 0.26 - May 2019 #7499

laurentlb · 2019-02-21T20:46:09Z

Target RC date: May 2nd

dslomov · 2019-04-30T12:01:20Z

We would like to make sure that at least experimenal implementation of #8201 is in this release.

aehlig · 2019-04-30T12:51:11Z

We would like to make sure that at least experimenal implementation of #8201 is in this release.

When do you expect this implementation to be committed to the bazel code base?

brandjon · 2019-04-30T20:08:19Z

I'd like to flip --incompatible_use_python_toolchains in this release, but I ended up being blocked on #5104, which would mean that flipping this flag will break users of --build_python_zip who are not currently overriding the default Python runtime. Not sure how I'll resolve this yet, will update here...

sergiocampama · 2019-05-01T14:50:51Z

I'd like to flip --incompatible_disable_objc_provider_resources, and I'm trying to submit this change before the May 2nd deadline, would it be fine to cherrypick if I don't make it by the 2nd?

EDIT: This has been submitted on May 1st.

brandjon · 2019-05-01T23:42:29Z

Submitted the flag flip for Python toolchains, so if the baseline is cut after today then I'm good.

dslomov · 2019-05-02T01:27:06Z

We would like to make sure that at least experimenal implementation of #8201 is in this release.

When do you expect this implementation to be committed to the bazel code base?

In the coming days.

aehlig · 2019-05-02T09:29:14Z

Current state of our downstream projects.

The last fully green nightly is from Apr 17, 2019
The nightly from May 1st is pretty good, with only "BUILD file generator" failing, "rules_gwt" broken, and "TensorFlow" failing to clone.

This flips --incompatible_use_python_toolchains, which deprecates --python_top (and for the most part, --python_path). See [#7899](#7899) for more on the change and migration procedure. RELNOTES[INC]: Python rules now determine the Python runtime using toolchains rather than `--python_top` and `--python_path`, which are deprecated. See [#7899](#7899) for information on declaring Python toolchains and migrating your code. As a side-benefit, this addresses #4815 (incorrect interpreter version used) on non-Windows platforms. You can temporarily opt out of this change with `--incompatible_use_python_toolchains=false`. Fixes #7899, fixes #7375, significant progress on #4815. PiperOrigin-RevId: 246219480

brandjon · 2019-05-02T14:34:39Z

Apparently my flag flip broken a mac test in postsubmit. Working on a simple forward fix to disable the flip for that test now...

aehlig · 2019-05-02T14:44:38Z

Apparently my flag flip broken a mac test in postsubmit. Working on a simple forward fix to disable the flip for that test now...

Looking at today's nightly, a lot of breakage seems python related. So, rolling back seems more appropriate, unless you verify that your forward fix fixes the python-related downstream breakages as well. With that large number of downstream breakages, we're stuck with the nightly from May 1st (i.e., commit baefeab) anyway.

brandjon · 2019-05-02T14:57:33Z

Sadly yes. I'll roll back my flag flip and work on downstream fixes.

brandjon · 2019-05-02T16:18:35Z

Rolled back by 4837e11, so you can cherrypick that or cut the baseline after.

aehlig · 2019-05-03T07:10:51Z

The current nightly has all the breakages of the one from May 1st and, on top of that, the ones tracked by #8227. So the the canonical base line would be baefeab which we should take according to our policy. However, I was ordered to delay the release to wait for #8201.

dslomov · 2019-05-03T09:55:35Z

For context #8201 is critical for the angular/angular#19058 - Angular team is blocked on this for enabling bazel in their 8.0 release (which is alerady in RC and is going GA late May/early June). I understand that this is somewhat frustrating, but @irengrig and other folks on the team are hard at work getting the last bits in.

irengrig · 2019-05-03T13:16:48Z

#8201 is submitted!!! :)

aehlig · 2019-05-03T14:29:34Z

To fix the non-determinism uncovered (but not caused) by 29f1932, will target as baseline commit https://bazel-review.googlesource.com/c/bazel/+/98615, once in.

aehlig · 2019-05-06T07:25:56Z

The latest nightly seems alright, at least not worse than the one of May 1st. "BUILD file generator" failing, "rules_gwt" failing to build, and "Tensorflow" failing to clone; the latter is a known issue.

So, let's take daa8ae5 as a baseline, and cherry-pick https://bazel-review.googlesource.com/c/bazel/+/98615 once in. Depending on whether there will be a patch release for 0.25.0 (#7498), we might need additional cherry-picks.

aehlig · 2019-05-06T07:57:59Z

Creating RC1 failed.

ERROR: /workdir/scripts/packages/debian/BUILD:69:1: in _pkg_deb rule //scripts/packages/debian:bazel-debian:
Traceback (most recent call last):
        File "/workdir/scripts/packages/debian/BUILD", line 69
                _pkg_deb(name = 'bazel-debian')
        File "/workdir/tools/build_defs/pkg/pkg.bzl", line 128, in _pkg_deb_impl
                ctx.outputs.deb.path
object of type 'NoneType' has no field 'path'
ERROR: Analysis of target '//scripts/packages/debian:bazel-debian.deb' failed; build aborted: Analysis of target '//scripts/packages/debian:bazel-debian' failed; build aborted

Investigating.

aehlig · 2019-05-06T08:24:46Z

This is caused by commit bbe47a1. I verified that reverting it makes the build of //scripts/packages/debian:bazel-debian.deb succeed. Will try to create a roll back.

aehlig · 2019-05-06T09:39:13Z

Rollback submitted as c2001a4

Between the previous baseline and this commit, there is another rollback which we should include anyway, as releases should include all rollbacks of commits in their baseline, and a commit adding a single test case, which shouldn't do much harm. So I'll bump the baseline to c2001a4 for my attempt to create RC2.

aehlig · 2019-05-06T11:00:37Z

RC2 is available at https://releases.bazel.build/0.26.0/rc2/index.html

The release announcement is off though; I'll have to investigate where our release script took the wrong baseline for generating the release notes. Nevertheless, please test the release candidate.

aehlig · 2019-05-06T12:06:17Z

Will try to create RC3 with same baseline and cherry-picking e67c961 as this is the fix for a non-determinism problem we observed while searching for a good baseline.

aehlig · 2019-05-06T12:29:28Z

RC3 is available at https://releases.bazel.build/0.26.0/rc3/index.html

The release notes are still off. It is really bazeline c2001a4 with cherry-pick e67c961.

aehlig · 2019-05-06T12:38:31Z

Tests for downstream projects

aehlig · 2019-05-06T14:45:03Z

The downstream runs for RC3 look reasonable. Please test RC3 and report regressions before May 14.

aehlig · 2019-05-28T10:48:13Z

And we have the first regression: #8475. Fix is being implemented.

So we will have a patch release, once this is fixed.

irengrig · 2019-05-29T14:53:05Z

The fix for #8475 is submitted.

steeve · 2019-05-31T09:47:47Z

Submitting #8522 for cherry-pick, which fixes #8520.
This fixes rules_go on iOS and Android.

katre · 2019-05-31T14:43:29Z

c3d2aa7 is submitted and fixes #8520.

aehlig · 2019-06-03T09:20:16Z

Creating rc1 for the patch release 0.26.1.

scripts/release/release.sh create 0.26.1 cb82ed84d44db0169a8fbf15f9cee434b77002bb d1c0d205945f5a765efb0a48593b1cd82699ce32 c3d2aa74ccd23dfb8a8173c2b3e2955f0c5892cb

Taking as baseline the release commit for 0.26.0, i.e. cb82ed8
cherry-picking d1c0d20 for "WORKSPACE file can not be a symlink" exception thrown: backward-incompatible bug #8475
cherry-picking c3d2aa7 for osx_archs uses arm for aarch64 #8520

It is available at https://releases.bazel.build/0.26.1/rc1/index.html
(Please don't get confused by the list of cherry-picks mentioned there; our release script got them wrong again; we only have the two cherry-picks mentioned here.)

aehlig · 2019-06-03T11:10:52Z

The downstream projects look good (the "Bazel Integration testing" failure is due to changed RBE configurations).

So please test 0.26.1rc1. I plan to release it on Wednesday, June 5, 2019, in the morning, Munich time.

jin · 2019-06-03T18:24:11Z

Please cherrypick b0403a7 for bazelbuild/intellij#845. Fixes Bazel with IntelliJ when building annotation processor targets.

aehlig · 2019-06-04T06:45:03Z

Please cherrypick b0403a7 for bazelbuild/intellij#845. Fixes Bazel with IntelliJ when building annotation processor targets.

Is this really a regression, i.e., something not working on 0.26.0 that was working on 0.25.3? Also note that cherry-picks are supposed to be small, well-understood changes where we can be reasonably sure that they do not break anything else; a bump of an opaque archive usually does not qualify as such.

philwo · 2019-06-04T06:54:44Z

@aehlig Absolutely a regression, I can’t even build Bazel itself anymore with IntelliJ inside a Google and thought I’m doing something wrong... I'm getting the Genclass error that’s mentioned in the linked commit. This always worked before 0.26.0.

aehlig · 2019-06-04T06:57:37Z

This is the diffstat(1) between the contents of the old and the bumped linux archive.

 BUILD                                                                           |   31 
 README.md                                                                       |    6 
 java_tools/JacocoCoverage_jarjar_deploy.jar                                     |binary
 java_tools/JavaBuilder_deploy.jar                                               |binary
 java_tools/VanillaJavaBuilder_deploy.jar                                        |binary
 java_tools/ijar/ijar                                                            |binary
 java_tools/ijar/zipper                                                          |binary
 java_tools/src/main/cpp/util/errors_windows.cc                                  |    5 
 java_tools/src/main/cpp/util/file_windows.cc                                    |    7 
 java_tools/src/main/cpp/util/path_platform.h                                    |    3 
 java_tools/src/main/cpp/util/path_windows.cc                                    |  154 --
 java_tools/src/main/cpp/util/profiler_windows.cc                                |    4 
 java_tools/src/main/cpp/util/strings.cc                                         |    7 
 java_tools/src/main/native/windows/file.cc                                      |  120 +-
 java_tools/src/main/native/windows/file.h                                       |   51 
 java_tools/src/main/native/windows/util.cc                                      |   25 
 java_tools/src/main/native/windows/util.h                                       |    6 
 java_tools/src/tools/singlejar/diag.h                                           |    6 
 java_tools/third_party/java/jacoco/LICENSE                                      |  544 ++--------
 java_tools/third_party/java/jacoco/jacocoagent-0.8.3.jar                        |binary
 java_tools/third_party/java/jacoco/jacocoagent.jar                              |binary
 java_tools/third_party/java/jacoco/org.jacoco.agent-0.7.5.201505241946-src.jar  |binary
 java_tools/third_party/java/jacoco/org.jacoco.agent-0.7.5.201505241946.jar      |binary
 java_tools/third_party/java/jacoco/org.jacoco.agent-0.8.3-sources.jar           |binary
 java_tools/third_party/java/jacoco/org.jacoco.agent-0.8.3.jar                   |binary
 java_tools/third_party/java/jacoco/org.jacoco.core-0.7.5.201505241946-src.jar   |binary
 java_tools/third_party/java/jacoco/org.jacoco.core-0.7.5.201505241946.jar       |binary
 java_tools/third_party/java/jacoco/org.jacoco.core-0.8.3-sources.jar            |binary
 java_tools/third_party/java/jacoco/org.jacoco.core-0.8.3.jar                    |binary
 java_tools/third_party/java/jacoco/org.jacoco.report-0.7.5.201505241946-src.jar |binary
 java_tools/third_party/java/jacoco/org.jacoco.report-0.7.5.201505241946.jar     |binary
 java_tools/third_party/java/jacoco/org.jacoco.report-0.8.3-sources.jar          |binary
 java_tools/third_party/java/jacoco/org.jacoco.report-0.8.3.jar                  |binary
 33 files changed, 406 insertions(+), 563 deletions(-)

This is more than a small, well-understood, change. Isn't there a way to just fix the regression without changing two binaries, 16 java archives, and the license?

aehlig · 2019-06-04T06:59:49Z

@aehlig Absolutely a regression, I can’t even build Bazel itself anymore with IntelliJ [...]

So how was that regression introduced into the code base. I doubt it was introduced by a downgrade of the java tools...

philwo · 2019-06-04T07:03:59Z

I don’t know, I’m just a user in this case who can no longer work until this is fixed.

aehlig · 2019-06-04T07:06:37Z

I don’t know, I’m just a user in this case who can no longer work until this is fixed.

OK, then we'll bisect to find the offending commit and try to cherry-pick a revert of it (the revert needn't go on master, it can stay on a side branch); but such a big library change to fix forward is too large for a patch release, in my opinion.

iirina · 2019-06-04T08:24:26Z

While b0403a7 fixes bazelbuild/intellij#845, I'm not sure that was the culprit. The typo fixed by the new java_tools release was there from the first java_tools version, so downgrading the java_tools version will not work. @aehlig's proposal seems the right way forward.

philwo · 2019-06-04T08:45:35Z

I'm bisecting this now.

philwo · 2019-06-04T09:16:04Z

This commit is the culprit: 8887169

Verified by checking out 0.26.0 and doing a git revert. I can then no longer repro the issue.

aehlig · 2019-06-04T09:23:48Z

This commit is the culprit: 8887169

Verified by checking out 0.26.0 and doing a git revert. I can then no longer repro the issue.

Thanks for the bisect. Now we're in trouble here, as the culprit is also a bump of an embedded library. Will have to have a look at both changes to make a call, what is less likely to break things. Any technical information about those two library changes would be greatly appreciated.

aehlig · 2019-06-04T09:30:27Z

Now we're in trouble here, as the culprit is also a bump of an embedded library.

Fortunately, the changes in the libraries hidden in the culript commit are comparatively small; 192 lines of BUILD file (and a README file). So, let's try cherry-picking the revert of the culprit.

aehlig · 2019-06-04T12:57:37Z

@iirina created a patch-release for the java tools with only the minimal changes needed to fix bazelbuild/intellij#845. Thank you! So we can get away with cherry-pick a much smaller change, 55e4205.

aehlig · 2019-06-04T13:10:35Z

scripts/release/release.sh create --force_rc=3 0.26.1 cb82ed84d44db0169a8fbf15f9cee434b77002bb d1c0d205945f5a765efb0a48593b1cd82699ce32 c3d2aa74ccd23dfb8a8173c2b3e2955f0c5892cb 55e42052a22a60b68d88a89932b2a068311b1a95
scripts/release/release.sh push

(There will be no rc2, as I forgot to fetch the branches on first attempt and thus decided it's safer to drop the rc number.)

aehlig · 2019-06-04T13:21:03Z

#8556 tracks our efforts of avoiding such regressions in the future.

aehlig · 2019-06-04T13:23:21Z

0.26.1rc3 is available at https://releases.bazel.build/0.26.1/rc3/index.html

aehlig · 2019-06-04T14:34:57Z

The downstream projects still look good, again with "Bazel Integration Testing" broken due to changed RBE configurations.

Please test carefully. I plan to release 0.26.1 on Thursday, Jun 6, 2019, early afternoon Munich time.

aehlig · 2019-06-06T10:59:12Z

Promoting 0.26.1rc3 to release...

aehlig · 2019-06-06T11:24:49Z

0.26.1 is available at https://releases.bazel.build/0.26.1/release/index.html

It accidentally used ubuntu1804, which means we built a version no longer compatible with Ubuntu 14.04: bazelbuild/bazel#7499 (comment) Note that Bazel 0.26 will be the last version built on Ubuntu 14.04, though, as was previously announced here: https://groups.google.com/forum/#!searchin/bazel-discuss/ubuntu$2014.04%7Csort:date/bazel-discuss/dPomOEOPseA/Jp849gJvBwAJ

laurentlb added the release label Feb 21, 2019

laurentlb assigned aehlig Feb 21, 2019

laszlocsomor mentioned this issue Apr 11, 2019

Windows: remove incompatible_windows_style_arg_escaping #8003

Closed

dslomov pinned this issue Apr 30, 2019

brandjon mentioned this issue May 2, 2019

incompatible_use_python_toolchains: The Python runtime is obtained from a toolchain rather than a flag #7899

Closed

aehlig reopened this May 28, 2019

davido mentioned this issue Jun 5, 2019

Bazel 0.26.1 davido/bazel-alpine-package#9

Merged

aehlig closed this as completed Jun 6, 2019

aehlig unpinned this issue Jun 6, 2019

Release 0.26 - May 2019 #7499

Release 0.26 - May 2019 #7499

Comments

laurentlb commented Feb 21, 2019

dslomov commented Apr 30, 2019

aehlig commented Apr 30, 2019

brandjon commented Apr 30, 2019

sergiocampama commented May 1, 2019 • edited Loading

brandjon commented May 1, 2019

dslomov commented May 2, 2019

aehlig commented May 2, 2019

brandjon commented May 2, 2019

aehlig commented May 2, 2019

brandjon commented May 2, 2019

brandjon commented May 2, 2019

aehlig commented May 3, 2019

dslomov commented May 3, 2019

irengrig commented May 3, 2019

aehlig commented May 3, 2019

aehlig commented May 6, 2019

aehlig commented May 6, 2019

aehlig commented May 6, 2019

aehlig commented May 6, 2019

aehlig commented May 6, 2019

aehlig commented May 6, 2019

aehlig commented May 6, 2019

aehlig commented May 6, 2019

aehlig commented May 6, 2019

aehlig commented May 28, 2019

irengrig commented May 29, 2019

steeve commented May 31, 2019

katre commented May 31, 2019

aehlig commented Jun 3, 2019

aehlig commented Jun 3, 2019

jin commented Jun 3, 2019

aehlig commented Jun 4, 2019

philwo commented Jun 4, 2019

aehlig commented Jun 4, 2019

aehlig commented Jun 4, 2019

philwo commented Jun 4, 2019

aehlig commented Jun 4, 2019

iirina commented Jun 4, 2019

philwo commented Jun 4, 2019

philwo commented Jun 4, 2019

aehlig commented Jun 4, 2019

aehlig commented Jun 4, 2019

aehlig commented Jun 4, 2019

aehlig commented Jun 4, 2019

aehlig commented Jun 4, 2019

aehlig commented Jun 4, 2019

aehlig commented Jun 4, 2019

aehlig commented Jun 6, 2019

aehlig commented Jun 6, 2019

sergiocampama commented May 1, 2019 •

edited

Loading