Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.27 - June 2019 #7816

Open
dslomov opened this issue Mar 22, 2019 · 35 comments

Comments

Projects
None yet
@dslomov
Copy link
Contributor

commented Mar 22, 2019

Target RC date: June 3rd

@dslomov dslomov added the release label Mar 22, 2019

@dslomov dslomov changed the title Release 0.27 - June Release 0.27 - June 2019 Mar 22, 2019

@bazelbuild bazelbuild deleted a comment Apr 19, 2019

@laurentlb laurentlb self-assigned this Apr 30, 2019

@aehlig aehlig pinned this issue May 28, 2019

@Ragora

This comment has been minimized.

Copy link

commented May 29, 2019

I would like 84c5dc9#diff-73cc3e84377e7c63ef4406039e060016 (or an equivalent thereof) to be in this release.

For my employer I started this fairly large migration to bazel seeing from documentation it did everything we needed, however, it seems pkg_deb does not have a templates parameter which we use quite heavily in our existing software meaning the resulting packages in the current state are kindof useless.

I'm at a point of having to make a bazel build from master just for this one feature.

Amazing tool otherwise, though! It's just this one thing is kindof a stickling point at the moment.

@laurentlb

This comment has been minimized.

Copy link
Member

commented May 29, 2019

It will definitely be included. Any change submitted in May will be included. Changes submitted in June are unlikely to part of 0.27.

Thanks for the feedback!

@Ragora

This comment has been minimized.

Copy link

commented May 29, 2019

Cool, was checking because I got confused about the 0.26 release still not having it, guess I just unlucky about the way times line up then. In the meantime I'll use a custom build, but at least it's not long to wait.

@laurentlb

This comment has been minimized.

Copy link
Member

commented May 29, 2019

Status of Bazel 0.27

I plan to keep this message up to date during the release process and provide the important information.

To report a release-blocking bug: file a bug, use the Release blocker label, and cc @laurentlb. Release blockers are regressions in Bazel relative to Bazel 0.25 or Bazel 0.26.

Task list:

@brandjon

This comment has been minimized.

Copy link
Member

commented May 31, 2019

I'm about to flip --incompatible_use_python_toolchains. This was introduced in 0.25, flipped at head for 0.26, and immediately rolled back when it broke downstream. So it's now had two months of migration window.

All known downstream breakages are summarized in this comment. Most breakages are because targets had been declaring themselves (perhaps by default) to be Python 3, even though they were really only compatible with Python 2. Previously they sometimes got ran under Python 2 anyway due to #4815, but this flag fixes that bug on every platform besides Windows. Migration usually consists of adding python_version = "PY2" to Python targets, and --host_force_python=PY2 if any of them are used in the host configuration.

+cc Green Team sheriffs @vladmos and @laszlocsomor.

@laurentlb

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

Creating 0.27.0rc1 with:

scripts/release/release.sh create 0.27.0 5935259

@jin

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

0.26.0 / 0.27.0 is broken in IntelliJ when building java_plugin targets - bazelbuild/intellij#845

Please cherry-pick b0403a7 (if it's after baseline)

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

Created blockers #8547 and #8549 to track my effort to add better diagnostics for users whose Python targets fail due to toolchains.

@philwo

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

Can we ensure that 8c3b3fb gets into the release? It resolves a major issue with —keep_going that causes failing builds to hide the error and exit with code 0 (#7674). This in turn causes Bazel’s CI to Mark pipelines as green, even though the projects are broken.

@laurentlb

This comment has been minimized.

Copy link
Member

commented Jun 4, 2019

Created 0.27rc2 with:

scripts/release/release.sh create --force_rc=2 0.27.0 8c3b3fb

(this includes both b0403a7 and 8c3b3fb)

You can download the release candidate here: https://releases.bazel.build/0.27.0/rc2/index.html

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 4, 2019

Of the two release blockers I filed, one is fixed and the other has a pending CL. Both concern the ease of migrating to toolchains but do not affect CI.

The already submitted one is 123c68d and can be cherrypicked now. Will post when the other is submitted.

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 5, 2019

And the other one is 052167e.

@meisterT

This comment has been minimized.

Copy link
Member

commented Jun 5, 2019

Laurent, this might require a cherry pick: #8539

@davido

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2019

Laurent, this might require a cherry pick: #8539

As pointed out in discussion in the above issue, bad5a2b introduced very serious regression (and is included in 0.27rc2):

The previous javac version had a non-standard modification to default to
the Java 8 language level, which wasn't carried forward to 11.

As the consequence, after the upgrade from 0.26 to 0.27rc2 wrong byte code is generated, when building with default_java_toolchain. Even though, Java 8 is used, byte code with major version 55 is produced that corresponds to Java 11.

  $ bazel version
  Build label: 0.27.0rc2

  $ $JAVA_HOME/bin/java -version
  openjdk version "1.8.0_212"

  $ bazel build java/com/google/gerrit/common:server 
  [...]
  Target //java/com/google/gerrit/common:server up-to-date:
  bazel-bin/java/com/google/gerrit/common/libserver.jar
INFO: Elapsed time: 46.555s, Critical Path: 30.31s
INFO: 36 processes: 27 linux-sandbox, 9 worker.
INFO: Build completed successfully, 38 total actions

  $ javap -verbose -cp bazel-bin/java/com/google/gerrit/common/libserver.jar com.google.gerrit.common.data.SubscribeSection | grep major 
  major version: 55

On Bazel 0.26 the last command would correctly print Java 8 byte code major version:

  $ javap -verbose -cp bazel-bin/java/com/google/gerrit/common/libserver.jar com.google.gerrit.common.data.SubscribeSection | grep major 
  major version: 52

As the consequence, after building Gerrit Code Review project with 0.27.rc2, the produced artifact cannot be run any more on Java 8 (all is fine on 0.26):

  $ $JAVA_HOME/bin/java -version
  openjdk version "1.8.0_212"

  $ bazel build release
  [...]

  $ $JAVA_HOME/bin/java -jar bazel-bin/release.war init -d ../test_site_xxx_yyy_zzz
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: Main has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)

Can this be fixed, or at very least, can this disruptive change be documented in the release documentation?

@philwo

This comment has been minimized.

Copy link
Member

commented Jun 6, 2019

Please note that we finished the migration to Ubuntu 16.04 for release builds and all other pipelines on Bazel CI now. This means that 0.27.0rc3 and following versions will be built on Ubuntu 16.04 and will no longer compatible with Ubuntu 14.04.

We will mention this in the release notes, too.

@iirina

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2019

@laurentlb 6ef6d87 fixes #8539 and should be cherry picked for backwards compatibility.

@laurentlb

This comment has been minimized.

Copy link
Member

commented Jun 6, 2019

rc3 created with:

scripts/release/release.sh create --force_rc=3 0.27.0 8c3b3fb 123c68d 052167e 6ef6d87

Available here: https://releases.bazel.build/0.27.0/rc3/index.html

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 6, 2019

Filed #8576 for bugs I just noticed in 052167e. (Sorry.) Working on it now.

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 7, 2019

Fixed #8576 with 50fa3ec, please cherrypick.

@irengrig

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2019

Please cherry-pick 6efc5b7, a fix for #8564.

@jin

This comment has been minimized.

Copy link
Member

commented Jun 7, 2019

Please cherry pick e2a626c for bazelbuild/rules_jvm_external#165 and googlesamples/android-testing#279

Thank you!

@laurentlb

This comment has been minimized.

Copy link
Member

commented Jun 7, 2019

rc4 created with:

scripts/release/release.sh create --force_rc=4 0.27.0 8c3b3fb 123c68d 052167e 6ef6d87 50fa3ec e2a626c

#8564 (comment) was not included.

Download rc4: https://releases.bazel.build/0.27.0/rc4/index.html
Status of downstream projects: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1027

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 10, 2019

Update on toolchain downstream breakages here.

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 11, 2019

#8536 is now release-blocking as I decide how to address the gerrit breakage. The error mode only occurs when

  1. using --incompatible_strict_action_env in combination with
  2. the default Python toolchain
  3. on a mac, and
  4. in a build that includes a genrule-within-a-genrule (or genrule underneath another host-configured target)

This combination should be pretty rare.

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 11, 2019

#8536 is fixed by a pending commit (blocked on transient infrastructure failures).

Laurent mentions that since this bug only manifests itself with --incompatible_strict_action_env, we may be allowed to break this use case and fix it in the next release with no cherrypick. However, some users have come to rely on this flag. It arguably functions more like an ordinary feature flag than an incompatible change flag since it has been around for a long time and apparently isn't going to be flipped.

I'll defer to @philwo, @aehlig, and @buchgr on whether it should be considered blocking.

@juliexxia

This comment has been minimized.

Copy link
Member

commented Jun 12, 2019

Request for a cherrypick for #8610

There's a change currently inside the release candidate that flips --experimental_build_setting_api to true. I've just found a pretty major incremental correctness concern with build settings so I think we should patch a flip back. Apologies for the added complexity!

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 12, 2019

#8536's fix is in as 3a4be3c. Please cherrypick if creating a new RC.

@laurentlb

This comment has been minimized.

Copy link
Member

commented Jun 12, 2019

I've created rc5 with 3 new cherry-picks. Unless I missed something, all cherry-pick requests from the thread have been included.

scripts/release/release.sh create --force_rc=5 0.27.0 c5f93b8 6efc5b7 3a4be3c 5c1005c

@laurentlb

This comment has been minimized.

Copy link
Member

commented Jun 12, 2019

Test of downstream projects: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1034
Expected release date: next Monday
Try the new candidate: https://releases.bazel.build/0.27.0/rc5/index.html

@juliexxia

This comment has been minimized.

Copy link
Member

commented Jun 13, 2019

Woof, Is it possible to add another cherrypick for #8633 ? 49862c5 is the commit. Right now bazel can crash because of the issue. Sorry to delay the release timeline again!

@davido

This comment has been minimized.

Copy link
Contributor

commented Jun 14, 2019

I have a call to document in the release notes for 0.27, that Python 3 is now required for Bazel.

I am able to build Gerrit Code Review on Mac OS on Bazel 0.26 without having Python3 installed, see this comment: [1].

With 0.27.0rc5 I am not able to build Gerrit Code Review any more. I have to install Python3. Looking into release notes on https://releases.bazel.build/0.27.0/rc5/index.html the only relevant section:

Python rules now determine the Python runtime using toolchains rather than --python_top and 
--python_path, which are deprecated.

doesn't clearly telling me, that Python3 is now de-facto pre-requisite for Bazel 0.27 to work.

@laurentlb

This comment has been minimized.

@davido

This comment has been minimized.

Copy link
Contributor

commented Jun 14, 2019

@laurentlb Thanks, that document clarifies the issue.

@brandjon

This comment has been minimized.

Copy link
Member

commented Jun 14, 2019

@davido, glad the (updated) release notes helped.

FTR, Bazel 0.27 shouldn't require Python 3 in order to function. But it may cause Bazel to try to use Python 3 by default where it didn't before. This can be worked around to restore the pre-existing behavior as per #7899.

@alexeagle

This comment has been minimized.

Copy link
Collaborator

commented Jun 14, 2019

@laurentlb the rc5 breaks rules_nodejs. This isn't visible in the BuildKite build because it doesn't build all the tests in nested Workspaces. A failure is visible on Ubuntu in https://circleci.com/gh/bazelbuild/rules_nodejs/20685#tests/containers/2 - this is from a PR where we tried to update the rules_nodejs monorepo to 0.27 rc5

Failure is in compiling ts_library rules.
Adding --strategy=TypeScriptCompile=standalone makes them build successfully. I tested it in earlier RCs, the problem is present in rc1 but not in 0.26.1.

EDIT: I tracked it down to --incompatible_list_based_execution_strategy_selection which is in the release notes. This has the subtle side effect of enabling worker mode for actions that support it, with no action from users. Maybe add this to the release notes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.