[master] Upgrade dependencies to fix CI cron build #11081

mkurz · 2021-12-30T20:43:32Z

Currently cron builds fail with:

[error] this can be overridden using libraryDependencySchemes or evictionErrorLevel
[error] (update) found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
[error] 
[error] 	* org.scala-lang.modules:scala-java8-compat_2.13:1.0.0 (early-semver) is selected over 0.9.1
[error] 	    +- com.typesafe.play:play-framework_2.13:2.8.1+1391-7eb9d1f1+20211230-1736 (depends on 1.0.0)
[error] 	    +- com.typesafe.akka:akka-actor_2.13:2.6.18           (depends on 1.0.0)
[error] 	    +- org.scala-lang.modules:scala-java8-compat_2.13:0.9.1 (depends on 0.9.1)

That is because of following combination:

The master branch uses sbt 1.5+ already, which (in contrast to sbt <1.5) fails if it detects major version conflicts
The cron build uses the latest akka snaphot which depends on scala-java8-compat 1.x, however Play depends on 0.x

Pull request builds did not fail however because they don't use the latest akka snapshot, but akka version 2.6.14, which did not yet depend on scala-java8-compat 1.x but still on 0.x.
Also cron builds for the 2.8.x branch did not fail because 2.8.x still is on sbt 1.3.13.

So the solution is we just need to upgrade scala-java8-compat to 1.0.2. And not just in Play, but also in play-ws, which luckily was done by Renato with play-ws 2.1.6 already: playframework/play-ws#620 (That was probably because he experienced the same problem when running the scripted tests with sbt 1.5.x when he upgraded akka for the Play 2.8.11 release: https://app.travis-ci.com/github/playframework/playframework/jobs/549889172#L8473).
So basically the problem was solved for the 2.8.11 release already anyway.

I originally tried to fix this in #10969 four month ago, however back then I somehow got lost and also tried to upgrade scala-parser-combinators and also tried to bump play-ws, twirl and interplay to their next major versions, which actually isn't necessary. Looks like I just got confused.

…ption#TCP_FASTOPEN

mkurz · 2021-12-30T20:49:36Z

...plugin/src/sbt-test/play-sbt-plugin/http-backend-netty-channel-options/conf/application.conf

@@ -9,8 +9,8 @@ play.server {
        SO_KEEPALIVE = true
        TCP_NODELAY = true
        TCP_FASTOPEN_CONNECT = true
+        "io.netty.channel.ChannelOption#TCP_FASTOPEN" = 3


Because the cron build for the master branch was broken since month, it never reported that this scripted test actually is broken, also see #11076 (comment)

…'s/parboiled

mkurz · 2022-01-04T14:39:43Z

Blocked by akka/akka#30871 unfortunately.

mkurz · 2022-01-07T09:34:34Z

Will hopefully be unblocked by akka/akka#31046

mkurz · 2022-01-07T12:43:13Z

.travis.yml

@@ -29,7 +29,7 @@ install: jabba install $(jabba ls-remote "adopt@~1.$TRAVIS_JDK.0-0" --latest=pat
 stages:
  - validations
  - test
-  - cron-test-sbt-1.5.x
+  - test-sbt-1.6.x


IMHO we should run the scripted test for each pr build, not just daily, to catched errors as early as possible. Actually that is what we did until we dropped Scala 2.12: https://github.com/playframework/playframework/pull/10956/files#diff-6ac3f79fc25d95cd1e3d51da53a4b21b939437392578a35ae8cd6d5366ca5485L82
So probably I just forgot to change that back then.

mkurz · 2022-01-07T12:43:40Z

.travis.yml

+      workspaces:
+        use: published-local
+    - name: "Run scripted tests (c) for sbt 1.6.x and Scala 2.13.x"
+      script: scripts/test-scripted $SCRIPTED_SBT_1_6 $TEST_SCALA_2_13 'play-sbt-plugin/*3of3'


Make the scripted tests run in parallel again, like we used to do: https://github.com/playframework/playframework/pull/10956/files#diff-6ac3f79fc25d95cd1e3d51da53a4b21b939437392578a35ae8cd6d5366ca5485L82

mkurz · 2022-01-07T12:47:13Z

project/Dependencies.scala

@@ -7,16 +7,16 @@ import Keys._
 import buildinfo.BuildInfo

 object Dependencies {
-  val akkaVersion: String = sys.props.getOrElse("akka.version", "2.6.14")
-  val akkaHttpVersion     = sys.props.getOrElse("akka.http.version", "10.1.14")
+  val akkaVersion: String = sys.props.getOrElse("akka.version", "2.6.18+21-4fb7bd9b-SNAPSHOT")


Only temporary, let's use this akka snapshot because it contains akka/akka#31046 to unblock this PR.

mkurz · 2022-01-07T12:48:40Z

transport/client/play-ahc-ws/src/main/java/play/libs/ws/ahc/AhcWSRequest.java

+  public WSRequest setDisableUrlEncoding(boolean disableUrlEncoding) {
+    return converter.apply(request.setDisableUrlEncoding(disableUrlEncoding));
+  }
+


These methods were added as part of playframework/play-ws#552

mkurz · 2022-01-17T23:57:17Z

core/play-integration-test/src/it/scala/play/it/http/UriHandlingSpec.scala

    "handle '/pat/resources/BodhiApplication?where={%22name%22:%22hsdashboard%22}' as a valid URI" in makeRequest(
      "/pat/resources/BodhiApplication?where={%22name%22:%22hsdashboard%22}"
    ) {
      case (endpoint, response) => {
        response.body.string must_=== """/pat/resources/BodhiApplication?where={"name":"hsdashboard"}"""
      }
    }
+     */


For now I disable these it tests, because they are breaking after upgrading to akka-http 10.2.x.
To move on lets fix that later, opened #11113

mkurz · 2022-01-18T00:02:26Z

dev-mode/sbt-scripted-tools/src/main/scala/play/sbt/scriptedtools/ScriptedTools.scala

@@ -30,6 +30,7 @@ object ScriptedTools extends AutoPlugin {

  override def projectSettings: Seq[Def.Setting[_]] = Def.settings(
    resolvers += Resolver.sonatypeRepo("releases"), // sync BuildSettings.scala
+    resolvers += Resolver.sonatypeRepo("snapshots"),


We can remove that one later, see #11114. It's needed so scripted tests can fetch the akka snapshot.

mkurz · 2022-01-18T09:31:44Z

...lugin/src/sbt-test/play-sbt-plugin/evolutions-auto-apply-false/conf/evolutions/default/1.sql

@@ -3,7 +3,7 @@
 # --- !Ups

 CREATE TABLE users (
-    id bigint(20) NOT NULL AUTO_INCREMENT,
+    id bigint GENERATED BY DEFAULT AS IDENTITY,


Necessary because master branch upgraded h2 from 1.x to 2.x, see https://stackoverflow.com/questions/70695039/org-h2-jdbc-jdbcsqlsyntaxerrorexception-after-h2-version-upgrade/70696184

octonato

LGTM

This partially reverts akka#31046, but not the changes there that were needed to unblock playframework/playframework#11081

This partially reverts #31046, but not the changes there that were needed to unblock playframework/playframework#11081

mkurz added 6 commits December 30, 2021 21:03

Netty deprecated EpollChannelOption#TCP_FASTOPEN in favor of ChannelO…

a88e5b2

…ption#TCP_FASTOPEN

Run scripted tests not just for cron build but also PR builds

9b5e296

Split scripted tests (like in the past) to run parallel

5448499

Bump versions

c6cc56a

Pin FluentLenium and Selenium deps to Java 8 compatible versions

650d828

Bump versions

7a8e3dc

mkurz commented Dec 30, 2021

View reviewed changes

mkurz added 3 commits December 31, 2021 00:03

Remove whitesource

ca316dd

Bump interplay (to not cross build against Scala 2.12 anymore)

18b7395

Downgrade sbt-assembly because its ASM version conflicts with pegdown…

7e25cde

…'s/parboiled

This was referenced Jan 2, 2022

Replace pegdown with a maintained library playframework/play-doc#35

Open

scala-parser-combinators 2.0.0 (was 1.1.2) playframework/cachecontrol#167

Merged

mkurz added 4 commits January 4, 2022 01:43

play-ws 2.2.0-M1

c9320b3

Upgrade akka-http and play-json

7b15588

twirl 1.6.0-M1

ed266ca

sbt 1.6.1

2f426ff

mkurz mentioned this pull request Jan 4, 2022

Remove Whitesource checks (backport #11003) #11080

Closed

Correctly set sbt 1.6.x

7d8aa51

This was referenced Jan 4, 2022

Upgrade versions #10969

Closed

Update ssl-config-core 0.6.0 (was 0.4.3) to avoid conflicting scala-parser-combinators 2.x. akka/akka#30871

Closed

Update ssl-config-core 0.6.0 (was 0.4.3) akka/akka#31046

Merged

Use akka snapshot for now until 2.6.19 is released

7f251a8

mkurz commented Jan 7, 2022

View reviewed changes

Temporary disable UriHandlingSpec, we can fix that later

7faf24d

mkurz force-pushed the version_bumps branch from 8603b66 to 7faf24d Compare January 17, 2022 21:47

Temporary use sonatype snapshots also for non-cron builds

aca90c3

mkurz mentioned this pull request Jan 17, 2022

Fix failing tests after upgrade to akka-http 10.2.x #11113

Closed

mkurz commented Jan 17, 2022

View reviewed changes

mkurz mentioned this pull request Jan 18, 2022

Remove temporary sonatype snapshot resolver when akka 2.6.19 arrives #11114

Closed

mkurz commented Jan 18, 2022

View reviewed changes

Evolutions needed a fix after upgrading h2 to v2.0.202

efac1ce

mkurz commented Jan 18, 2022

View reviewed changes

Update interplay

eb6b750

octonato approved these changes Jan 18, 2022

View reviewed changes

mkurz added the status:merge-when-green label Jan 18, 2022

mergify bot merged commit 7c3954f into playframework:master Jan 18, 2022

mergify bot removed the status:merge-when-green label Jan 18, 2022

mkurz deleted the version_bumps branch January 18, 2022 11:49

mkurz mentioned this pull request Jan 23, 2022

Update akka-http-core, akka-http2-support to 10.2.x #10411

Closed

raboof added a commit to raboof/akka that referenced this pull request Mar 17, 2022

Revert: Update ssl-config-core 0.6.0 (was 0.4.3)

530e467

This partially reverts akka#31046, but not the changes there that were needed to unblock playframework/playframework#11081

raboof mentioned this pull request Mar 17, 2022

Revert: Update ssl-config-core 0.6.0 (was 0.4.3) akka/akka#31252

Merged

raboof added a commit to raboof/akka that referenced this pull request Mar 17, 2022

Revert: Update ssl-config-core 0.6.0 (was 0.4.3)

af71665

This partially reverts akka#31046, but not the changes there that were needed to unblock playframework/playframework#11081

raboof added a commit to akka/akka that referenced this pull request Mar 18, 2022

Revert: Update ssl-config-core 0.6.0 (was 0.4.3) (#31252)

af18e82

This partially reverts #31046, but not the changes there that were needed to unblock playframework/playframework#11081

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[master] Upgrade dependencies to fix CI cron build #11081

[master] Upgrade dependencies to fix CI cron build #11081

mkurz commented Dec 30, 2021

mkurz Dec 30, 2021

mkurz commented Jan 4, 2022

mkurz commented Jan 7, 2022

mkurz Jan 7, 2022

mkurz Jan 7, 2022

mkurz Jan 7, 2022

mkurz Jan 7, 2022

mkurz Jan 17, 2022 •

edited

mkurz Jan 18, 2022 •

edited

mkurz Jan 18, 2022

octonato left a comment

[master] Upgrade dependencies to fix CI cron build #11081

[master] Upgrade dependencies to fix CI cron build #11081

Conversation

mkurz commented Dec 30, 2021

mkurz Dec 30, 2021

Choose a reason for hiding this comment

mkurz commented Jan 4, 2022

mkurz commented Jan 7, 2022

mkurz Jan 7, 2022

Choose a reason for hiding this comment

mkurz Jan 7, 2022

Choose a reason for hiding this comment

mkurz Jan 7, 2022

Choose a reason for hiding this comment

mkurz Jan 7, 2022

Choose a reason for hiding this comment

mkurz Jan 17, 2022 • edited

Choose a reason for hiding this comment

mkurz Jan 18, 2022 • edited

Choose a reason for hiding this comment

mkurz Jan 18, 2022

Choose a reason for hiding this comment

octonato left a comment

Choose a reason for hiding this comment

mkurz Jan 17, 2022 •

edited

mkurz Jan 18, 2022 •

edited