Skip to content

[2.x] Remote caching support#7525

Merged
eed3si9n merged 6 commits into
sbt:developfrom
eed3si9n:wip/remoteapis
Apr 15, 2024
Merged

[2.x] Remote caching support#7525
eed3si9n merged 6 commits into
sbt:developfrom
eed3si9n:wip/remoteapis

Conversation

@eed3si9n

@eed3si9n eed3si9n commented Apr 5, 2024

Copy link
Copy Markdown
Member

This implements Bazel-compat remote caching support via a plugin. See https://eed3si9n.com/sbt-remote-cache-with-bazel-compat/ for details.

@eed3si9n
eed3si9n requested a review from adpi2 April 5, 2024 04:13
@eed3si9n
eed3si9n force-pushed the wip/remoteapis branch 2 times, most recently from 13ba037 to ebb6414 Compare April 5, 2024 05:23
@eed3si9n

eed3si9n commented Apr 5, 2024

Copy link
Copy Markdown
Member Author

Most test failures seem to have the stacktrace of

[info] [error] java.lang.IllegalArgumentException: No companions was stored!
[info] [error] 	at sbt.internal.inc.binary.converters.ProtobufDefaults$EmptyLazyCompanions$.get(ProtobufDefaults.scala:87)
[info] [error] 	at sbt.internal.inc.binary.converters.ProtobufDefaults$EmptyLazyCompanions$.get(ProtobufDefaults.scala:86)
[info] [error] 	at xsbti.api.AnalyzedClass.api(AnalyzedClass.java:76)
[info] [error] 	at sbt.internal.inc.binary.converters.ProtobufWriters.toAnalyzedClass(ProtobufWriters.scala:625)
[info] [error] 	at sbt.internal.inc.binary.converters.ProtobufWriters.$anonfun$8(ProtobufWriters.scala:726)
[info] [error] 	at sbt.internal.inc.binary.converters.ProtobufWriters.toApis$$anonfun$2(ProtobufWriters.scala:729)
[info] [error] 	at scala.collection.immutable.Map$Map1.foreach(Map.scala:258)
[info] [error] 	at sbt.internal.inc.binary.converters.ProtobufWriters.toApis(ProtobufWriters.scala:729)
[info] [error] 	at sbt.internal.inc.binary.converters.ProtobufWriters.toApisFile(ProtobufWriters.scala:740)
[info] [error] 	at sbt.internal.inc.binary.BinaryAnalysisFormat.writeAPIs(BinaryAnalysisFormat.scala:39)
[info] [error] 	at sbt.internal.inc.FileAnalysisStore$BinaryFileStore.set$$anonfun$1(FileAnalysisStore.scala:97)
[info] [error] 	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[info] [error] 	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[info] [error] 	at sbt.io.Using.apply(Using.scala:28)

Potentially some incompatibility due to protobuf?

@adpi2

adpi2 commented Apr 9, 2024

Copy link
Copy Markdown
Member

Potentially some incompatibility due to protobuf?

I got the same issue while working on #7522, and fixed it in 1c03463 by using a MixedAnalyzingCompiler.staticCachedStore instead of a FileAnalysisStore.binary. But then I removed that part of the code completely in 85fcb01.

@eed3si9n

eed3si9n commented Apr 9, 2024

Copy link
Copy Markdown
Member Author

Potentially some incompatibility due to protobuf?

I got the same issue while working on #7522

In sbt/zinc#1347 I originally tried to backport "Bump protobuf-java 3.25.3" to Zinc 1.x and realized some tests failed, so probably the incompatibility between protobuf-java 3.25.3 (and ScalaPB?) is the issue, which I did upgrade in the Zinc 2.x branch.

@adpi2

adpi2 commented Apr 11, 2024

Copy link
Copy Markdown
Member

Hey I rebased the changes to fix the conflicts and I force pushed. @eed3si9n You'll need to pull again if you want to change anything.

@adpi2 adpi2 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks quite good overall!

Comment thread main/src/main/scala/sbt/Defaults.scala
Comment thread main/src/main/scala/sbt/RemoteCache.scala
Comment thread main/src/main/scala/sbt/internal/Aggregation.scala
Comment thread util-cache/src/main/contraband/actionresult.contra
Comment thread util-cache/src/main/contraband/remotecache.contra
Comment thread util-cache/src/main/contraband/remotecache.contra
Comment thread util-cache/src/main/scala/sbt/util/ActionCacheStore.scala Outdated
eed3si9n and others added 2 commits April 11, 2024 10:45
Co-authored-by: adpi2 <adrien.piquerez@gmail.com>
Co-authored-by: adpi2 <adrien.piquerez@gmail.com>
@eed3si9n
eed3si9n merged commit facd7ab into sbt:develop Apr 15, 2024
@eed3si9n
eed3si9n deleted the wip/remoteapis branch April 15, 2024 22:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants