-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[2.x] Remove the Protocol Buffer usage #1388
Conversation
This PR switches the default format to the new consistent one, and the following tests are failing. Haven't dug into what's happening:
|
I wonder if we need to pretend that the compilation took place 1s after 2010-01-01T00:00:42Z for each round of incremental compilation entry /cc @szeiger |
3b0e7ee
to
c094e64
Compare
val singleOutput = setup.output().getSingleOutputAsPath() | ||
val outputPath = singleOutput match { | ||
case o if o.isPresent() && o.get().getFileName().toString().endsWith(".jar") => | ||
Analysis.dummyOutputJarPath | ||
case _ => Analysis.dummyOutputPath | ||
} | ||
out.string(outputPath.toString()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a bug fix, which we should backport to 1.x.
c50b915
to
df070ed
Compare
**Problem/Solution** We currently have three (3) implementations of Analysis persistence. Given that the "consistent" binary is the best, we should remove the protobuf one, so we don't need to update three code bases as we refactor for 2.x.
df070ed
to
8bc656d
Compare
@@ -11,8 +11,7 @@ | |||
|
|||
package sbt.internal.inc | |||
|
|||
import sbt.internal.prof.Zprof | |||
|
|||
// import sbt.internal.prof.Zprof |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you mean to leave all of this in as comments?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, I'm just going to comment out the protobuf usage, and later figure out what to do with the InvalidationProfiler instead of outright deleting it, if that's ok.
I am thinking if we should instead not rely on timestamp information in places such as In def detectAPIChanges(
recompiledClasses: collection.Set[String],
oldAPI: String => AnalyzedClass,
newAPI: String => AnalyzedClass
) already takes |
@Friendseeker For sbt 2.x I've implemented testQuick invalidation based on the file contents, and documented it as a blog post here - https://eed3si9n.com/sudori-part5/ |
Wow! So I definitely think the same idea can be used here! We don't even need to transitively compute the hash, but just the single class file hash is probably sufficient. I shall do some further investigation. Ideally we can avoid timestamp/hash comparison in the first place but if it is unavoidable, I guess we shall do class file hash comparison. |
I tried using the following patch on Index: internal/zinc-core/src/main/scala/sbt/internal/inc/IncrementalCommon.scala
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/internal/zinc-core/src/main/scala/sbt/internal/inc/IncrementalCommon.scala b/internal/zinc-core/src/main/scala/sbt/internal/inc/IncrementalCommon.scala
--- a/internal/zinc-core/src/main/scala/sbt/internal/inc/IncrementalCommon.scala (revision cbf85da6a065e37ac1b52359d33408af32c571fb)
+++ b/internal/zinc-core/src/main/scala/sbt/internal/inc/IncrementalCommon.scala (date 1726642366119)
@@ -309,8 +309,9 @@
// log.debug(s"[zinc] detectAPIChanges(recompiledClasses = $recompiledClasses)")
def classDiff(className: String, a: AnalyzedClass, b: AnalyzedClass): Option[APIChange] = {
// log.debug(s"[zinc] classDiff($className, ${a.name}, ${b.name})")
- if (a.compilationTimestamp() == b.compilationTimestamp() && (a.apiHash == b.apiHash)) None
- else {
+ if (a.compilationTimestamp() == b.compilationTimestamp() && (a.apiHash == b.apiHash)) {
+ throw new IllegalStateException("Timestamp should be always different")
+ } else {
val hasMacro = a.hasMacro || b.hasMacro
if (hasMacro && IncOptions.getRecompileOnMacroDef(options)) {
Some(APIChangeDueToMacroDefinition(className)) The following 6 scripted tests failed.
I will try to figure out why did the scripted test fail, but considering |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's great! If we can deprecate and remove the compilation timestamp it would be even better.
@eed3si9n Are you currently drafting a 2nd PR to get rid of the timestamp usage in If you are not working on it currently, I can work on it right now. Also should we backport the change to |
All incidences of failures are from here: zinc/internal/zinc-core/src/main/scala/sbt/internal/inc/IncrementalCommon.scala Lines 423 to 430 in 5a7012f
So the reason is that Zinc does not know whether an upstream source file is recompiled and needs to rely on timestamp to make the determination. Noteworthy fact is that we have at least 13 scripted tests featuring subproject and only 6 of these failed. I shall do more digging. I did more thinking and I think that we have to hash the class files. For downstream compilation. Zinc must know the exact set of upstream sources changed to perform initial invalidation. We cannot simply get rid of if else, as during upstream incremental compilation it can be pretty common for only a small subset of upstream sources to be recompiled and Zinc need to know. Looks like Stamper is already doing some content hashing. I shall investigate if the hashes already computed by Stamper suffices for the purpose. |
Problem/Solution
We currently have three (3) implementations of Analysis persistence. Given that the "consistent" binary (#1326) is the best, we should remove the protobuf one, so we don't need to update three code bases as we refactor for 2.x.