Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Fix for SerialVersionUID instability. #1673

Merged
merged 4 commits into from

4 participants

Paul Phillips scala-jenkins Mark Harrah som-snytt
Paul Phillips

Can hardly believe this has been broken for a decade or so, but
there it is - see test case. Four classes attempt to set their
SerialVersionUID to 13. One succeeds. No warnings or errors. The
output before this patch (for me anyway - your random numbers may
differ) was:

860336111422349646
13
8409527228024057943
-7852527872932878365

There was already code in place for rejecting annotations
with non-constant args when constant args are required, but
that check is only performed on ClassfileAnnotations, and
SerialVersionUID was a StaticAnnotation. Maybe people don't
reach for ClassfileAnnotation because of this giant warning
which I see no way to suppress:

warning: Implementation restriction: subclassing Classfile does
not make your annotation visible at runtime. If that is what you
want, you must write the annotation class in Java.

Why did I change the name of the field from uid to value?
If you don't use the name 'value', you have to name the argument
every time you use it, even if it's the only parameter. I didn't
relish breaking every usage of scala's @SerialVersionUID in the
known universe.

Paul Phillips paulp Fix for SerialVersionUID instability.
Can hardly believe this has been broken for a decade or so, but
there it is - see test case. Four classes attempt to set their
SerialVersionUID to 13. One succeeds. No warnings or errors. The
output before this patch (for me anyway - your random numbers may
differ) was:

  860336111422349646
  13
  8409527228024057943
  -7852527872932878365

There was already code in place for rejecting annotations
with non-constant args when constant args are required, but
that check is only performed on ClassfileAnnotations, and
SerialVersionUID was a StaticAnnotation. Maybe people don't
reach for ClassfileAnnotation because of this giant warning
which I see no way to suppress:

  warning: Implementation restriction: subclassing Classfile does
  not make your annotation visible at runtime. If that is what you
  want, you must write the annotation class in Java.

Why did I change the name of the field from uid to value?
If you don't use the name 'value', you have to name the argument
every time you use it, even if it's the only parameter. I didn't
relish breaking every usage of scala's @SerialVersionUID in the
known universe.
4267444
Paul Phillips

Opened against master because it's presumably binary-incompatible.

Paul Phillips

Review by anyone with a clue about serialization. @jsuereth, @lrytz ?

scala-jenkins

Started jenkins job pr-scala-testsuite-linux-opt at https://scala-webapps.epfl.ch/jenkins/job/pr-scala-testsuite-linux-opt/1735/

scala-jenkins

jenkins job pr-scala-testsuite-linux-opt: Failed - https://scala-webapps.epfl.ch/jenkins/job/pr-scala-testsuite-linux-opt/1735/

paulp added some commits
Paul Phillips paulp Eliminate some one-arg asserts.
The only thing more fun than debugging non-deterministic
scaladoc crashes unrelated to one's change is doing so with
all one-argument asserts.
a854529
Paul Phillips paulp Account for existence of scala's ClassfileAnnotation.
Apparently this thing is not real well tested, as the
scaladoc code was written as if it does not exist.
5573281
Paul Phillips

PLS REBUILD ALL

Paul Phillips paulp Disabled part of a test.
Hmmm, the giant blob of binary data embedded in a test
suddenly stopped working. What does one do in this spot.
b9e01a0
scala-jenkins

Started jenkins job pr-scala-testsuite-linux-opt at https://scala-webapps.epfl.ch/jenkins/job/pr-scala-testsuite-linux-opt/1736/

scala-jenkins

jenkins job pr-scala-testsuite-linux-opt: Success - https://scala-webapps.epfl.ch/jenkins/job/pr-scala-testsuite-linux-opt/1736/

Paul Phillips

Okay, review by @harrah, you must know something about serialization.

Mark Harrah

LGTM

Paul Phillips paulp merged commit 4b2330b into from
Jason Zaugg retronym referenced this pull request from a commit in retronym/scala
Jason Zaugg retronym SI-8459 Honour the @SerialVersionUID annotatation
In PR #1673 / 4267444, the annotation `SerialVersionId` was
changed from a `StaticAnnotation` to `ClassFileAnnotation` in
order to avoid silently ignoring non-literal UIDs like:

    @SerialVersionUID(0 - 12345L) class C

And to flag non-constant UIDs:

    @SerialVersionUID("!!!".length)

While this indeed was fold constants, the change was incomplete.
The compiler API for reading the argument from a `ClassFileAnnoation`
is different, on must look for a `LiteralAnnotArg`, rather than a
`Literal`.

This commit:

  - amends the backend accordingly
  - removes relevant duplication between `GenASM` and `GenBCode`
  - tests that the static field is generated accordingly

This will mean that we will break deserialization of objects from
Scalal 2.11.0 that use this annotation.
6a1b863
Jason Zaugg retronym referenced this pull request from a commit in retronym/scala
Jason Zaugg retronym SI-8549 Honour the @SerialVersionUID annotatation
In PR #1673 / 4267444, the annotation `SerialVersionId` was
changed from a `StaticAnnotation` to `ClassFileAnnotation` in
order to avoid silently ignoring non-literal UIDs like:

    @SerialVersionUID(0 - 12345L) class C

And to flag non-constant UIDs:

    @SerialVersionUID("!!!".length)

While this indeed was fold constants, the change was incomplete.
The compiler API for reading the argument from a `ClassFileAnnoation`
is different, on must look for a `LiteralAnnotArg`, rather than a
`Literal`.

This commit:

  - amends the backend accordingly
  - removes relevant duplication between `GenASM` and `GenBCode`
  - tests that the static field is generated accordingly

This will mean that we will break deserialization of objects from
Scalal 2.11.0 that use this annotation.
ecbc9d0
Paul Phillips

Did "I didn't relish breaking every usage of scala's @SerialVersionUID in the known universe" win any prestigious irony awards?

som-snytt

"Known universe" was too restrictive a domain, so it was actually up for Understatement. But the other comment that "it's presumably binary-incompatible" won one of those technical awards they give out the day before the big event which no one attends except of a couple of stringers. I think the category was, "Irony in an observation that applies to everything except this." But next year they're starting a special category to recognize binary incompatibility; the statuette shows one hand washing another, with the inscription on the base: "mea paulpa."

Lukas Rytz lrytz referenced this pull request from a commit in lrytz/scala
Lukas Rytz lrytz SI-8960 Bring back the SerialVersionUID to anonymous function classes
In PR #1673 / 4267444, the annotation `SerialVersionId` was
changed from a `StaticAnnotation` to `ClassFileAnnotation` in
order to enforce annotation arguments to be constants.

The ID value in the AnnotationInfo moved from `args` to `assocs`, but
the backend was not adjusted. This was fixed in PR #3711 / ecbc9d0.

Unfortunately, the synthetic AnnotationInfo that is added to anonymous
function classes still used the old constructor (`args` instead of
`assocs`), so extracting the value failed, and no field was added to
the classfile.
93f646e
Lukas Rytz lrytz referenced this pull request from a commit in lrytz/scala
Lukas Rytz lrytz SI-8960 Bring back the SerialVersionUID to anonymous function classes
In PR #1673 / 4267444, the annotation `SerialVersionId` was changed
from a `StaticAnnotation` to `ClassFileAnnotation` in order to enforce
annotation arguments to be constants. That was 2.11.0.

The ID value in the AnnotationInfo moved from `args` to `assocs`, but
the backend was not adjusted. This was fixed in PR #3711 / ecbc9d0 for
2.11.1.

Unfortunately, the synthetic AnnotationInfo that is added to anonymous
function classes still used the old constructor (`args` instead of
`assocs`), so extracting the value failed, and no field was added to
the classfile.
fdf60bb
Lukas Rytz lrytz referenced this pull request from a commit in lrytz/scala
Lukas Rytz lrytz SI-8960 Bring back the SerialVersionUID to anonymous function classes
In PR #1673 / 4267444, the annotation `SerialVersionId` was changed
from a `StaticAnnotation` to `ClassFileAnnotation` in order to enforce
annotation arguments to be constants. That was 2.11.0.

The ID value in the AnnotationInfo moved from `args` to `assocs`, but
the backend was not adjusted. This was fixed in PR #3711 / ecbc9d0 for
2.11.1.

Unfortunately, the synthetic AnnotationInfo that is added to anonymous
function classes still used the old constructor (`args` instead of
`assocs`), so extracting the value failed, and no field was added to
the classfile.
21a44fe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 26, 2012
  1. Paul Phillips

    Fix for SerialVersionUID instability.

    paulp authored
    Can hardly believe this has been broken for a decade or so, but
    there it is - see test case. Four classes attempt to set their
    SerialVersionUID to 13. One succeeds. No warnings or errors. The
    output before this patch (for me anyway - your random numbers may
    differ) was:
    
      860336111422349646
      13
      8409527228024057943
      -7852527872932878365
    
    There was already code in place for rejecting annotations
    with non-constant args when constant args are required, but
    that check is only performed on ClassfileAnnotations, and
    SerialVersionUID was a StaticAnnotation. Maybe people don't
    reach for ClassfileAnnotation because of this giant warning
    which I see no way to suppress:
    
      warning: Implementation restriction: subclassing Classfile does
      not make your annotation visible at runtime. If that is what you
      want, you must write the annotation class in Java.
    
    Why did I change the name of the field from uid to value?
    If you don't use the name 'value', you have to name the argument
    every time you use it, even if it's the only parameter. I didn't
    relish breaking every usage of scala's @SerialVersionUID in the
    known universe.
  2. Paul Phillips

    Eliminate some one-arg asserts.

    paulp authored
    The only thing more fun than debugging non-deterministic
    scaladoc crashes unrelated to one's change is doing so with
    all one-argument asserts.
  3. Paul Phillips

    Account for existence of scala's ClassfileAnnotation.

    paulp authored
    Apparently this thing is not real well tested, as the
    scaladoc code was written as if it does not exist.
  4. Paul Phillips

    Disabled part of a test.

    paulp authored
    Hmmm, the giant blob of binary data embedded in a test
    suddenly stopped working. What does one do in this spot.
This page is out of date. Refresh to see the latest.
36 src/compiler/scala/tools/nsc/doc/model/ModelFactory.scala
View
@@ -233,8 +233,8 @@ class ModelFactory(val global: Global, val settings: doc.Settings) {
* exists, but should not be documented (either it's not included in the source or it's not visible)
*/
class NoDocTemplateImpl(sym: Symbol, inTpl: TemplateImpl) extends EntityImpl(sym, inTpl) with TemplateImpl with HigherKindedImpl with NoDocTemplate {
- assert(modelFinished)
- assert(!(noDocTemplatesCache isDefinedAt sym))
+ assert(modelFinished, this)
+ assert(!(noDocTemplatesCache isDefinedAt sym), (sym, noDocTemplatesCache(sym)))
noDocTemplatesCache += (sym -> this)
def isDocTemplate = false
}
@@ -269,7 +269,7 @@ class ModelFactory(val global: Global, val settings: doc.Settings) {
* All ancestors of the template and all non-package members.
*/
abstract class DocTemplateImpl(sym: Symbol, inTpl: DocTemplateImpl) extends MemberTemplateImpl(sym, inTpl) with DocTemplateEntity {
- assert(!modelFinished)
+ assert(!modelFinished, (sym, inTpl))
assert(!(docTemplatesCache isDefinedAt sym), sym)
docTemplatesCache += (sym -> this)
@@ -620,7 +620,7 @@ class ModelFactory(val global: Global, val settings: doc.Settings) {
*/
def createTemplate(aSym: Symbol, inTpl: DocTemplateImpl): Option[MemberImpl] = {
// don't call this after the model finished!
- assert(!modelFinished)
+ assert(!modelFinished, (aSym, inTpl))
def createRootPackageComment: Option[Comment] =
if(settings.docRootContent.isDefault) None
@@ -636,7 +636,7 @@ class ModelFactory(val global: Global, val settings: doc.Settings) {
}
def createDocTemplate(bSym: Symbol, inTpl: DocTemplateImpl): DocTemplateImpl = {
- assert(!modelFinished) // only created BEFORE the model is finished
+ assert(!modelFinished, (bSym, inTpl)) // only created BEFORE the model is finished
if (bSym.isAliasType && bSym != AnyRefClass)
new DocTemplateImpl(bSym, inTpl) with AliasImpl with AliasType { override def isAliasType = true }
else if (bSym.isAbstractType)
@@ -842,24 +842,28 @@ class ModelFactory(val global: Global, val settings: doc.Settings) {
lazy val annotationClass =
makeTemplate(annot.symbol)
val arguments = { // lazy
- def noParams = annot.args map { _ => None }
+ def annotArgs = annot.args match {
+ case Nil => annot.assocs collect { case (_, LiteralAnnotArg(const)) => Literal(const) }
+ case xs => xs
+ }
+ def noParams = annotArgs map (_ => None)
+
val params: List[Option[ValueParam]] = annotationClass match {
case aClass: DocTemplateEntity with Class =>
(aClass.primaryConstructor map { _.valueParams.head }) match {
case Some(vps) => vps map { Some(_) }
- case None => noParams
+ case _ => noParams
}
case _ => noParams
}
- assert(params.length == annot.args.length)
- (params zip annot.args) flatMap { case (param, arg) =>
- makeTree(arg) match {
- case Some(tree) =>
- Some(new ValueArgument {
- def parameter = param
- def value = tree
- })
- case None => None
+ assert(params.length == annotArgs.length, (params, annotArgs))
+
+ params zip annotArgs flatMap { case (param, arg) =>
+ makeTree(arg) map { tree =>
+ new ValueArgument {
+ def parameter = param
+ def value = tree
+ }
}
}
}
2  src/library/scala/SerialVersionUID.scala
View
@@ -12,4 +12,4 @@ package scala
* Annotation for specifying the `static SerialVersionUID` field
* of a serializable class.
*/
-class SerialVersionUID(uid: Long) extends scala.annotation.StaticAnnotation
+class SerialVersionUID(value: Long) extends scala.annotation.ClassfileAnnotation
10 test/files/neg/serialversionuid-not-const.check
View
@@ -0,0 +1,10 @@
+serialversionuid-not-const.scala:1: error: annotation argument needs to be a constant; found: 13L.toLong
+@SerialVersionUID(13l.toLong) class C1 extends Serializable
+ ^
+serialversionuid-not-const.scala:3: error: annotation argument needs to be a constant; found: 13.asInstanceOf[Long]
+@SerialVersionUID(13.asInstanceOf[Long]) class C3 extends Serializable
+ ^
+serialversionuid-not-const.scala:4: error: annotation argument needs to be a constant; found: Test.bippy
+@SerialVersionUID(Test.bippy) class C4 extends Serializable
+ ^
+three errors found
16 test/files/neg/serialversionuid-not-const.scala
View
@@ -0,0 +1,16 @@
+@SerialVersionUID(13l.toLong) class C1 extends Serializable
+@SerialVersionUID(13l) class C2 extends Serializable
+@SerialVersionUID(13.asInstanceOf[Long]) class C3 extends Serializable
+@SerialVersionUID(Test.bippy) class C4 extends Serializable
+
+object Test {
+ val bippy = 13L
+
+ def show(c: Class[_]) = println(java.io.ObjectStreamClass.lookup(c).getSerialVersionUID)
+ def main(args: Array[String]): Unit = {
+ show(classOf[C1])
+ show(classOf[C2])
+ show(classOf[C3])
+ show(classOf[C4])
+ }
+}
3  test/files/run/t5374.check
View
@@ -2,5 +2,4 @@ ListBuffer(1, 2, 3, 1)
ListBuffer(1, 2, 3, 1)
ListBuffer()
List(1, 2, 3, 4, 5)
-List(1, 2, 3)
-ok
+ok
46 test/files/run/t5374.scala
View
@@ -7,15 +7,15 @@ import java.io._
object Test {
-
+
def main(args: Array[String]) {
ticketExample()
emptyListBuffer()
list()
- legacyList()
+ // legacyList()
objectWithMultipleLists()
}
-
+
def inAndOut[T <: AnyRef](obj: T): T = {
val baos = new ByteArrayOutputStream
val oos = new ObjectOutputStream(baos)
@@ -24,53 +24,53 @@ object Test {
val ois = new ObjectInputStream(bais)
ois.readObject.asInstanceOf[T]
}
-
+
def ticketExample() {
val lb = inAndOut(ListBuffer(1, 2, 3))
val lb2 = ListBuffer[Int]() ++= lb
-
+
lb2 ++= List(1)
lb ++= List(1)
println(lb)
println(lb2)
}
-
+
def emptyListBuffer() {
val lb = inAndOut(ListBuffer[Int]())
-
+
println(lb)
}
-
+
def list() {
val l = inAndOut(List(1, 2, 3, 4, 5))
-
+
println(l)
}
-
+
// this byte array corresponds to what List(1, 2, 3) used to be serialized to prior to this fix
val listBytes = Array[Byte](-84, -19, 0, 5, 115, 114, 0, 39, 115, 99, 97, 108, 97, 46, 99, 111, 108, 108, 101, 99, 116, 105, 111, 110, 46, 105, 109, 109, 117, 116, 97, 98, 108, 101, 46, 36, 99, 111, 108, 111, 110, 36, 99, 111, 108, 111, 110, -118, 92, 99, 91, -10, -40, -7, 109, 3, 0, 2, 76, 0, 43, 115, 99, 97, 108, 97, 36, 99, 111, 108, 108, 101, 99, 116, 105, 111, 110, 36, 105, 109, 109, 117, 116, 97, 98, 108, 101, 36, 36, 99, 111, 108, 111, 110, 36, 99, 111, 108, 111, 110, 36, 36, 104, 100, 116, 0, 18, 76, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 79, 98, 106, 101, 99, 116, 59, 76, 0, 2, 116, 108, 116, 0, 33, 76, 115, 99, 97, 108, 97, 47, 99, 111, 108, 108, 101, 99, 116, 105, 111, 110, 47, 105, 109, 109, 117, 116, 97, 98, 108, 101, 47, 76, 105, 115, 116, 59, 120, 112, 115, 114, 0, 17, 106, 97, 118, 97, 46, 108, 97, 110, 103, 46, 73, 110, 116, 101, 103, 101, 114, 18, -30, -96, -92, -9, -127, -121, 56, 2, 0, 1, 73, 0, 5, 118, 97, 108, 117, 101, 120, 114, 0, 16, 106, 97, 118, 97, 46, 108, 97, 110, 103, 46, 78, 117, 109, 98, 101, 114, -122, -84, -107, 29, 11, -108, -32, -117, 2, 0, 0, 120, 112, 0, 0, 0, 1, 115, 113, 0, 126, 0, 4, 0, 0, 0, 2, 115, 113, 0, 126, 0, 4, 0, 0, 0, 3, 115, 114, 0, 44, 115, 99, 97, 108, 97, 46, 99, 111, 108, 108, 101, 99, 116, 105, 111, 110, 46, 105, 109, 109, 117, 116, 97, 98, 108, 101, 46, 76, 105, 115, 116, 83, 101, 114, 105, 97, 108, 105, 122, 101, 69, 110, 100, 36, -118, 92, 99, 91, -9, 83, 11, 109, 2, 0, 0, 120, 112, 120)
-
- def legacyList() {
- val bais = new ByteArrayInputStream(listBytes)
- val ois = new ObjectInputStream(bais)
- val l = ois.readObject()
-
- println(l)
- }
-
+
+ // def legacyList() {
+ // val bais = new ByteArrayInputStream(listBytes)
+ // val ois = new ObjectInputStream(bais)
+ // val l = ois.readObject()
+
+ // println(l)
+ // }
+
class Foo extends Serializable {
val head = List(1, 2, 3)
val last = head.tail.tail
def structuralSharing: Boolean = head.tail.tail eq last
-
+
assert(structuralSharing)
}
-
+
def objectWithMultipleLists() {
val foo = inAndOut(new Foo)
-
+
if (foo.structuralSharing) println("ok")
else println("no structural sharing")
}
-
+
}
Something went wrong with that request. Please try again.