Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to 2.13.0 #248

Open
wants to merge 13 commits into
base: develop
from
@@ -26,6 +26,7 @@ before_cache:
scala:
- 2.11.12
- 2.12.8
- 2.13.0

jdk:
- oraclejdk8
@@ -72,7 +72,7 @@ locally).
When appropriate, use [ScalaCheck][scalacheck] to write property-based
tests for your code. This will often produce more thorough and effective
inputs for your tests. We use ScalaTest's
[GeneratorDrivenPropertyChecks][gendrivenprop] as the entry point for
[ScalaCheckDrivenPropertyChecks][gendrivenprop] as the entry point for
writing these tests.

## Compatibility
@@ -12,17 +12,17 @@ val zkDependency = "org.apache.zookeeper" % "zookeeper" % zkVersion excludeAll(
ExclusionRule("javax.jms", "jms")
)
val slf4jVersion = "1.7.21"
val jacksonVersion = "2.9.8"
val jacksonVersion = "2.9.9"

This comment has been minimized.

Copy link
@yufangong

yufangong Jul 3, 2019

Contributor

Is this a necessary change along with this diff? or we can bump up the version separately?

This comment has been minimized.

Copy link
@martijnhoekstra

martijnhoekstra Jul 3, 2019

Author Contributor

There is no 2.9.8 for 2.13.0, so we have to bump it independently, we have to bump it before and rebase this on top of that.

I have a branch where that's done --develop...martijnhoekstra:pre213 is a diff of a reasonable bump of dependencies and elimination of deprecations that go along with that. But to be honest, I'm not sure what value that brings, other than better baseline for benchmarking maybe? I doubt it would matter much.


val guavaLib = "com.google.guava" % "guava" % "19.0"
val caffeineLib = "com.github.ben-manes.caffeine" % "caffeine" % "2.3.4"
val jsr305Lib = "com.google.code.findbugs" % "jsr305" % "2.0.1"
val scalacheckLib = "org.scalacheck" %% "scalacheck" % "1.13.4" % "test"
val scalacheckLib = "org.scalacheck" %% "scalacheck" % "1.14.0" % "test"
val slf4jApi = "org.slf4j" % "slf4j-api" % slf4jVersion

val defaultProjectSettings = Seq(
scalaVersion := "2.12.8",
crossScalaVersions := Seq("2.11.12", "2.12.8")
crossScalaVersions := Seq("2.11.12", "2.12.8", "2.13.0")
)

val baseSettings = Seq(
@@ -32,10 +32,19 @@ val baseSettings = Seq(
unmanagedClasspath in Compile += Attributed.blank(new java.io.File("doesnotexist")),
libraryDependencies ++= Seq(
// See https://www.scala-sbt.org/0.13/docs/Testing.html#JUnit
"org.scala-lang.modules" %% "scala-collection-compat" % "2.1.1",
"com.novocode" % "junit-interface" % "0.11" % "test",
"org.mockito" % "mockito-all" % "1.10.19" % "test",
"org.scalatest" %% "scalatest" % "3.0.0" % "test"
"org.scalatest" %% "scalatest" % "3.0.8" % "test"
),
fork in Test := true,
unmanagedSourceDirectories in Compile += {
val sourceDir = (sourceDirectory in Compile).value
CrossVersion.partialVersion(scalaVersion.value) match {
case Some((2, n)) if n >= 13 => sourceDir / "scala-2.13+"
case _ => sourceDir / "scala-2.12-"
}
},

ScoverageKeys.coverageHighlighting := true,
resolvers +=
@@ -48,7 +57,8 @@ val baseSettings = Seq(
"-feature",
"-encoding", "utf8",
// Needs -missing-interpolator due to https://issues.scala-lang.org/browse/SI-8761
"-Xlint:-missing-interpolator"
"-Xlint:-missing-interpolator",
"-Yrangepos"
),

// Note: Use -Xlint rather than -Xlint:unchecked when TestThriftStructure
@@ -216,7 +226,7 @@ lazy val utilCore = Project(
caffeineLib % "test",
scalacheckLib,
"org.scala-lang" % "scala-reflect" % scalaVersion.value,
"org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.4"
"org.scala-lang.modules" %% "scala-parser-combinators" % "1.1.2"
),
resourceGenerators in Compile += Def.task {
val projectName = name.value
@@ -368,7 +378,14 @@ lazy val utilStats = Project(
sharedSettings
).settings(
name := "util-stats",
libraryDependencies ++= Seq(caffeineLib, jsr305Lib, scalacheckLib)
libraryDependencies ++= Seq(caffeineLib, jsr305Lib, scalacheckLib) ++ {
CrossVersion.partialVersion(scalaVersion.value) match {
case Some((2, major)) if major >= 13 =>
Seq("org.scala-lang.modules" %% "scala-parallel-collections" % "0.2.0" % "test")
case _ =>
Seq()
}
}
).dependsOn(utilCore, utilLint)

lazy val utilTest = Project(
@@ -1 +1 @@
sbt.version=1.1.4
sbt.version=1.3.0-RC2
@@ -2,6 +2,5 @@ resolvers += Classpaths.sbtPluginReleases
resolvers += Resolver.sonatypeRepo("snapshots")

addSbtPlugin("com.typesafe.sbt" % "sbt-site" % "1.3.0")
addSbtPlugin("io.get-coursier" % "sbt-coursier" % "1.0.0-RC13")
addSbtPlugin("org.scoverage" % "sbt-scoverage" % "1.5.1")
addSbtPlugin("pl.project13.scala" % "sbt-jmh" % "0.3.4")
addSbtPlugin("org.scoverage" % "sbt-scoverage" % "1.6.0")
addSbtPlugin("pl.project13.scala" % "sbt-jmh" % "0.3.4")
@@ -313,7 +313,7 @@ trait App extends Closable with CloseAwaitably {
deadline: Time
): Future[Unit] = {
Future
.collectToTry(lastExits.asScala.toSeq.map(_.close(deadline)))
.collectToTry(lastExits.asScala.map(_.close(deadline)).toSeq)
.by(shutdownTimer, deadline)
.transform {
case Return(results) =>
@@ -6,6 +6,7 @@ import java.net.{URISyntaxException, URLClassLoader, URI}
import java.nio.charset.MalformedInputException
import java.util.jar.{JarEntry, JarFile}
import scala.collection.mutable
import scala.collection.mutable.Builder
import scala.collection.JavaConverters._
import scala.io.Source

@@ -51,84 +52,81 @@ private[app] sealed abstract class ClassPath[CpInfo <: ClassPath.Info] {
protected def ignoredPackages: Set[String]

def browse(loader: ClassLoader): Seq[CpInfo] = {
val buf = mutable.Buffer[CpInfo]()
val buf = Vector.newBuilder[CpInfo]
val seenUris = mutable.HashSet[URI]()

for ((uri, loader) <- getEntries(loader)) {
browseUri0(uri, loader, buf, seenUris)
seenUris += uri
}
buf
buf.result
}

// package protected for testing
private[app] def getEntries(loader: ClassLoader): Seq[(URI, ClassLoader)] = {
val ents = mutable.Buffer[(URI, ClassLoader)]()
val parent = loader.getParent
if (parent != null)
ents ++= getEntries(parent)

loader match {
case urlLoader: URLClassLoader =>
Option(urlLoader.getURLs) match {
case Some(urls) =>
urls.foreach { url =>
if (url != null)
ents += (url.toURI -> loader)
}
case _ =>
}
case _ =>
}
val parent = Option(loader.getParent)

val ownURIs: Vector[(URI, ClassLoader)] = for {
This conversation was marked as resolved by martijnhoekstra

This comment has been minimized.

Copy link
@martijnhoekstra

martijnhoekstra Apr 17, 2019

Author Contributor

simplified to get rid of the mutable buffer

urlLoader <- Vector(loader).collect {case u: URLClassLoader => u}
urls <- Option(urlLoader.getURLs()).toVector
url <- urls if url != null
} yield (url.toURI -> loader)

val p = parent.toSeq.flatMap(getEntries)

ownURIs ++ p

ents
}

// package protected for testing
private[app] def browseUri(uri: URI, loader: ClassLoader, buf: mutable.Buffer[CpInfo]): Unit =
private[app] def browseUri(uri: URI, loader: ClassLoader, buf: Builder[CpInfo, Seq[CpInfo]]): Unit =
This conversation was marked as resolved by martijnhoekstra

This comment has been minimized.

Copy link
@martijnhoekstra

martijnhoekstra Apr 17, 2019

Author Contributor

General pattern used to prevent returning a mutable buffer: use a Builder and return the immutable result.

browseUri0(uri, loader, buf, mutable.Set[URI]())

private[this] def browseUri0(
uri: URI,
loader: ClassLoader,
buf: mutable.Buffer[CpInfo],
buf: Builder[CpInfo, Seq[CpInfo]],
history: mutable.Set[URI]
): Unit = {
if (uri.getScheme != "file")
return

val f = new File(uri)
if (!(f.exists() && f.canRead))
return

if (f.isDirectory)
browseDir(f, loader, "", buf)
else
browseJar(f, loader, buf, history)
if (!history.contains(uri)) {

This comment has been minimized.

Copy link
@martijnhoekstra

martijnhoekstra Apr 17, 2019

Author Contributor

Done URI checking local to this function -- forking in test resulted in duplicate CpInfo's in the result

history.add(uri)
if (uri.getScheme != "file")
return

val f = new File(uri)
if (!(f.exists() && f.canRead))
return

if (f.isDirectory)
browseDir(f, loader, "", buf)
else
browseJar(f, loader, buf, history)
}
}

private[this] def browseDir(
dir: File,
loader: ClassLoader,
prefix: String,
buf: mutable.Buffer[CpInfo]
buf: Builder[CpInfo, Seq[CpInfo]]
): Unit = {
if (ignoredPackages.contains(prefix))
return

for (f <- dir.listFiles)
if (f.isDirectory && f.canRead)
if (f.isDirectory && f.canRead) {
browseDir(f, loader, prefix + f.getName + "/", buf)
}
else
processFile(prefix, f, buf)
}

protected def processFile(prefix: String, file: File, buf: mutable.Buffer[CpInfo]): Unit
protected def processFile(prefix: String, file: File, buf: Builder[CpInfo, Seq[CpInfo]]): Unit

private def browseJar(
file: File,
loader: ClassLoader,
buf: mutable.Buffer[CpInfo],
buf: Builder[CpInfo, Seq[CpInfo]],
seenUris: mutable.Set[URI]
): Unit = {
val jarFile = try new JarFile(file)
@@ -138,10 +136,7 @@ private[app] sealed abstract class ClassPath[CpInfo <: ClassPath.Info] {

try {
for (uri <- jarClasspath(file, jarFile.getManifest)) {
if (!seenUris.contains(uri)) {
seenUris += uri
browseUri0(uri, loader, buf, seenUris)
}
}

for {
@@ -161,16 +156,17 @@ private[app] sealed abstract class ClassPath[CpInfo <: ClassPath.Info] {
protected def processJarEntry(
jarFile: JarFile,
entry: JarEntry,
buf: mutable.Buffer[CpInfo]
buf: Builder[CpInfo, Seq[CpInfo]]
): Unit

private def jarClasspath(jarFile: File, manifest: java.util.jar.Manifest): Seq[URI] =
for {
m <- Option(manifest).toSeq
attr <- Option(m.getMainAttributes.getValue("Class-Path")).toSeq
el <- attr.split(" ")
el <- attr.split(" ").toSeq
uri <- uriFromJarClasspath(jarFile, el)
} yield uri


private def uriFromJarClasspath(jarFile: File, path: String): Option[URI] =
try {
@@ -196,7 +192,7 @@ private[app] class FlagClassPath extends ClassPath[ClassPath.FlagInfo] {
protected def processFile(
prefix: String,
file: File,
buf: mutable.Buffer[ClassPath.FlagInfo]
buf: Builder[ClassPath.FlagInfo, Seq[ClassPath.FlagInfo]]
): Unit = {
val name = file.getName
if (isClass(name)) {
@@ -207,7 +203,7 @@ private[app] class FlagClassPath extends ClassPath[ClassPath.FlagInfo] {
protected def processJarEntry(
jarFile: JarFile,
entry: JarEntry,
buf: mutable.Buffer[ClassPath.FlagInfo]
buf: Builder[ClassPath.FlagInfo, Seq[ClassPath.FlagInfo]]
): Unit = {
val name = entry.getName
if (isClass(name)) {
@@ -232,7 +228,7 @@ private[app] class LoadServiceClassPath extends ClassPath[ClassPath.LoadServiceI

private[app] def readLines(source: Source): Seq[String] = {
try {
source.getLines().toArray.flatMap { line =>
source.getLines().toVector.flatMap { line =>
val commentIdx = line.indexOf('#')
val end = if (commentIdx != -1) commentIdx else line.length
val str = line.substring(0, end).trim
@@ -248,7 +244,7 @@ private[app] class LoadServiceClassPath extends ClassPath[ClassPath.LoadServiceI
protected def processFile(
prefix: String,
file: File,
buf: mutable.Buffer[ClassPath.LoadServiceInfo]
buf: Builder[ClassPath.LoadServiceInfo, Seq[ClassPath.LoadServiceInfo]]
): Unit = {
for (iface <- ifaceOfName(prefix + file.getName)) {
val source = Source.fromFile(file, "UTF-8")
@@ -260,7 +256,7 @@ private[app] class LoadServiceClassPath extends ClassPath[ClassPath.LoadServiceI
protected def processJarEntry(
jarFile: JarFile,
entry: JarEntry,
buf: mutable.Buffer[ClassPath.LoadServiceInfo]
buf: Builder[ClassPath.LoadServiceInfo, Seq[ClassPath.LoadServiceInfo]]
): Unit = {
for (iface <- ifaceOfName(entry.getName)) {
val source = Source.fromInputStream(jarFile.getInputStream(entry), "UTF-8")
@@ -172,14 +172,14 @@ object Flaggable {
private[app] class SetFlaggable[T: Flaggable] extends Flaggable[Set[T]] {
private val flag = implicitly[Flaggable[T]]
assert(flag.default.isEmpty)
override def parse(v: String): Set[T] = v.split(",").map(flag.parse(_)).toSet
override def parse(v: String): Set[T] = v.split(",").iterator.map(flag.parse(_)).toSet
override def show(set: Set[T]): String = set.map(flag.show).mkString(",")
}

private[app] class SeqFlaggable[T: Flaggable] extends Flaggable[Seq[T]] {
private val flag = implicitly[Flaggable[T]]
assert(flag.default.isEmpty)
def parse(v: String): Seq[T] = v.split(",").map(flag.parse)
def parse(v: String): Seq[T] = v.split(",").toSeq.map(flag.parse)
override def show(seq: Seq[T]): String = seq.map(flag.show).mkString(",")
}

@@ -225,7 +225,7 @@ object Flaggable {
implicit def ofJavaList[T: Flaggable]: Flaggable[JList[T]] = new Flaggable[JList[T]] {
val seqFlaggable = new SeqFlaggable[T]
override def parse(v: String): JList[T] = seqFlaggable.parse(v).asJava
override def show(list: JList[T]): String = seqFlaggable.show(list.asScala)
override def show(list: JList[T]): String = seqFlaggable.show(list.asScala.toSeq)
}

implicit def ofJavaMap[K: Flaggable, V: Flaggable]: Flaggable[JMap[K, V]] = {
@@ -196,7 +196,7 @@ class Flags(argv0: String, includeGlobal: Boolean, failFastUntilParsed: Boolean)
if (helpFlag())
Help(usage)
else
Ok(remaining)
Ok(remaining.toSeq)
}

/**
@@ -379,7 +379,7 @@ class Flags(argv0: String, includeGlobal: Boolean, failFastUntilParsed: Boolean)
): (Iterable[String], Iterable[String]) = {
val (set, unset) = getAll(includeGlobal, classLoader).partition { _.get.isDefined }

(set.map { _ + " \\" }, unset.map { _ + " \\" })
(set.map { _.toString + " \\" }, unset.map { _.toString + " \\" })
}

/**
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.