Support for Scala 2.12

[Bomme]

Hi,
I was wondering if you are planning to officially support Scala 2.12?
Scala 2.12 is out for quite some time now and a lot major libraries support it.
Any comment on this would be highly appreciated!

[Craigacp]

I attempted to do this and it basically involved updating a bunch of dependencies for the parsing and json infrastructure, and then three changes to marginal to remove a type it couldn't infer. Scala 2.12 does give a lot of warnings when compiling the current codebase, and the scalatest versions don't seem to be compatible. It's not too much work to do though.

[unoexperto]

Hi @Craigacp ! Any chance you could look into this issue please ?

[Craigacp]

It's a fairly straightforward change, but I don't understand the Marginal class so I'm not sure if the changes I tried are actually correct. One of the UMass people would need to work on it.

[Craigacp]

Java 9 only runs with Scala 2.12 and up (I tested 2.12.3, and the release of 2.12.0 I had before didn't work) so this is becoming more important. In addition to minor code changes, I think a bunch of the maven plugins need updated version numbers too, as older versions of Maven have issues with Java 9.

[andrewresearch]

I have forked factorie and created a cut down version for nlp use (removed command line, mongodb, and docs etcs), and also removed the custom build and replaced it with a standard SBT build. Too far away from this code base to offer as a pull request, but working for nlp with 2.12.3 nevertheless.
It is here: https://github.com/andrewresearch/nlpfactorie

[lvilnis]

Sorry for the delay. I made a branch that compiles under Scala 2.12.4, when I build with either the latest Maven or IntelliJ. But... it needs Java 8 (I just set my JAVA_HOME before running Maven to get things running quick.

https://github.com/factorie/factorie/tree/scala-2.12

Java 9 seems to have trouble compiling with the scala-maven-plugin, and I tried all kinds of versions of the plugin but still ran into the same NPE. Adam, if you got it working before maybe you could take a peek at this branch and see if you can make it build with Java 9?

[Craigacp]

Scala 2.12 requires Java 8, as it rewrites Scala's lambdas to use the new JVM support.

@lvilnis Turning off incremental compilation makes it compile fine under Java 9. It looks like zinc generates something weird when it tries to inspect the classpath (which has changed with the introduction of modules in Java 9). It might be due to the zinc interop code in scala-maven-plugin being quite old, looks like it hasn't been touched in years.

The pom file diff is:

diff --git a/pom.xml b/pom.xml
index f05b9516a..2e1bad706 100644
--- a/pom.xml
+++ b/pom.xml
@@ -279,7 +279,6 @@ limitations under the License.
           </execution>
         </executions>
         <configuration>
-          <recompileMode>incremental</recompileMode>
           <useZincServer>false</useZincServer>
           <scalaVersion>${scala.majorVersion}${scala.minorVersion}</scalaVersion>
           <args>
@@ -635,7 +634,6 @@ limitations under the License.
                   </execution>
                 </executions>
                 <configuration>
-                  <recompileMode>incremental</recompileMode>
                   <useZincServer>false</useZincServer>
                   <scalaVersion>${scala.majorVersion}${scala.minorVersion}</scalaVersion>
                 </configuration>

[lvilnis]

Thank you! That was killing me yesterday.

I'll ask around but I think having incremental compilation turned off in maven by default works for me (I use my IDE for incremental compilation, and maven I always do clean compiles to make packages and stuff). @strubell do you have thoughts on switching this to default no incremental comp for maven?

[strubell]

I actually don't think incremental compilation was/is even working for quite a long time, so it shouldn't be a problem.

…

On Fri, Nov 17, 2017 at 5:27 PM Luke Vilnis ***@***.***> wrote: Thank you! That was killing me yesterday. I'll ask around but I think having incremental compilation turned off in maven by default works for me (I use my IDE for incremental compilation, and maven I always do clean compiles to make packages and stuff). @strubell <https://github.com/strubell> do you have thoughts on switching this to default no incremental comp for maven? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#380 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADHZtypzj7f-lq9xhIsrKDhroCB5Acdfks5s3ghjgaJpZM4MQ3Hk> .

[lvilnis]

OK I'm gonna turn it off then.

[lvilnis]

So... I should probably merge this and get going with a release soon.

[unoexperto]

@lvilnis Hi Luke! Any progress with this ticket? Could you push new build to maven repo please ?

[unoexperto]

Also I can't compile master for 2.12.4 using cross-buildin

[error] Modules were resolved with conflicting cross-version suffixes in {file:/home/expert/work/sideprojects/test/factorie/}factorie:
[error]    org.scala-lang.modules:scala-xml _2.12, _2.11
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml
[error] Total time: 9 s, completed Mar 15, 2018 8:38:54 PM

Any ideas what causes it ?

I see following warning as well. Maybe related ?

[warn] Binary version (2.11) for dependency org.scala-lang#scala-compiler;2.11.8
[warn] 	in cc.factorie_2.11#factorie_2.12;1.3-SNAPSHOT differs from Scala binary version in project (2.12).

I was able to build after changing scala version in Build.scala but there are more missing dependencies. For example

[error] (*:update) sbt.ResolveException: unresolved dependency: org.json4s#json4s-jackson_2.12;3.2.9: not found
[error] (*:update) sbt.ResolveException: unresolved dependency: org.scalatest#scalatest_2.12;2.2.2: not found

Switching to 3.4.2 and 3.0.0 helps. Aaand then I get TONS of compilation errors

[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/SharedParsers.scala:16: object parsing is not a member of package util
[error] import scala.util.parsing.combinator._
[error]                   ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/SharedParsers.scala:18: not found: type RegexParsers
[error] private[parser] trait SharedParsers extends RegexParsers {
[error]                                             ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/SharedParsers.scala:29: not found: type Parser
[error]   implicit def r(reg: String): Parser[String] = regex(reg.r)
[error]                                ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/SharedParsers.scala:28: not found: type Parser
[error]   implicit def c(x: Char): Parser[Char] = accept(x)
[error]                            ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/DocumentParser.scala:54: value parseAll is not a member of object cc.factorie.app.bib.parser.DocumentParser.Impl
[error]     val res = Impl.parseAll(Impl.bibTex, input)
[error]                    ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/DocumentParser.scala:65: value ? is not a member of String
[error]     lazy val freeComment = "[^@]*".? ^^ (s => CommentEntry(s.getOrElse("")))
[error]                                    ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/DocumentParser.scala:83: not found: type Parser
[error]     def entryBody[T](parser: => Parser[T]): Parser[T] = {
[error]                                             ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/DocumentParser.scala:83: not found: type Parser
[error]     def entryBody[T](parser: => Parser[T]): Parser[T] = {
[error]                                 ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/DocumentParser.scala:85: value ~> is not a member of String
[error]       ("\\{\\s*" ~> p <~ "\\s*\\}") |
[error]                  ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/DocumentParser.scala:86: value ~> is not a member of String
[error]       ("\\(\\s*" ~> p <~ "\\s*\\)")
[error]                  ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/DocumentParser.scala:93: not found: type Parser
[error]     lazy val value: Parser[Value] = literalOrSymbol ~ ("\\s*#\\s*" ~> value).? ^^ {
[error]                     ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/DocumentParser.scala:102: value ^^ is not a member of String
[error]     lazy val numericLiteral = "\\d+(\\.\\d+)?" ^^ (Literal(_))
[error]                                                ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/SharedParsers.scala:24: not found: type Parser
[error]   lazy val BRACE_DELIMITED_STRING: Parser[String] =
[error]                                    ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/DocumentParser.scala:104: value ~> is not a member of Char
[error]       '"' ~> (BRACE_DELIMITED_STRING | """\\.""" | """[^"]""").* <~ '"' ^^ (xs => Literal(xs.mkString))
[error]           ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/SharedParsers.scala:22: not found: type Parser
[error]   lazy val BRACE_DELIMITED_STRING_NO_OUTER: Parser[String] =
[error]                                             ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/NameParser.scala:96: value parseAll is not a member of object cc.factorie.app.bib.parser.NameParser.NameLexer
[error]     NameLexer.parseAll(NameLexer.nameLexer, namesString).getOrElse(Nil)
[error]               ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/NameParser.scala:122: value ^^ is not a member of String
[error]     lazy val and = "and" ^^ (_ => AND)
[error]                          ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/NameParser.scala:123: value ^^ is not a member of String
[error]     lazy val comma = "," ^^ (_ => COMMA)
[error]                          ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/NameParser.scala:124: value | is not a member of String
[error]     lazy val hyphen = ("-" | "~") ^^ (_ => HYPHEN)
[error]                            ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/NameParser.scala:125: value ^^ is not a member of String
[error]     lazy val initial = "[A-Za-z]\\." ^^ (FRAGMENT(_))
[error]                                      ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/SharedParsers.scala:25: value ~> is not a member of Char
[error]     '{' ~> (BRACE_DELIMITED_STRING | """\\.""" | """[^}{]""").* <~ '}' ^^
[error]         ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/SharedParsers.scala:28: not found: value accept
[error]   implicit def c(x: Char): Parser[Char] = accept(x)
[error]                                           ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/bib/parser/SharedParsers.scala:29: not found: value regex
[error]   implicit def r(reg: String): Parser[String] = regex(reg.r)
[error]                                                 ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:23: object parsing is not a member of package util
[error] import scala.util.parsing.combinator.{ImplicitConversions, Parsers}
[error]                   ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:47: not found: type Parsers
[error] class DatePhraseFinder(usePosTag:Boolean) extends DocumentAnnotator with Parsers with ImplicitConversions {
[error]                                                                          ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:47: not found: type ImplicitConversions
[error] class DatePhraseFinder(usePosTag:Boolean) extends DocumentAnnotator with Parsers with ImplicitConversions {
[error]                                                                                       ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:24: object parsing is not a member of package util
[error] import scala.util.parsing.input.{Position, Reader}
[error]                   ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:92: not found: type Parser
[error]   implicit def hasString(stringRegex: String): Parser[Token] = acceptIf(_.string.matches(stringRegex))(err)
[error]                                                ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:65: not found: type Parser
[error]   val monthAbbr: Parser[Token] = acceptIf(_.string.toLowerCase.matches(monthAbbrRegex.toLowerCase))(err)
[error]                  ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:65: not found: value acceptIf
[error]   val monthAbbr: Parser[Token] = acceptIf(_.string.toLowerCase.matches(monthAbbrRegex.toLowerCase))(err)
[error]                                  ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:67: not found: type Parser
[error]   val monthParser: Parser[Token] = acceptIf(_.string.toLowerCase.matches(monthRegex.toLowerCase))(err)
[error]                    ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:67: not found: value acceptIf
[error]   val monthParser: Parser[Token] = acceptIf(_.string.toLowerCase.matches(monthRegex.toLowerCase))(err)
[error]                                    ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:68: not found: type Parser
[error]   val monthNumber: Parser[Token] = "0?[1-12]"
[error]                    ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:69: not found: type Parser
[error]   val monthDayNumber: Parser[(Token, Int)] = hasString(dayOfMonthRegex) ^^ { case dayToken =>
[error]                       ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:73: not found: type Parser
[error]   val weekDayAbbr: Parser[Token] = acceptIf(_.string.toLowerCase.matches("(Mon|Tue|Tues|Wed|Thu|Thurs|Fri)\\.?".toLowerCase))(err)
[error]                    ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:73: not found: value acceptIf
[error]   val weekDayAbbr: Parser[Token] = acceptIf(_.string.toLowerCase.matches("(Mon|Tue|Tues|Wed|Thu|Thurs|Fri)\\.?".toLowerCase))(err)
[error]                                    ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:74: not found: type Parser
[error]   val weekDay: Parser[(Token, Int)] = (weekDayAbbr | acceptIf(_.string.toLowerCase.matches("Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday".toLowerCase))(err)) ^^ {
[error]                ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:78: not found: type Parser
[error]   val temporalPreps: Parser[Token] = hasLemma("in|from|to|until|since")
[error]                      ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:90: not found: type Parser
[error]   def hasLemma(lemmaRegex: String): Parser[Token] = acceptIf(_.lemmaString.matches(lemmaRegex))(err)
[error]                                     ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:79: not found: type Parser
[error]   val digits: Parser[(Token, Int)] = acceptIf(_.isDigits)(err) ^^ { case t => (t, t.string.toInt)}
[error]               ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:79: not found: value acceptIf
[error]   val digits: Parser[(Token, Int)] = acceptIf(_.isDigits)(err) ^^ { case t => (t, t.string.toInt)}
[error]                                      ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:80: not found: type Parser
[error]   val bcAd: Parser[Token] = "B\\.?C\\.?|A\\.?D\\.?"
[error]             ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:81: not found: type Parser
[error]   val year: Parser[(Token, Int)] = bcAd.? ~ digits ~ bcAd.? ^^ { case bcAdOpt1 ~ y ~ bcAdOpt2 =>
[error]             ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:90: not found: value acceptIf
[error]   def hasLemma(lemmaRegex: String): Parser[Token] = acceptIf(_.lemmaString.matches(lemmaRegex))(err)
[error]                                                     ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:92: not found: value acceptIf
[error]   implicit def hasString(stringRegex: String): Parser[Token] = acceptIf(_.string.matches(stringRegex))(err)
[error]                                                                ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:98: not found: type Parser
[error]   val onlyMonth: Parser[(Token, Int)] = (monthParser | monthAbbr) ^^ { case monthToken => (monthToken, DatePhraseFinder.monthToNr(monthToken.string.substring(0, 3).toLowerCase))}
[error]                  ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:99: not found: type Parser
[error]   val monthOnly: Parser[DatePhrase] = onlyMonth ^^ { case m => new DatePhrase(m._1, month = m._2)}
[error]                  ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:117: value ^^ is not a member of String
[error]   val yearMonthDayFromTokenNr = "(19|20)?[0-9]{2}[\\-/][0-3]?[0-9][\\-/][0-3]?[0-9]" ^^ { case ymdToken =>
[error]                                                                                      ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:124: value ^^ is not a member of String
[error]   val monthDayYearFromTokenNr = "0?[1-12][\\-/][0-3]?[0-9][\\-/][0-9]{2,}" ^^ { case mdyToken =>
[error]                                                                            ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:189: not found: type Parser
[error]   val parser: Parser[DatePhrase] = dayMonthYear | monthDayYear | yearMonthDay | yearMonthDayAdap | yearMonthDayString | dayMonthYearAdap | dayMonthYearString | monthDayYearFromTokenNr | yearMonthDayFromTokenNr | yearOnly | monthOnly
[error]               ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:191: object parsing is not a member of package util
[error]   def reader(ts: Iterable[Token]): scala.util.parsing.input.Reader[Token] = new scala.util.parsing.input.Reader[Token] {
[error]                                               ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:191: object parsing is not a member of package util
[error]   def reader(ts: Iterable[Token]): scala.util.parsing.input.Reader[Token] = new scala.util.parsing.input.Reader[Token] {
[error]                                                                                            ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:196: not found: type Position
[error]     override def pos: Position = new Position {
[error]                       ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:196: not found: type Position
[error]     override def pos: Position = new Position {
[error]                                      ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:204: not found: type Reader
[error]     override def rest: Reader[Token] = if (atEnd) reader(ts) else reader(ts.tail)
[error]                        ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:213: not found: value Success
[error]         case Success(s, rest) =>
[error]              ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:216: not found: value Failure
[error]         case Failure(_, rest) => r = rest.rest
[error]              ^
[error] /home/expert/work/sideprojects/test/factorie/src/main/scala/cc/factorie/app/nlp/phrase/DatePhraseFinder.scala:217: object Error is not a case class, nor does it have an unapply/unapplySeq member
[error]         case Error(_, rest) => r = rest.rest
[error]              ^
[error] 58 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 61 s, completed Mar 15, 2018 9:32:21 PM

which was fixed by adding

"org.scala-lang.modules" %% "scala-parser-combinators" % "1.1.0"

to dependencies.

[unoexperto]

Guys, I published builds of master branch for 2.11 and 2.12 if somebody needs them. Add Resolver.bintrayRepo("cppexpert", "maven") to your resolvers. And "cc.factorie" %% "factorie" % "1.3-master-20180316" to dependencies.

@lvilnis Overall build script needs some cleaning for cross-builds to work.