Port Scalameta to Scala Native #772

DavidDudson · 2017-03-29T00:57:01Z

DavidDudson · 2017-03-29T00:58:40Z

@densh I'm sure you can answer a lot of these questions I have.

olafurpg · 2017-03-29T07:37:35Z

This was one big motivation for porting to Scala.js/crossProject. I already tested the waters by trying to build scalafmt against scala-native, which resulted in these linking errors scala-native/scala-native#557

densh · 2017-04-20T09:29:20Z

@olafurpg Most of the linking errors from that list are done now, we should retry and see if there is anything missing still.

olafurpg · 2017-04-22T17:07:06Z

@densh I just opened scalameta/scalafmt#898 porting scalafmt-core to Scala.js. We can work from that branch to try build scalafmt with native, I don't suspect we won't hit on any linking errors in core, but maybe we need more work in scalafmt-cli.

I will try to look into porting to native tomorrow.

olafurpg · 2017-04-27T11:49:26Z

I am super excited to confirm that scalafmt links with scala-native 0.2!

Getting it to work only required removing a single Enumeration in the scalameta parser since it seems those use runtime reflection. The rest was simple build changes.

xeno-by · 2017-04-27T13:06:37Z

Amazing news! Can we get these changes in? Is the entire test suite crosscompiling?

…

On Thu, 27 Apr 2017 at 04:49, Ólafur Páll Geirsson ***@***.***> wrote: I am super excited to confirm that scalafmt links with scala-native 0.2! [image: scalafmt-native] <https://cloud.githubusercontent.com/assets/1408093/25481965/f4d2da18-2b4f-11e7-91df-125f818ab82e.gif> Getting it to work only required removing a single Enumeration in the scalameta parser since it seems those use runtime reflection. The rest was simple build changes. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#772 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAlLgCh9wUlxInQ3iF4mT8u2UvC1JA1qks5r0IDHgaJpZM4MsX58> .

olafurpg · 2017-04-27T13:08:43Z

(Moved to description)

xeno-by · 2017-04-27T23:06:51Z

@densh Can you comment on what it would take to support these libraries?

densh · 2017-04-28T12:32:07Z

I have no data on scalatest as they have rather complicated build that's not easily migratable to native.

utest/sourcecode/fastparse seem to have enough APIs implemented in 0.2 already, it's just a matter of cross-publishing it, build changes are trivial. We can't run the tests automatically from the sbt yet due to scala-native/scala-native#339, but @Duhemm might have time to look at it in 0.3 milestone.

Scalacheck should be either working already or require minimum changes from us to support it.

olafurpg · 2017-08-13T16:32:55Z

Fastparse 0.4.4 is out now with native support. Only remaining dependency is ScalaTest. Our options are to either

don't run tests in native projects (booo!)
write a thin FunSuite wrapper using utests.

I lean towards the second option. utest assert macro error messages are IME better than scalatest macro assert errors so I wouldn't mind switching to utest. Once com-lihaoyi/utest#113 is merged then utest will also have pretty console reports.

ShaneDelmore · 2017-08-13T17:41:05Z

👍 for writing a FunSuite wrapper for uTest.

xeno-by · 2017-08-14T02:35:09Z

👍

xeno-by · 2018-01-23T18:51:05Z

I tried crosscompiling a small part of Scalameta to Native, and it worked pretty well, except for a minor inconvenienced described in #1228 (comment). All in all, this looks doable, and I'm looking forward to a native build of Scalatest to commence a full-scale migration.

olafurpg · 2018-01-23T23:07:35Z

scalatest 3.2 SNAP release is due out this week with native support! https://gitter.im/scalatest/scalatest?at=5a660edc517037a212dc18ee

xeno-by · 2018-01-23T23:18:55Z

\o/ Thank you, @bvenners!

densh · 2018-01-29T12:46:33Z

Issue with artifact names has been fixed by @sjrd (portable-scala/sbt-crossproject#81) and is going to be part of the next release of sbt-crossproject.

xeno-by · 2018-01-29T17:28:47Z

@densh Thanks for the great news! Is there a gitter channel or something where you guys announce new releases?

sjrd · 2018-01-29T17:33:07Z

I announced the release on Twitter a few hours ago ;)

xeno-by · 2018-02-01T19:26:57Z

I started working on crosscompilation to native in https://github.com/xeno-by/scalameta/tree/topic/native. Here's a problem that I ran into:

> metapNative/run /Users/eburmako/Projects/scalameta/sandbox/META-INF/semanticdb/Library.semanticdb
[info] Compiling 1 protobuf files to /Users/eburmako/Projects/scalameta/semanticdb/semanticdb3/.native/target/scala-2.11/src_managed/main
[info] Compiling schema /Users/eburmako/Projects/scalameta/semanticdb/semanticdb3/semanticdb3.proto
protoc-jar: protoc version: 340, detected platform: mac os x/x86_64
protoc-jar: executing: [/var/folders/pg/v06175sd2_qb9jbg7c28xr040000gn/T/protocjar4860536607164199621/bin/protoc.exe, --plugin=protoc-gen-scala=/var/folders/pg/v06175sd2_qb9jbg7c28xr040000gn/T/protocbridge5139572597291050554, --scala_out=flat_package,grpc:/Users/eburmako/Projects/scalameta/semanticdb/semanticdb3/.native/target/scala-2.11/src_managed/main, -I/Users/eburmako/Projects/scalameta/semanticdb/semanticdb3, -I/Users/eburmako/Projects/scalameta/semanticdb/semanticdb3/.native/target/protobuf_external, /Users/eburmako/Projects/scalameta/semanticdb/semanticdb3/semanticdb3.proto]
[info] Compiling protobuf
[info] Protoc target directory: /Users/eburmako/Projects/scalameta/semanticdb/semanticdb3/.native/target/scala-2.11/src_managed/main
[info] Compiling 9 Scala sources to /Users/eburmako/Projects/scalameta/semanticdb/semanticdb3/.native/target/scala-2.11/classes...
[info] Compiling 1 Scala source to /Users/eburmako/Projects/scalameta/semanticdb/metap/.native/target/scala-2.11/classes...
[info] Linking (1780 ms)
[error] cannot link: @scala.meta.internal.semanticdb3.SymbolInformation$Kind$CLASS$
[error] cannot link: @scala.meta.internal.semanticdb3.SymbolInformation$Kind$class$
[error] cannot link: @scala.meta.internal.semanticdb3.SymbolInformation$Kind$class$::$init$_scala.meta.internal.semanticdb3.SymbolInformation$Kind_unit
[error] unable to link
[error] (metapNative/compile:nativeLinkNIR) unable to link
[error] Total time: 14 s, completed Feb 1, 2018 11:23:32 AM

@densh, can you comment? Is this a native bug? If yes, is there a workaround that would allow me to ignore it and make progress regardless?

upd. Renaming SymbolInformation.Kind.CLASS to SymbolInformation.Kind.CLAZZ successfully works around the problem (xeno-by@b7a0691). We would still like to use the name CLASS, not CLAZZ, so it would be great if you could look into that.

upd. Submitted a ticket at scala-native/scala-native#1155.

xeno-by · 2018-02-01T19:45:23Z

Here's another problem that I ran into. This one may be related to ScalaTest:

> testsNative/test
[info] Compiling 1 protobuf files to /Users/eburmako/Projects/scalameta/semanticdb/semanticdb3/.native/target/scala-2.11/src_managed/main
...
[info] Compiling 1 Scala source to /Users/eburmako/Projects/scalameta/tests/native/target/scala-2.11/test-classes...
[error] /Users/eburmako/Projects/scalameta/tests/native/target/scala-2.11/src_managed/nativetest/FrameworksMap.scala:4: type ErrorSuite is not a member of package <root>
[error]   override val tests = Map[String, AnyRef]("scala.meta.tests.contrib.StatExtractionTest" -> new _root_.scala.meta.tests.contrib.StatExtractionTest, ...
[error] one error found
[error] (testsNative/nativetest:compileIncremental) Compilation failed
[error] Total time: 129 s, completed Feb 1, 2018 11:42:59 AM

@bvenners, any idea what could be causing the issue?

upd. Moving the test suite from the root package to any other package successfully works around the problem (xeno-by@44f57a0).

shadaj · 2018-02-01T22:40:25Z

Looks like https://github.com/scala-native/scala-native/blob/17cf3c557a10b63898b2fa4251b8a298177716df/sbt-scala-native/src/main/scala/scala/scalanative/sbtplugin/TestUtilities.scala#L38 is generating the tests map with _root_ prepended to every test class, which leads to the failure since _root_ can't be used to access values or classes in the root package.

@Duhemm would it be possible to generate the TestMain into the root package so that _root_ is not needed to reference test classes/modules?

xeno-by · 2018-02-02T00:22:23Z

I've just submitted a pull request with the first stab at crosscompiling Scalameta to Scala Native: #1243. I had to apply several workarounds, but I managed to get testsNative/test to green state. If you're interested, check out the pull request for details.

xeno-by · 2018-02-06T06:59:07Z

Here's another pull request that crosscompiles just semanticdb/: #1263, but without any workarounds and hacks.

xeno-by · 2018-02-07T01:19:42Z

Successfully merged #1263 and published a Scala Native build of semanticdb3. Already using it in Rsc: xeno-by/rsc@e1b8e2e.

olafurpg · 2018-05-11T06:51:24Z

Fixed in #1529, native support for scalameta + contrib modules has been merged into master 🚀

DavidDudson added the Build label Mar 29, 2017

xeno-by changed the title ~~Investigate crossbuilding with ScalaNative~~ Port scala.meta to Scala Native Mar 30, 2017

olafurpg changed the title ~~Port scala.meta to Scala Native~~ Port scalameta to Scala Native Aug 18, 2017

olafurpg mentioned this issue Aug 18, 2017

Git testing framework and fix for some git-file-fetching issues scalameta/scalafmt#1028

Merged

xeno-by mentioned this issue Jan 10, 2018

Support for Scala Native scalapb/ScalaPB#387

Closed

xeno-by changed the title ~~Port scalameta to Scala Native~~ Port Scalameta to Scala Native Jan 13, 2018

xeno-by mentioned this issue Jan 23, 2018

Factor out the SemanticDB schema into a separate project #1228

Merged

olafurpg closed this as completed May 11, 2018

marcgrue mentioned this issue Jul 9, 2019

Test suite on root level won't compile scala-native/scala-native#1647

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port Scalameta to Scala Native #772

Port Scalameta to Scala Native #772

DavidDudson commented Mar 29, 2017 •

edited by olafurpg

Loading

DavidDudson commented Mar 29, 2017

olafurpg commented Mar 29, 2017

densh commented Apr 20, 2017

olafurpg commented Apr 22, 2017

olafurpg commented Apr 27, 2017

xeno-by commented Apr 27, 2017 via email

olafurpg commented Apr 27, 2017 •

edited

Loading

xeno-by commented Apr 27, 2017

densh commented Apr 28, 2017 •

edited

Loading

olafurpg commented Aug 13, 2017

ShaneDelmore commented Aug 13, 2017

xeno-by commented Aug 14, 2017

xeno-by commented Jan 23, 2018 •

edited

Loading

olafurpg commented Jan 23, 2018

xeno-by commented Jan 23, 2018

densh commented Jan 29, 2018

xeno-by commented Jan 29, 2018

sjrd commented Jan 29, 2018

xeno-by commented Feb 1, 2018 •

edited

Loading

xeno-by commented Feb 1, 2018 •

edited

Loading

shadaj commented Feb 1, 2018

xeno-by commented Feb 2, 2018

xeno-by commented Feb 6, 2018

xeno-by commented Feb 7, 2018 •

edited

Loading

olafurpg commented May 11, 2018

Port Scalameta to Scala Native #772

Port Scalameta to Scala Native #772

Comments

DavidDudson commented Mar 29, 2017 • edited by olafurpg Loading

DavidDudson commented Mar 29, 2017

olafurpg commented Mar 29, 2017

densh commented Apr 20, 2017

olafurpg commented Apr 22, 2017

olafurpg commented Apr 27, 2017

xeno-by commented Apr 27, 2017 via email

olafurpg commented Apr 27, 2017 • edited Loading

xeno-by commented Apr 27, 2017

densh commented Apr 28, 2017 • edited Loading

olafurpg commented Aug 13, 2017

ShaneDelmore commented Aug 13, 2017

xeno-by commented Aug 14, 2017

xeno-by commented Jan 23, 2018 • edited Loading

olafurpg commented Jan 23, 2018

xeno-by commented Jan 23, 2018

densh commented Jan 29, 2018

xeno-by commented Jan 29, 2018

sjrd commented Jan 29, 2018

xeno-by commented Feb 1, 2018 • edited Loading

xeno-by commented Feb 1, 2018 • edited Loading

shadaj commented Feb 1, 2018

xeno-by commented Feb 2, 2018

xeno-by commented Feb 6, 2018

xeno-by commented Feb 7, 2018 • edited Loading

olafurpg commented May 11, 2018

DavidDudson commented Mar 29, 2017 •

edited by olafurpg

Loading

olafurpg commented Apr 27, 2017 •

edited

Loading

densh commented Apr 28, 2017 •

edited

Loading

xeno-by commented Jan 23, 2018 •

edited

Loading

xeno-by commented Feb 1, 2018 •

edited

Loading

xeno-by commented Feb 1, 2018 •

edited

Loading

xeno-by commented Feb 7, 2018 •

edited

Loading