Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port sourcecode to Dotty #80

Merged
merged 33 commits into from Dec 15, 2019
Merged
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
57e0518
Add dotty support
alexarchambault Nov 27, 2018
109fc9b
Tweak dotty-specific source directory (#62)
alexarchambault Nov 28, 2018
7ed1a5a
Switch to dotty 0.12 (nightly)
alexarchambault Nov 28, 2018
b62b1e9
Fix symbol literals
liufengyun Jan 23, 2019
0359aea
Use new splice syntax
nicolasstucki Feb 19, 2019
5820d0e
Use new quote syntax
nicolasstucki Feb 19, 2019
1a36bc6
Fix splice
nicolasstucki Feb 19, 2019
347c6a2
Use new symbol API
nicolasstucki Mar 20, 2019
3be0218
Update soucre file API
nicolasstucki Mar 25, 2019
6d01f18
Avoid using toolbox inside a macro
nicolasstucki Apr 1, 2019
c556b0f
Import the full scala.quoted package to have extension methods
nicolasstucki May 9, 2019
68e029f
Avoid using internal API
nicolasstucki Jun 5, 2019
77a35e9
Add Dotty support to the Mill project
anatoliykmetyuk Nov 11, 2019
86affd4
Split macros into separate sources for Scala 2 and Scala 3
anatoliykmetyuk Nov 11, 2019
340c5f2
WIP: update Dotty sources to the latest Dotty version
anatoliykmetyuk Nov 11, 2019
698d2f1
FileName macro implemented
anatoliykmetyuk Nov 12, 2019
7818a3f
Apply tests pass
anatoliykmetyuk Nov 12, 2019
a8874ed
Implicits tests pass
anatoliykmetyuk Nov 12, 2019
7cb30fb
Debug println removed
anatoliykmetyuk Nov 12, 2019
a3cfdb9
myLazy tests run on Dotty
anatoliykmetyuk Nov 13, 2019
f480cc1
Regressions test works on Dotty
anatoliykmetyuk Nov 13, 2019
affced6
Fix TextTests to run on Dotty
anatoliykmetyuk Nov 13, 2019
a18d9db
Adapt mill build to Dotty
anatoliykmetyuk Nov 13, 2019
9d8a4b1
Adapt mill build to Dotty
anatoliykmetyuk Nov 13, 2019
49c20bd
Disable javadoc publishing for dotty as mill does not support it yet
anatoliykmetyuk Nov 20, 2019
7a84bb9
Merge branch 'dotty-mill' of https://github.com/dotty-staging/sourcec…
anatoliykmetyuk Nov 26, 2019
26e2d95
Disable doc jars only for Dotty
anatoliykmetyuk Nov 26, 2019
a9ceadf
Fix TextTests non-dotty tests
anatoliykmetyuk Nov 26, 2019
db11447
Use local mill script for testing
anatoliykmetyuk Nov 26, 2019
719ebfe
FileName fully cleans synthetic prefixes and suffixes
anatoliykmetyuk Dec 9, 2019
71b875c
Follow ruby conventions in Scala 3
anatoliykmetyuk Dec 10, 2019
7177d37
Idiomatically check whether the symbol exists
anatoliykmetyuk Dec 10, 2019
6ecc0c5
Use Dotty Nightly build
anatoliykmetyuk Nov 26, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

@@ -12,4 +12,4 @@ script:
- curl -L -o ~/bin/mill https://github.com/lihaoyi/mill/releases/download/0.4.0/0.4.0-12-102ddf && chmod +x ~/bin/mill
- curl https://raw.githubusercontent.com/scala-native/scala-native/master/scripts/travis_setup.sh | bash -x
- export PATH=~/bin/mill:$PATH
- mill __.test.run
- ./mill __.test.run
@@ -25,10 +25,12 @@ trait SourcecodeMainModule extends CrossScalaModule {

def offset: os.RelPath = os.rel

def compileIvyDeps = Agg(
ivy"org.scala-lang:scala-reflect:${scalaVersion()}",
ivy"org.scala-lang:scala-compiler:${scalaVersion()}"
)
def compileIvyDeps =
if (crossScalaVersion.startsWith("2")) Agg(
ivy"org.scala-lang:scala-reflect:${crossScalaVersion}",
ivy"org.scala-lang:scala-compiler:${crossScalaVersion}"
)
else Agg.empty[Dep]

def sources = T.sources(
super.sources()
@@ -62,7 +64,7 @@ trait SourcecodeTestModule extends ScalaModule {
}

object sourcecode extends Module {
object jvm extends Cross[JvmSourcecodeModule]("2.11.12", "2.12.8", "2.13.0")
object jvm extends Cross[JvmSourcecodeModule]("2.11.12", "2.12.8", "2.13.0", "0.21.0-bin-20191125-a64725c-NIGHTLY")
class JvmSourcecodeModule(val crossScalaVersion: String)
extends SourcecodeMainModule with ScalaModule with SourcecodeModule {

@@ -71,6 +73,15 @@ object sourcecode extends Module {
def moduleDeps = Seq(JvmSourcecodeModule.this)
val crossScalaVersion = JvmSourcecodeModule.this.crossScalaVersion
}

override def docJar =
if (crossScalaVersion.startsWith("2")) super.docJar
else T {
val outDir = T.ctx().dest
val javadocDir = outDir / 'javadoc
os.makeDir.all(javadocDir)
mill.api.Result.Success(mill.modules.Jvm.createJar(Agg(javadocDir))(outDir))
}
This conversation was marked as resolved by anatoliykmetyuk

This comment has been minimized.

Copy link
@lihaoyi

lihaoyi Nov 28, 2019

Owner

This is fine. I have successfully published empty docjars to maven central before. As long as a .jar file is provided, it works

}

object js extends Cross[JsSourcecodeModule](
37 mill
@@ -0,0 +1,37 @@
#!/usr/bin/env sh

# This is a wrapper script, that automatically download mill from GitHub release pages
# You can give the required mill version with MILL_VERSION env variable
# If no version is given, it falls back to the value of DEFAULT_MILL_VERSION
DEFAULT_MILL_VERSION=0.5.0

set -e

if [ -z "$MILL_VERSION" ] ; then
if [ -f ".mill-version" ] ; then
MILL_VERSION="$(head -n 1 .mill-version 2> /dev/null)"
elif [ -f "mill" ] && [ "$BASH_SOURCE" != "mill" ] ; then
MILL_VERSION=$(grep -F "DEFAULT_MILL_VERSION=" "mill" | head -n 1 | cut -d= -f2)
else
MILL_VERSION=$DEFAULT_MILL_VERSION
fi
fi

MILL_DOWNLOAD_PATH="$HOME/.mill/download"
MILL_EXEC_PATH="${MILL_DOWNLOAD_PATH}/$MILL_VERSION"

if [ ! -x "$MILL_EXEC_PATH" ] ; then
mkdir -p $MILL_DOWNLOAD_PATH
DOWNLOAD_FILE=$MILL_EXEC_PATH-tmp-download
MILL_DOWNLOAD_URL="https://github.com/lihaoyi/mill/releases/download/${MILL_VERSION%%-*}/$MILL_VERSION-assembly"
curl --fail -L -o "$DOWNLOAD_FILE" "$MILL_DOWNLOAD_URL"
chmod +x "$DOWNLOAD_FILE"
mv "$DOWNLOAD_FILE" "$MILL_EXEC_PATH"
unset DOWNLOAD_FILE
unset MILL_DOWNLOAD_URL
fi

unset MILL_DOWNLOAD_PATH
unset MILL_VERSION

exec $MILL_EXEC_PATH "$@"
@@ -0,0 +1,7 @@
package sourcecode

object TestUtil {

val isDotty = false

}
@@ -0,0 +1,17 @@
package sourcecode

object TestUtil {

// FIXME In dotty, scala.util.Properties.versionNumberString is still like 2.12.x
lazy val isDotty = {
val cl: ClassLoader = Thread.currentThread().getContextClassLoader
try {
cl.loadClass("dotty.DottyPredef")
true
} catch {
case _: ClassNotFoundException =>
false
}
}

}
@@ -0,0 +1,7 @@
package sourcecode

object TestUtil {

val isDotty = false

}
@@ -0,0 +1,254 @@
package sourcecode

import scala.language.implicitConversions
import scala.quoted._
import scala.tasty.Reflection

trait NameMacros {
inline implicit def generate: Name =
${ Macros.nameImpl }
}

trait NameMachineMacros {
inline implicit def generate: Name.Machine =
${ Macros.nameMachineImpl }
}

trait FullNameMacros {
inline implicit def generate: FullName =
${ Macros.fullNameImpl }
}

trait FullNameMachineMacros {
inline implicit def generate: FullName.Machine =
${ Macros.fullNameMachineImpl }
}

trait FileMacros {
inline implicit def generate: sourcecode.File =
${ Macros.fileImpl }
}

trait FileNameMacros {
inline implicit def generate: sourcecode.FileName =
${ Macros.fileNameImpl }
}

trait LineMacros {
inline implicit def generate: sourcecode.Line =
${ Macros.lineImpl }
}

trait EnclosingMacros {
inline implicit def generate: Enclosing =
${ Macros.enclosingImpl }
}

trait EnclosingMachineMacros {
inline implicit def generate: Enclosing.Machine =
${ Macros.enclosingMachineImpl }
}

trait PkgMacros {
inline implicit def generate: Pkg =
${ Macros.pkgImpl }
}

trait TextMacros {
inline implicit def generate[T](v: => T): Text[T] = ${ Macros.text('v) }
inline def apply[T](v: => T): Text[T] = ${ Macros.text('v) }
}

trait ArgsMacros {
inline implicit def generate: Args =
${ Macros.argsImpl }
}

object Util{
def isSynthetic(c: Reflection)(s: c.Symbol) = isSyntheticName(getName(c)(s))
def isSyntheticName(name: String) = {
name == "<init>" || (name.startsWith("<local ") && name.endsWith(">"))
}
def getName(c: Reflection)(s: c.Symbol) = {
import c.given
s.name.trim
.stripSuffix("$") // meh
}
}

object Macros {

def actualOwner(c: Reflection)(owner: c.Symbol): c.Symbol = {
import c.given
var owner0 = owner
// second condition is meh
while(Util.isSynthetic(c)(owner0) || Util.getName(c)(owner0) == "ev") {
This conversation was marked as resolved by anatoliykmetyuk

This comment has been minimized.

Copy link
@liufengyun

liufengyun Dec 3, 2019

Contributor

simpler: owner0.flags.is(Flags.Synthetic), but it may not work, as the semantics for Util.isSynthetic seem to be special.

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Dec 10, 2019

Author Collaborator

I tried it, didn't work

owner0 = owner0.owner
}
owner0
}

def nameImpl(given ctx: QuoteContext): Expr[Name] = {
import ctx.tasty.given
val owner = actualOwner(ctx.tasty)(ctx.tasty.rootContext.owner)
val simpleName = Util.getName(ctx.tasty)(owner)
'{Name(${simpleName.toExpr})}

This comment has been minimized.

Copy link
@lihaoyi

lihaoyi Nov 28, 2019

Owner

Is the reference to free terms like Name in these splices hygienic? We use the ${c.prefix} incantation in the Scala 2.x implementation to ensure that, not sure if somethign similar is required in Dotty

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Nov 29, 2019

Author Collaborator

As far as my understanding of hygiene in Scala 2 macros goes, in Scala 3 it is fine. Scala 3 quotes are compiled on definition site wheres Scala 2 quasiquotes are compiled on expansion site. So in Scala 2, if e.g. you use Name instead of ${c.prefix} and comment out package sourcecode in Apply.scala tests, you'll get a bunch of errors saying that Name is not found. Whereas it works with the package sourcecode uncommented. In Scala 3, the context of expansion is irrelevant since the quote is already compiled. @nicolasstucki maybe you can provide more precise info on this.

}

private def adjustName(s: String): String =
// Required to get the same name from dotty
if (s.startsWith("<local ") && s.endsWith("$>"))
s.stripSuffix("$>") + ">"
else
s

def nameMachineImpl(given ctx: QuoteContext): Expr[Name.Machine] = {
import ctx.tasty.given
val owner = ctx.tasty.rootContext.owner
val simpleName = adjustName(Util.getName(ctx.tasty)(owner))
'{Name.Machine(${simpleName.toExpr})}
}

def fullNameImpl(given ctx: QuoteContext): Expr[FullName] = {
import ctx.tasty.given
@annotation.tailrec def cleanChunk(chunk: String): String =
val refined = chunk.stripPrefix("_$").stripSuffix("$")
if chunk != refined then cleanChunk(refined) else refined

val owner = actualOwner(ctx.tasty)(ctx.tasty.rootContext.owner)
val fullName =
owner.fullName.trim
.split("\\.", -1)
.filterNot(Util.isSyntheticName)
.map(cleanChunk)
.mkString(".")
'{FullName(${fullName.toExpr})}
}

def fullNameMachineImpl(given ctx: QuoteContext): Expr[FullName.Machine] = {
import ctx.tasty.given
val owner = ctx.tasty.rootContext.owner
val fullName = owner.fullName.trim
.split("\\.", -1)
.map(_.stripPrefix("_$").stripSuffix("$")) // meh
.map(adjustName)
.mkString(".")
'{FullName.Machine(${fullName.toExpr})}
}

def fileImpl(given ctx: QuoteContext): Expr[sourcecode.File] = {
import ctx.tasty.given
val file = ctx.tasty.rootPosition.sourceFile.jpath.toAbsolutePath.toString
'{sourcecode.File(${file.toExpr})}
}

def fileNameImpl(given ctx: QuoteContext): Expr[sourcecode.FileName] = {
import ctx.tasty.given
val name = ctx.tasty.rootPosition.sourceFile.jpath.getFileName.toString
'{sourcecode.FileName(${Expr(name)})}
}

def lineImpl(given ctx: QuoteContext): Expr[sourcecode.Line] = {
import ctx.tasty.given
val line = ctx.tasty.rootPosition.startLine + 1
'{sourcecode.Line(${line.toExpr})}
}

def enclosingImpl(given ctx: QuoteContext): Expr[Enclosing] = {
val path = enclosing(ctx.tasty)(
!Util.isSynthetic(ctx.tasty)(_)
)

'{Enclosing(${path.toExpr})}
}

def enclosingMachineImpl(given ctx: QuoteContext): Expr[Enclosing.Machine] = {
val path = enclosing(ctx.tasty, machine = true)(_ => true)
'{Enclosing.Machine(${path.toExpr})}
}

def pkgImpl(given ctx: QuoteContext): Expr[Pkg] = {
import ctx.tasty.given
val path = enclosing(ctx.tasty) {
case s if s.isPackageDef => true
case _ => false
}

'{Pkg(${path.toExpr})}
}

def argsImpl(given ctx: QuoteContext): Expr[Args] = {
import ctx.tasty.{ _, given }

val param: List[List[ctx.tasty.ValDef]] = {
def nearestEnclosingMethod(owner: ctx.tasty.Symbol): List[List[ctx.tasty.ValDef]] =
owner match {
case defSym if defSym.isDefDef =>
defSym.tree.asInstanceOf[DefDef].paramss
case classSym if classSym.isClassDef =>
classSym.tree.asInstanceOf[ClassDef].constructor.paramss
case _ =>
nearestEnclosingMethod(owner.owner)
}

nearestEnclosingMethod(ctx.tasty.rootContext.owner)
}

val texts0 = param.map(_.foldRight('{List.empty[Text[_]]}) {
case (vd @ ValDef(nme, _, optV), l) =>
'{Text(${optV.fold('None)(_.seal)}, ${nme.toExpr}) :: $l}
})
val texts = texts0.foldRight('{List.empty[List[Text[_]]]}) {
case (l, acc) =>
'{$l :: $acc}
}

'{Args($texts)}
}


def text[T: Type](v: Expr[T])(given ctx: QuoteContext): Expr[sourcecode.Text[T]] = {
import ctx.tasty.given
val txt = v.unseal.pos.sourceCode
'{sourcecode.Text[T]($v, ${txt.toExpr})}
}

sealed trait Chunk
object Chunk{
case class PkgObj(name: String) extends Chunk
case class ClsTrt(name: String) extends Chunk
case class ValVarLzyDef(name: String) extends Chunk

}

def enclosing(c: Reflection, machine: Boolean = false)(filter: c.Symbol => Boolean): String = {
import c.{ _, given }

var current = c.rootContext.owner
if (!machine)
current = actualOwner(c)(current)
var path = List.empty[Chunk]
while(current != Symbol.noSymbol && current != defn.RootPackage && current != defn.RootClass){
if (filter(current)) {

val chunk = current match {
case sym if
sym.isValDef || sym.isDefDef => Chunk.ValVarLzyDef
case sym if
sym.isPackageDef ||
sym.moduleClass != Symbol.noSymbol => Chunk.PkgObj
case sym if sym.isClassDef => Chunk.ClsTrt
case _ => Chunk.PkgObj
}

path = chunk(Util.getName(c)(current).stripSuffix("$")) :: path
}
current = current.owner
}
path.map{
case Chunk.PkgObj(s) => adjustName(s) + "."
case Chunk.ClsTrt(s) => adjustName(s) + "#"
case Chunk.ValVarLzyDef(s) => adjustName(s) + " "
}.mkString.dropRight(1)
}
}
@@ -42,8 +42,8 @@ object Apply {

val enclosing = sourcecode.Enclosing()
assert(
(enclosing == "sourcecode.Apply.applyRun myLazy$lzy Bar#enclosing") ||
(enclosing == "sourcecode.Apply.applyRun myLazy Bar#enclosing") // encoding changed in Scala 2.12
enclosing == "sourcecode.Apply.applyRun myLazy$lzy Bar#enclosing" ||
enclosing == "sourcecode.Apply.applyRun myLazy Bar#enclosing" // encoding changed in Scala 2.12
)
}
val b = new Bar{}
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.