@olafurpg olafurpg released this Sep 18, 2018 · 21 commits to master since this release

Assets 2

We are excited to announce the release of Scalameta v4.0.0. Scalameta is a library to read, analyze, transform and generate Scala programs.

This release has been under development since April 2018 and includes contributions from 15 contributors closing 189 tickets.

Documentation 📘

We have a new website! https://scalameta.org/

The website aggregates all Scalameta documentation in one place:

SemanticDB 🌳

SemanticDB is a data model for semantic information such as symbols and types about programs in Scala and other languages. Scalameta v4.0 introduces several SemanticDB improvements.

The symbol format has gone through several iterations, with highlights including:

  • package symbols end with slash / instead of dot . making it easier to perform common tasks like "find enclosing top-level class".
    • before: java/lang/ and scala/collection/
    • after: java.lang. and scala.collection.
  • method symbols no longer include parameter types making it possible to produce valid method symbols with a parser alone.
    • before: assert(). and assert(String).
    • after: assert(). and assert(+1).
  • VAL METHOD symbols exceptionally use dot . instead of method format ().
    • before: scala.Either().
    • after: scala/Either.

Synthetics are now an official part of SemanticDB and documented in the spec. Synthetics are code such as inferred type arguments, implicit parameters and desugarings that are added by the compiler and don't appear in the original source code.

Type has been split into Type (value types like TypeRef, SingletonType, etc) and Signature (non-value types like ClassSignature, MethodSignature, etc). This has improved the precision of the data model - both for SymbolInformation and other parts of SemanticDB, because signatures and types no longer intersect. Additionally, SingletonType has been split into SingleType, ThisType, SuperType and ConstantType.

The TextDocument message now has an md5: String field to verify SemanticDB payloads are up-to-date with files on disk even when text is empty.

The following modules have been updated:

  • semanticdb-scalac: updated to include latest SemanticDB spec changes including new additions such as synthetics.
  • semanticdb-javac: new Java compiler plugin to produce SemanticDB symbols.
  • metap: improved command-line interface to support latest SemanticDB spec changes.
  • metacp: improved command-line interface with finer-grained control over error handling.

Scala Native 🚀

The syntactic API (parsers, trees, tokens, ...) is now published for Scala Native. This makes it possible to use Scalameta to build binaries with fast startup.

Breaking changes ⚠️

  • There are no breaking changes for the syntactic API such as scala.meta.{Tree,Token}.
  • scala.meta.{Database,Denotation,Synthetic,ResolvedName,ResolvedSymbol,Document} have been removed. There is no longer a public API to access SemanticDB payloads, and we encourage tool authors to use SemanticDB protobuf bindings directly.
  • scala.meta.Input.{Denotation,Synthetic} have been removed along with scala.meta.{Denotation,Synthetic}
  • scala.meta.{Multipath,Sourcepath,Fragment} have been removed along with scala.meta.Database.
  • The package org.langmeta has moved to scala.meta, to migrate
    find . -type f -name '*.scala' -exec sed -i '' 's/org.langmeta/scala.meta/g' {} +
    

Milestone release notes

Contributors

$ git shortlog -sn --no-merges v3.7.4..v4.0.0
Eugene Burmako
Ólafur Páll Geirsson
Max Ovsiankin
Guillaume Massé
Gabriele Petronella
Loïc Girault
Seth Tisue
Dmytro Mitin
Hugo van Rijswijk
Devon Stewart
Filip Krikava
Andy Scott
Marcelo Cenerino
tanishiking24
Jozef Koval
Sep 12, 2018
Aug 28, 2018
Aug 23, 2018
Pre-release

@xeno-by xeno-by released this Aug 21, 2018 · 55 commits to master since this release

Assets 2

This is the 9th, and hopefully the last, milestone release for Scalameta v4. All tickets in the 4.0.0-RC1 milestone have been either fixed or postponed, and we're zeroing in on the first release candidate.

SemanticDB specification

We have codified the distinction between symbol names (i.e. fragments of global symbol format for Scala and Java) and display names (i.e. fragments of SymbolInformation payloads). Previously, both concepts were ambiguously called names, which led to occasional confusion, now they have been renamed - both in the protobuf schema (SymbolInformation.name has becomeSymbolInformation.display_name) and in the documentation.

Moreover, we have updated the spec to accommodate a recent sneaky change in the local symbols format. Previously, it was localNNN, where N was any digit, now it's localXXX, where X is any symbol except slash and semicolon (to distinguish local symbols from global and multi symbols).

semanticdb

Descriptor.name: String in the semanticdb library has been changed to Descriptor.name: n.Name, where n stands for import scala.meta.internal.semanticdb.Scala.{Names => n}.

The newly introduced n.Name is an ADT that has two cases: n.TermName and n.TypeName. We have noticed that several users of the semanticdb library had need in a concept of a name that is either a term name or a type name, just like in Scala, so we decided follow suit. For details, check out the source code at https://github.com/scalameta/scalameta/blob/v4.0.0-M9/semanticdb/semanticdb/src/main/scala/scala/meta/internal/semanticdb/Scala.scala#L163-L168.

To provide access to the string behind the descriptor in cases when n.Name is unnecessary, we have introduced Descriptor.value: String.

Pull requests

  • #1747 Last-minute changes before the 4.0 code freeze
Pre-release

@olafurpg olafurpg released this Aug 9, 2018 · 65 commits to master since this release

Assets 2

This is the 8th milestone release for Scalameta v4. Follow
https://github.com/scalameta/scalameta/milestone/26 to keep track of progress
and stay updated about the projected release date of Scalameta v4.

Synthetics

Synthetics are now an official part of SemanticDB and documented in the spec.
Synthetics are code such as inferred type arguments, implicit parameters and
desugarings that are added by the compiler and don't appear in the original
source code. To read more about synthetics, see the
specification.

This change is a major milestone for SemanticDB since inferred type
parameters and implicits are an important aspect of Scala programs.
Synthetics have so far only been accessible through undocumented and poorly
supported APIs, now they are accessible to all SemanticDB-based tools. Big
thanks to @maxov for delivering this feature!

Access

The schema for Accessibility has been refactored into a new protobuf
message Access that uses the same "sealed oneof" protobuf encoding as
Type and Signature. This change improves the ergonomics or reading and
producing Access data and makes it impossible to represent invalid data
like private[this] data with a non-empty symbol.

DEFAULT property

The SemanticDB specification added a new SymbolInformation property p.DEFAULT
to represent Scala default parameters and Java default interface methods.

Metac

The semanticdb-scalac compiler plugin now defaults to failing compilation on
errors. Previously, the plugin reported a warning on errors. To revert to the
old behavior add the compiler option -P:semanticdb:failures:warning.

The semanticdb-scalac compiler plugin has changed how the option -P:semanticdb:symbols
is handled. The valid values are:

  • all (default): persist all symbol definitions
  • local-only: persist only local symbol definitions
  • none: persist no symbol definitions

Metai

The metai command-line tool has seen multiple improvements making it more
robust when run against larger classpaths. Big thanks to @xeno-by for
discovering the issues and improving robustness all around.

Metacp

Like metai, the metacp command-line tool has also improved error handling
when processing larger classpath.

The metacp command-line tool now also supports a --usejavacp flag to
automatically add the JDK and scala-library to the dependency classpath.
Previously, it was necessary to explicitly provide those jars making it
un-ergonomic to experiment with the cli.

Pull requests

  • #1743 by @olafurpg, Fix multiple tickets assigned for the v4 milestone
  • #1740 by @maxov, Change metap macro synthetic printout to explain the expandee
  • #1738 by @xeno-by, Fix multiple tickets assigned for the v4 milestone
  • #1735 by @olafurpg, Re-integrate ScalaPB into the build
  • #1734 by @olafurpg, Improve metacp ergonomics
  • #1732 by @SethTisue, Upgrade sbt-jdi-tools to a JDK 9+ friendly version
  • #1726 by @xeno-by, Assorted improvements to metacp and metai that make everything work on a case study project
  • #1723 by @xeno-by, Assorted improvements
  • #1722 by @xeno-by, A few specification cleanups
  • #1716 by @xeno-by, Assorted changes to enable RscCompat
  • #1712 by @maxov, Specify changes to SemanticDB synthetics
  • #1709 by @maxov, Refactor Synthetics into a tree-based data structure

Contributors

Thanks to the 4 contributors who contributed to this release!

$ git shortlog -sn --no-merges v4.0.0-M6..v4.0.0-M8
Eugene Burmako
Max Ovsiankin
Ólafur Páll Geirsson
Seth Tisue
Jul 24, 2018
Pre-release

@olafurpg olafurpg released this Jul 17, 2018 · 180 commits to master since this release

Assets 2

This is the 6th milestone release for Scalameta v4. Follow https://github.com/scalameta/scalameta/milestone/26 to keep track of progress and stay updated about the projected release date of Scalameta v4.

Growing team

We are excited to welcome Max Ovsiankin (@maxov) to the Scalameta team. Max made a strategically important contribution to the Scalameta vision by implementing a Javac compiler plugin that produces SemanticDB payloads.

This is a critical advance of our vision to make SemanticDB a polyglot format for semantic information, and we'd like to officially recognize that by making Max part of the Scalameta team.

SemanticDB specification

  • The SemanticDB protobuf schema is unchanged since v4.0.0-M4

  • PACKAGE symbol descriptors now use forward slash / instead of dot .

    • Before: scala.Predef.String#
    • After: scala/Predef.String#
  • PACKAGE symbols are no longer included in TextDocument.symbols since their
    information can be computed on the fly.

  • VAL METHOD symbol descriptors now exceptionally no longer have a disambiguator

    • Before: scala/package.Either().
    • After: scala/package.Either.
  • VAL METHOD, OBJECT and FIELD symbols no longer contribute to the
    disambiguator tag for overloaded definitions.

    // In Scala
    trait A {
      object m1           // Now: A#m1.     Before: A#m1.
      def m1(a: Int): Int // Now: A#m1().   Before: A#m1(+1)
      val m2              // Now: A#m2.     Before: A#m1().
      def m2(a: Int): Int // Now: A#m2().   Before: A#m1(+1)
    }
    // In Java
    class A {
      int value;                 // Now: A#value.    Before: A#value.
      int value() { return 1; }  // Now: A#value().  Before: A#value(+1).
    }

metacp

  • The metacp cli requires a --dependency-classpath flag to compute correct external symbol references.

  • Thanks to --dependency-classpath, a bug in metacp that resulted incorrect references to
    external Java static inner classes is now fixed.

    trait A {
        // Before: java/util/Map.Entry# (incorrect, because Java OBJECT do not exist)
        // Now:    java/util/Map#Entry# (correct, according to the spec)
        def entry: java.util.Map.Entry[Int, Int] 
    }
  • The metacp cli no longer globally caches processed output and the --cache-dir flag has been removed.
    Every metacp re-invocation will now re-process the full provided <classpath>.

  • The metacp cli accepts a new --out flag to specify the output directory for produced artifacts.
    Default value is the current working directory.

  • Since metacp no longer caches processed results, the dependency on soc/directories-jvm has been removed

  • The source-code declaration order of methods is now preserved for Java classes.

semanticdb-scalac

  • An issue where occurrences would reference symbols that had no associated SymbolInformation is now fixed.
  • SymbolInformation for public members from Java compilation units in hybrid Java/Scala projects are now persisted.
  • .semanticidx generation has been removed in favor of a standalone command-line tool metai

semanticdb-javac

  • New compiler plugin org.scalameta:semanticdb-javac that is a Java compiler plugin that emits SemanticDB SymbolInformation.
  • Instructions to install and use the plugin are in the SemanticDB guide.
  • The compiler plugin currently only produces SymbolInformation, not SymbolOccurrence.

metai

  • New command-line org.scalameta:metai tool to produce META-INF/semanticdb.semanticidx from a SemanticDB classpath.
  • Index generation has been removed from metacp and metac
  • New Index.entries: Map[String, Entry] schema for semanticidx.proto schema
  • Binary payloads in semanticdb.semanticidx are now concatenable
  • Package.members has been removed from semanticidx.proto schema

symtab

  • New library module org.scalameta:symtab to lookup SymbolInformation from a classpath.

  • The symtab API lives in the package scala.meta.internal.symtab because it depends on
    the internal API scala.meta.internal.semanticdb.SymbolInformation.

  • Example usage:

    import scala.meta.io.Classpath
    import scala.meta.internal.symtab._
    val jdk = Classpath(...)
    val scalaLibrary = Classpath(...)
    val symtab = GlobalSymbolTable(jdk ++ scalaLibrary)
    symtab.info("scala/Predef.String#")
    // Some(SymbolInformation(
    //   "scala/Predef.String#", SCALA, TYPE, 0, "String",
    //   TypeSignature(
    //     Some(Scope(List(), List())),
    //     TypeRef(Empty, "java/lang/String#", List()),
    //     TypeRef(Empty, "java/lang/String#", List())
    //   ),
    //   List(), Some(Accessibility(PUBLIC, ""))
    // ))
  • Refer to SymbolTableSuite for more example usage.

Syntactic APIs

  • An issue where Windows-style \r\n line feeds resulted in InvariantFailedException during parsing is now fixed.
  • An issue where unicode escape characters in pattern position were lost is now fixed
  • An issue where the pretty-printer failed to backtick certain symbolic identifiers is now fixed.

Pull requests

Contributors

Big thanks to the 9 contributors who made this release possible!

$ git shortlog -sn --no-merges v4.0.0-M4..v4.0.0-M5
Ólafur Páll Geirsson
Eugene Burmako
Max Ovsiankin
Seth Tisue
Dmytro Mitin
Hugo van Rijswijk
Guillaume Massé
Devon Stewart
tanishiking24
Pre-release

@xeno-by xeno-by released this Jun 21, 2018 · 377 commits to master since this release

Assets 2

This is the 4th milestone release for Scalameta v4. Follow https://github.com/scalameta/scalameta/milestone/26 to keep track of progress and stay updated about the projected release date of Scalameta v4.

SemanticDB specification

  • TextDocument.schema has been bumped to SEMANTICDB4 to distinguish new-style payloads from old-style payloads, which are at this point significantly different.
  • TextDocument.md5 has been added to provide a way to quickly verify SemanticDB payloads are up-to-date.
  • Type has been split into Type (value types like TypeRef, SingletonType, etc) and Signature (non-value types like ClassSignature, MethodSignature, etc). This has improved the precision of the data model - both for SymbolInformation and other parts of SemanticDB, because signatures and types no longer intersect. Check out the specification for details.
  • SingletonType has been split into SingleType, ThisType, SuperType and ConstantType. The unification of all those types didn't quite work out, so we decided to separate them. Check out the specification for details.
  • TypeRef.prefix and SingleType.prefix now work slightly differently. Our definition of what "trivial prefix" means has changed, which means that some types that used to have non-empty prefixes before will now have empty prefixes. Search for "trivial prefix" in the specification for details.
  • ClassSignature.self has been added to track optional self types.
  • SymbolInformation.tpe has been renamed to SymbolInformation.signature. It now returns Signature instead of Type.

SemanticDB bindings

  • org.scalameta %% semanticdb3 has been renamed to org.scalameta %% semanticdb.
  • scala.meta.{Symbol,Signature} have been removed, they will find a new home in scalafix.v0.{Symbol,Signature} to maintain compatibility for existing Scalafix rules.
  • scala.meta.internal.semanticdb3 has been renamed to scala.meta.internal.semanticdb.
  • Encoding of Type has been dramatically improved to make use of the new functionality contributed to ScalaPB by @olafurpg. Type.Tag has been removed, and Type is now an idiomatic ADT, i.e. a sealed trait with multiple case classes / case objects extending it.

Metac and semanticdb-scalac

  • Added -P:semanticdb:md5:{on, off} to enable or disable generation of TextDocument.md5. By default, this compiler plugin option is enabled.
  • Changed -P:semanticdb:text:{on, off} to be disabled by default. Now what we have TextDocument.md5, we no longer need the original source code to verify integrity of SemanticDB payloads.
  • .semanticidx generation is now incremental.

Artifacts

  • org.scalameta %% semanticdb that used to contain high-level semantic APIs has been removed.
  • org.scalameta %% semanticdb3 that contains low-level semantic APIs (ScalaPB bindings as well as some extension methods) has been renamed to org.scalameta %% semanticdb.
  • org.scalameta %% semanticdb now includes ScalaPB-generated source code. This may come in handy for jumping to definitions of SemanticDB bindings in IDEs.
  • soc/directories-jvm has been upgraded to the latest version to prevent classpath conflicts that were arising when Scalameta libraries were used together with some other libraries.

Pull requests

Contributors

$ git shortlog -sn --no-merges v4.0.0-M3..v4.0.0-M4
Ólafur Páll Geirsson
Eugene Burmako
Jun 21, 2018
Merge pull request #1624 from olafurpg/test-release
Post 4.0.0-M4 cleanups [ci skip]