New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semantic API #604

Closed
xeno-by opened this Issue Jan 29, 2017 · 3 comments

Comments

2 participants
@xeno-by
Member

xeno-by commented Jan 29, 2017

Scala.meta 1.5 supports inspections of the structure of Scala code (syntax and tokens), but it doesn't provide any facilities to inspect semantics (what's the type of this fragment of code? what does this name point to? etc). We had this in scala.reflect, and we need to have this in scala.meta, too, because that's one of the most basic features one expects in a metaprogramming framework.

In the past, we've made two unsuccessful attempts at implementing the semantic API for the Scala compiler (first and second). Both of them were based on the concept of a scala.reflect -> scala.meta tree converter, i.e. a module that would convert attributed compiler trees into equivalent scala.meta trees with semantic information encoded in a platform-independent way. Long story short, eventually we realized that it is prohibitively hard to implement a useful converter, so we had to start looking for other ways to build a semantic API (see #148 for a detailed explanation).

Around Scala eXchange 2016, we had an idea to give up on converters altogether and obtain semantic information about scala.meta trees using positions. Instead of eagerly converting attributed scala.reflect trees into attributed scala.meta trees, we handle semantic requests dynamically. For every request to a scala.meta tree, we use its position to identify a corresponding attributed scala.reflect tree and then use that tree's attributes to complete the request. After several hacking sessions in December and January, I'm now convinced that this approach is solid enough to become the basis of the semantic API.

After some consideration, we have also decided to cut the scope of the initial vision of the semantic API as designed ca. 2013. Concretely, in the first iteration of the API (scala.meta.semantic.v1), we're going to temporarily introduce the notion of symbols, and we're going to only support fully attributed and fully unattributed ASTs. We are still planning to eventually implement the initial vision (scala.meta.semantic.v2, see #623 for more information), but we'd like to ship something partially useful as soon as possible.

(April 2017). After more deliberation, we have further cut the scope of the semantic API. For the time being, we're going to give up on supporting unattributed trees and on the unification of symbols and trees. Therefore, scala.meta.semantic.v2 bites the dust and scala.meta.semantic.v1 becomes scala.meta.semantic.

Implementation plan:

  • Support for semantic databases (#605)
  • Minimal scala.meta.semantic.v1:
    • Online mirror with support for Ref.symbol (#606)
    • Rudimentary functionality to work with symbols (#609)
    • Ref.symbol for unattributed trees (#608)
    • Offline mirror with support for Ref.symbol (#607)
    • Term.tpe (#611)
    • Functionality to work with types (#612)
  • Full-blown scala.meta.semantic.v1:
    • Term.desugaring and other issues labelled with Semantic (list)

@xeno-by xeno-by added the Semantic label Jan 29, 2017

@xeno-by xeno-by changed the title from semantic API to Semantic API Jan 30, 2017

@xeno-by xeno-by referenced this issue Jan 30, 2017

Closed

Unicorn semantic API #623

0 of 3 tasks complete
@smarter

This comment has been minimized.

Show comment
Hide comment
@smarter

smarter Jan 31, 2017

Contributor

For every request to a scala.meta tree, we use its position to identify a corresponding attributed scala.reflect tree and then use that tree's attributes to complete the request.

Interesting! This seems similar to the kind of things IDEs do to handle features like type-at-point/symbol-at-point (which incidentally should soon be very easy to do in dotty :))

Contributor

smarter commented Jan 31, 2017

For every request to a scala.meta tree, we use its position to identify a corresponding attributed scala.reflect tree and then use that tree's attributes to complete the request.

Interesting! This seems similar to the kind of things IDEs do to handle features like type-at-point/symbol-at-point (which incidentally should soon be very easy to do in dotty :))

@xeno-by

This comment has been minimized.

Show comment
Hide comment
@xeno-by

xeno-by Jan 31, 2017

Member

@smarter Sounds very nice! Can you provide more details?

Upd. Here are some links from our hangout call with @smarter:

Member

xeno-by commented Jan 31, 2017

@smarter Sounds very nice! Can you provide more details?

Upd. Here are some links from our hangout call with @smarter:

xeno-by added a commit to xeno-by/scalameta that referenced this issue Feb 1, 2017

remove obsolete semantic infrastructure in trees
This functionality is no longer necessary according to the new approach
described in scalameta#604.

Therefore, I’m gladly removing it from scala.meta. Both the codegen part
and the whole withAttrs/setTypechecked business were terribly complicated
for the value they offered.

xeno-by added a commit to xeno-by/scalameta that referenced this issue Feb 1, 2017

remove obsolete semantic infrastructure in trees
This functionality is no longer necessary according to the new approach
described in scalameta#604.

Therefore, I’m gladly removing it from scala.meta. Both the codegen part
and the whole withAttrs/setTypechecked business were terribly complicated
for the value they offered.

xeno-by added a commit to xeno-by/scalameta that referenced this issue Feb 1, 2017

remove obsolete semantic infrastructure in trees
This functionality is no longer necessary according to the new approach
described in scalameta#604.

Therefore, I’m gladly removing it from scala.meta. Both the codegen part
and the whole withAttrs/setTypechecked business were terribly complicated
for the value they offered.

@tkroman tkroman referenced this issue Feb 4, 2017

Open

scala meta #114

@xeno-by xeno-by modified the milestones: v1.9.0, v2.0.0 May 10, 2017

@xeno-by xeno-by self-assigned this May 10, 2017

@xeno-by

This comment has been minimized.

Show comment
Hide comment
@xeno-by

xeno-by Sep 6, 2017

Member

Recently, we decided to limit the surface of our work to actionable items only - either bugs in existing code or feature requests that are warranted by the roadmap.

While semantic API is definitely on our roadmap, we're going to figure out desired APIs from use cases, not from reflecting on scala.reflect and pondering what functionality from there we should add to Scalameta. That's why I'll be closing this issue and letting feature requests be submitted naturally by our users.

Member

xeno-by commented Sep 6, 2017

Recently, we decided to limit the surface of our work to actionable items only - either bugs in existing code or feature requests that are warranted by the roadmap.

While semantic API is definitely on our roadmap, we're going to figure out desired APIs from use cases, not from reflecting on scala.reflect and pondering what functionality from there we should add to Scalameta. That's why I'll be closing this issue and letting feature requests be submitted naturally by our users.

@xeno-by xeno-by closed this Sep 6, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment