Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate scala-client to SDK.v3 + libuast.v3 #83

Closed
bzz opened this issue May 20, 2019 · 8 comments

Comments

@bzz
Copy link
Contributor

commented May 20, 2019

An umbrella issue for client migration to new SDK.

Takes similar approach to bblfsh/client-python. but aims to avoid having all the changes in a single PR, so will be broken down.

  • SDK.v2 - parse API
    • parse() impl, that uses new protocol.v2
    • decode() impl, so the binary wire protocol can be read (on Go side)
    • root() impl, so the decoded (on Go side) UAST can be exposed to JVM
    • load() impl, so the Go-backed UAST can be transferred to host JVM memory #91
    • encode() impl. Reverse: JVM memory objects transferred to Go
    • expose mode argument for parse (native, annotated, semantic) #111
  • SDK.v2 - filter API
    • iterator() impl, so UAST can be iterated with and without loading it to JVM #106
    • filter() impl, so XPath filtering works #109
      take early feedback on API into account
    • Context impl (JVM-side factory), so multiple concurrent filter() calls from the client are supported
  • SDK.v2 - list languages API
    Support for languages/version request (as soon as bblfsh/sdk#400 is released)
  • doc: add JNI debugging instructions #115
  • doc: update API examples in README and CLI tool
  • investigate JNI warnings moved to #112
  • enable timeouts at v2 API again moved to #100
  • CI: refactor to multiple modular jobs (tests, releases per-platform, etc) moved to #87

@bzz bzz self-assigned this May 20, 2019

@bzz bzz referenced this issue May 20, 2019
6 of 13 tasks complete
@bzz

This comment has been minimized.

Copy link
Contributor Author

commented May 20, 2019

New API example

Protocol.v2 response is protobuf-like encoded binary blob, so libuast now provides a method to decode() it on the Go side. Decoding will result only in in-memory UAST representation on the Go side, thus in order to be exposed to the host environment (JVM) as objects - the root node is also needs to load()ed.

Examples below use explicit type excessively in order to highlight the API internals, real applications \w same code will be more concise.

decode + load (Go->JVM)

import org.bblfsh.client.v2.*

// parse
//// read the binary UASTv2
val resp = client.parse(fileName, fileContent)

//// decode it, UAST is backed by Go
val ctx: Context = resp.uast.decode()
val rootNode: Node = ctx.root()

//// load the UAST from Go to JVM
val rootJvm: Map[String, _] = rootNode.load()

Iterate UAST (JVM)

// iterate the tree from root
def processRecursive(node: Any): Unit = node match {
  case m: Map[_, _] => {
    for ((k, v) <- m) {
      println(k)
      processRecursive(v)
    }
  }
  case s: Seq[_] => {
    for (e <- s) {
      processRecursive(e)
    }
  }
  case a: Any => println(a)
}

processRecursive(rootJvm)

Query (Go->JVM)

Print every node

val nodes: List[Node] = ctx.filter(rootNode, "//*")
nodes.foreach { n =>
    println(s"Node: $n")
}

@bzz bzz changed the title Migrate scala-client to SDK.v2 + libuast.v3 Migrate scala-client to SDK.v3 + libuast.v3 May 20, 2019

@dennwc

This comment has been minimized.

Copy link
Member

commented May 21, 2019

Maybe we can call NodeExt just Node (and same for Context)? Native nodes will be Map[string, _] (or whatever), so we won't need a native node type (opposed to NodeExt).

Also, the Context work is mentioned in the filter, but it is required to make decode work, so it probably belongs to that task group.

@bzz

This comment has been minimized.

Copy link
Contributor Author

commented May 21, 2019

Thank you for the suggestion! I assumed that it will be simpler to make the decode part work first, with a global context.

@bzz

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2019

#83 (comment) updated to include better type names and USAT iteration example.

bzz added a commit to bzz/client-scala that referenced this issue May 29, 2019

ci: refactor to multiple jobs
It also adds a temp job that will be kept updated and
is expected to pass for bblfsh#83
and track it's progress.

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz added a commit to bzz/client-scala that referenced this issue May 29, 2019

ci: refactor to multiple jobs
It also adds a temp job that will be kept updated and
is expected to pass for bblfsh#83
and track it's progress.

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

@dennwc dennwc pinned this issue May 29, 2019

@bzz bzz unpinned this issue May 29, 2019

@bzz

This comment has been minimized.

Copy link
Contributor Author

commented Jul 9, 2019

A pre-release RC1 was cut from the current state of v2 migration https://github.com/bblfsh/scala-client/releases/tag/v2.0-rc1

Update: RC1 is blocked by #98 and will be re-published once that is merged.

Update: RC1 has been published.

@bzz

This comment has been minimized.

Copy link
Contributor Author

commented Aug 1, 2019

Update: RC3 has been published.

This was referenced Aug 6, 2019

@bzz bzz added this to the v2.0.0 milestone Aug 14, 2019

@bzz

This comment has been minimized.

Copy link
Contributor Author

commented Aug 14, 2019

The only missing part before doing v2.0.0 release is a developer documentation!
Will submit a PR first thing tomorrow and start preparing a release notes.

@bzz bzz referenced this issue Aug 16, 2019
5 of 5 tasks complete
@bzz

This comment has been minimized.

Copy link
Contributor Author

commented Aug 16, 2019

Everything seems to be ready for initial release 🎉

@bzz bzz closed this Aug 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.