Skip to content
Permalink
Browse files

Merge pull request #353 from alexarchambault/develop

Various things
  • Loading branch information...
alexarchambault committed Apr 12, 2019
2 parents a4cffe0 + ca62c85 commit 753818b02a7f9408ee5a2f9094fe07ca20225a32
Showing with 3,191 additions and 778 deletions.
  1. +1 −1 .travis.yml
  2. +1 −0 build.sbt
  3. +108 −12 docs/pages/api-ammonite.md
  4. +80 −25 docs/pages/api-jupyter.md
  5. +1 −1 docs/website/sidebars.json
  6. +164 −0 examples/displays.ipynb
  7. +1 −1 modules/echo/src/main/scala/almond/echo/EchoKernel.scala
  8. +8 −8 modules/scala/almond-rx/src/main/scala/almondrx/package.scala
  9. +1 −1 ...lmond-spark/src/main/scala/org/apache/spark/sql/almondinternals/NotebookSparkSessionBuilder.scala
  10. +21 −2 modules/scala/jupyter-api/src/main/scala/almond/api/JupyterApi.scala
  11. +24 −26 modules/scala/jupyter-api/src/main/scala/almond/api/helpers/Display.scala
  12. +31 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Data.scala
  13. +76 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Display.scala
  14. +60 −0 modules/scala/jupyter-api/src/main/scala/almond/display/FileLink.scala
  15. +32 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Html.scala
  16. +68 −0 modules/scala/jupyter-api/src/main/scala/almond/display/IFrame.scala
  17. +157 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Image.scala
  18. +32 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Javascript.scala
  19. +46 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Json.scala
  20. +32 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Latex.scala
  21. +32 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Markdown.scala
  22. +37 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Math.scala
  23. +90 −0 modules/scala/jupyter-api/src/main/scala/almond/display/ProgressBar.scala
  24. +32 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Svg.scala
  25. +32 −0 modules/scala/jupyter-api/src/main/scala/almond/display/Text.scala
  26. +83 −0 modules/scala/jupyter-api/src/main/scala/almond/display/TextDisplay.scala
  27. +48 −0 modules/scala/jupyter-api/src/main/scala/almond/display/UpdatableDisplay.scala
  28. +38 −0 modules/scala/jupyter-api/src/main/scala/almond/input/Input.scala
  29. +128 −0 modules/scala/scala-interpreter/src/main/java/almond/internals/HtmlAnsiOutputStream.java
  30. +11 −10 ...ala/scala-interpreter/src/main/scala-2.11_2.12/almond/internals/ScalaInterpreterInspections.scala
  31. +16 −2 ...es/scala/scala-interpreter/src/main/scala-2.13/almond/internals/ScalaInterpreterInspections.scala
  32. +249 −0 modules/scala/scala-interpreter/src/main/scala/almond/AmmInterpreter.scala
  33. +390 −0 modules/scala/scala-interpreter/src/main/scala/almond/Execute.scala
  34. +57 −0 modules/scala/scala-interpreter/src/main/scala/almond/JupyterApiImpl.scala
  35. +206 −0 modules/scala/scala-interpreter/src/main/scala/almond/ReplApiImpl.scala
  36. +73 −570 modules/scala/scala-interpreter/src/main/scala/almond/ScalaInterpreter.scala
  37. +32 −0 modules/scala/scala-interpreter/src/main/scala/almond/ScalaInterpreterParams.scala
  38. +10 −7 modules/scala/scala-interpreter/src/main/scala/almond/internals/UpdatableFuture.scala
  39. +23 −4 modules/scala/scala-interpreter/src/main/scala/almond/internals/UpdatableResults.scala
  40. +34 −17 modules/scala/scala-interpreter/src/test/scala/almond/ScalaInterpreterTests.scala
  41. +405 −54 modules/scala/scala-interpreter/src/test/scala/almond/ScalaKernelTests.scala
  42. +31 −0 modules/scala/scala-kernel-api/src/main/scala/almond/api/FullJupyterApi.scala
  43. +1 −1 modules/scala/scala-kernel-api/src/main/scala/almond/api/JupyterAPIHolder.scala
  44. +24 −0 modules/scala/scala-kernel-api/src/main/scala/almond/api/internal/Lazy.scala
  45. +20 −0 modules/scala/scala-kernel-api/src/main/scala/almond/api/internal/Modifiable.scala
  46. +74 −0 modules/scala/scala-kernel-api/src/main/scala/almond/display/PrettyPrint.scala
  47. +5 −1 modules/scala/scala-kernel/src/main/scala/almond/Options.scala
  48. +40 −31 modules/scala/scala-kernel/src/main/scala/almond/ScalaKernel.scala
  49. +2 −2 modules/shared/interpreter/src/main/scala/almond/interpreter/InterpreterToIOInterpreter.scala
  50. +23 −0 modules/shared/interpreter/src/main/scala/almond/interpreter/util/AsyncInterpreterOps.scala
  51. +1 −2 ...nternals → shared/interpreter/src/main/scala/almond/interpreter/util}/CancellableFuturePool.scala
@@ -27,7 +27,7 @@ stages:
if: (branch = master AND type = push) OR (tag IS present)
jobs:
include:
- env: VALIDATE_EXAMPLES=1 # unused from the script, just to know what the job does from the Travis UI
- name: "Validate examples"
addons:
apt:
sources:
@@ -136,6 +136,7 @@ lazy val `scala-interpreter` = project
Nil
}
},
libraryDependencies += "org.fusesource.jansi" % "jansi" % "1.18",
crossVersion := CrossVersion.full,
testSettings
)
@@ -21,27 +21,90 @@ The following instances are available:

### Load dependencies

???
`interp.load.ivy` accepts one or several
[`coursier.Dependency`](https://github.com/coursier/coursier/blob/ac5a6efa3e13925f0fb1409ea45d6b9a29865deb/modules/coursier/shared/src/main/scala/coursier/package.scala#L19).

Load simple dependencies
```scala
interp.load.ivy("org.platanios" %% "tensorflow-data" % "0.4.1")
```

Load dependencies while adjusting some parameters
```scala
interp.load.ivy(
coursier.Dependency(
module = coursier.Module("org.platanios", "tensorflow_2.12"),
version = "0.4.1",
// replace with linux-gpu-x86_64 on linux with nvidia gpu or with darwin-cpu-x86_64 on macOS
attributes = coursier.Attributes("", "linux-cpu-x86_64")
)
)
```

The dependencies can then be used in the cell right _after_ the one calling
`interp.load.ivy`.

Note that in the case of simple dependencies, when directly entering code
in a notebook,
the following syntax is preferred
and allows to use the dependency in the current cell rather than the next one:
```scala
import $ivy.`org.platanios::tensorflow-data:0.4.1`
```

### Load compiler plugins

???
`interp.load.plugin.ivy` accepts one or several
[`coursier.Dependency`](https://github.com/coursier/coursier/blob/ac5a6efa3e13925f0fb1409ea45d6b9a29865deb/modules/coursier/shared/src/main/scala/coursier/package.scala#L19).

```scala
interp.load.plugin.ivy("org.spire-math" %% "kind-projector" % "0.9.9")
```
The plugins can then be used in the cell right _after_ the one calling
`interp.load.plugin.ivy`.


Note that when directly entering code in a notebook, the following syntax
is more concise, and allows to use the compiler plugin in the current cell:
```scala
import $plugin.$ivy.`org.spire-math::kind-projector:0.9.9`
// example of use
trait T[F[_]]
type T2 = T[Either[String, ?]]
```

### Add repositories

???
One can add extra
[`coursier.Repository`](https://github.com/coursier/coursier/blob/ac5a6efa3e13925f0fb1409ea45d6b9a29865deb/modules/coursier/shared/src/main/scala/coursier/package.scala#L69) via

```scala
interp.repositories() ++= Seq(MavenRepository(
"https://nexus.corp.com/content/repositories/releases",
authentication = Some(Authentication("user", "pass"))
))
```

### Add exit hooks

???
```scala
interp.beforeExitHooks += { _ =>
// called before the kernel exits
}
```

### Configure compiler options

???
```scala
// enable warnings
interp.configureCompiler(_.settings.nowarn.value = false)
```

## `ReplAPI`

[`ReplAPI`](https://github.com/lihaoyi/Ammonite/blob/master/amm/repl/src/main/scala/ammonite/repl/ReplAPI.scala) allows to
- [access the pretty-printer and customize its behavior](#pretty-printer),
- [access the latest thrown exception](#latest-thrown-exception),
- [access the command history](#command-history),
- [request the compiler instance to be re-created](#refresh-compiler-instance),
@@ -51,30 +114,63 @@ The following instances are available:
- [get the class names and byte code](#byte-code-of-repl-inputs) of the code entered during the current
session.

### Pretty-printer

```scala
class Foo(val x: Int)
repl.pprinter() = {
val p = repl.pprinter()
p.copy(
additionalHandlers = p.additionalHandlers.orElse {
case f: Foo =>
pprint.Tree.Lazy(_ => Iterator(fansi.Color.Yellow(s"foo: ${f.x}").render))
}
)
}
```

### Latest thrown exception

???
```scala
repl.lastException // last thrown exception, or null if none were thrown
```

### Command history

???
```scala
repl.history // current session history
repl.fullHistory // shared history
```

### Refresh compiler instance

???
```scala
repl.newCompiler()
```

### Get compiler instance

???
```scala
repl.compiler // has type scala.tools.nsc.Global
```

### Get current imports

???
```scala
repl.imports
```

### Evaluate code

???
```scala
repl.load("val a = 2")
```

### Byte code of REPL inputs

???
```scala
repl.sess.frames.flatMap(_.classloader.newFileDict).toMap
// Map[String, Array[Byte]], keys: class names, values: byte code
```

@@ -6,13 +6,68 @@ The Almond Jupyter API can be accessed via an instance of [`almond.api.JupyterAP
instance is created by almond upon start-up. This instance accessible via the `kernel` variable and in the
implicit scope via e.g. `implicitly[almond.api.JupyterAPI]`.

A number of higher level helpers rely on it, and provide [a more convenient API to display objects](#display).

## High level API

### Display

A number of classes under [`almond.display`](https://github.com/almond-sh/almond/tree/master/modules/scala/jupyter-api/src/main/scala/almond/display)
provide an API similar to the
[IPython display module](https://ipython.readthedocs.io/en/7.4.0/api/generated/IPython.display.html).

Examples:
```scala
// These can be used to display things straightaway
Html("<b>Bold</b>")
```

```scala
// A handle can also be retained, to later update or clear things
val handle = Markdown("""
# title
## section
text
""")
// can be updated in later cells
// (this updates the previous display)
handle.withContent("""
# updated title
## new section
_content_
""").update()
// can be later cleared
handle.clear()
```

### Input

```scala
// Request input from user
val result = Input().request()
```

```scala
val result = Input().withPrompt(">>> ").request()
```

```scala
// Request input via password field
val result = Input().withPassword().request()
```


## `JupyterAPI`

`almond.api.JupyterAPI` allows to
- [request input](#request-input) (password input in particular),
- [exchange comm messages](#comm-messages) with the front-end.
- [display data](#display-data) (HTML, text, images, …) in the front-end while a cell is running,
- [update a previous display](#updatable-display-data) in the background (while the initial cell is running or not),
- [exchange comm messages](#comm-messages) with the front-end.

Note that most of its capabilities have more convenient alternatives, see [High level API](#high-level-api).

### Request input

@@ -24,6 +79,30 @@ kernel.stdin(prompt = ">> ", password = true) // password input, with custom pro

![](/demo/stdin.gif)

### Comm messages

[Comm messages](https://jupyter-notebook.readthedocs.io/en/5.7.2/comms.html) are part of the
[Jupyter messaging protocol](https://jupyter-client.readthedocs.io/en/5.2.3/messaging.html). They
allow the exchange of arbitrary messages between code running in the front-end (typically JavaScript code)
and kernels.

The comm API can be used to receive messages, or send them.

`kernel.comm.receiver` allows to register a target to receive messages from the front-end, like
```scala
val id = java.util.UUID.randomUUID().toString
kernel.publish.html("Waiting", id)
kernel.comm.receiver("A") { data =>
// received message `data` from front-end
kernel.publish.updateHtml(s"<code>$data</code>", id)
}
```

![](/demo/comm-receive.gif)

TODO Send to client

### Display data

The `publish` field, of type [`almond.interpreter.api.OutputHandler`](https://github.com/almond-sh/almond/blob/master/modules/shared/interpreter-api/src/main/scala/almond/interpreter/api/OutputHandler.scala), has numerous methods to push display data to the front-end.
@@ -74,27 +153,3 @@ kernel.publish.updateHtml("Got all items", id)
```

![](/demo/updatable.gif)

### Comm messages

[Comm messages](https://jupyter-notebook.readthedocs.io/en/5.7.2/comms.html) are part of the
[Jupyter messaging protocol](https://jupyter-client.readthedocs.io/en/5.2.3/messaging.html). They
allow the exchange of arbitrary messages between code running in the front-end (typically JavaScript code)
and kernels.

The comm API can be used to receive messages, or send them.

`kernel.comm.receiver` allows to register a target to receive messages from the front-end, like
```scala
val id = java.util.UUID.randomUUID().toString
kernel.publish.html("Waiting", id)
kernel.comm.receiver("A") { data =>
// received message `data` from front-end
kernel.publish.updateHtml(s"<code>$data</code>", id)
}
```

![](/demo/comm-receive.gif)

TODO Send to client
@@ -4,7 +4,7 @@
"Try it": ["try-mybinder", "try-docker"],
"Installation": ["quick-start-install", "install-options", "install-multiple", "install-versions", "install-other"],
"Usage": ["usage-plotting", "usage-spark"],
"User API": ["api", "api-access-instances", "api-ammonite", "api-jupyter"],
"User API": ["api", "api-ammonite", "api-jupyter", "api-access-instances"],
"Development": ["dev-from-sources", "dev-custom-kernel", "dev-libraries", "dev-website"]
}
}
Oops, something went wrong.

0 comments on commit 753818b

Please sign in to comment.
You can’t perform that action at this time.