Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enso Integration with Ideal Graph Visualizer #3533

Merged
merged 23 commits into from
Jun 23, 2022
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
057aa6c
Ideal Graph Visualizer and NetBeans support for Enso language
JaroslavTulach Jun 16, 2022
75b5ba4
GitHub Actions workflow that builds enso4igv using Maven
JaroslavTulach Jun 16, 2022
b1eb32b
Specify path to the LICENSE file
JaroslavTulach Jun 16, 2022
8eaee7a
Upload .nbm file as a ZIP artifact
JaroslavTulach Jun 16, 2022
16bfe00
Describing download steps
JaroslavTulach Jun 17, 2022
52ebbf2
One needs to be logged into GitHub
JaroslavTulach Jun 17, 2022
e828675
Set the version based on GitHub run number
JaroslavTulach Jun 17, 2022
9d4aed6
Screenshots guiding thru the installation process
JaroslavTulach Jun 17, 2022
0559799
Example of using IGV with Enso integration to analyze sieve.enso program
JaroslavTulach Jun 20, 2022
4255973
Highlight locations of the toolbar icons
JaroslavTulach Jun 20, 2022
342b43e
Only trigger rebuild when Enso for IGV files project are changed
JaroslavTulach Jun 20, 2022
f2fcfd4
Prettier formatting of enso4igv documents
JaroslavTulach Jun 20, 2022
03d462b
Formatting
JaroslavTulach Jun 20, 2022
870ef47
Set working directory
JaroslavTulach Jun 20, 2022
d651c83
IGV and Enso language support are operating system neutral
JaroslavTulach Jun 20, 2022
e95041c
Note in CHANGELOG.md
JaroslavTulach Jun 20, 2022
a6ecd23
Don't include large png snapshots in the repository
JaroslavTulach Jun 20, 2022
b7a797d
npx prettier
JaroslavTulach Jun 20, 2022
3f92169
Launchers have to recognize --dump-graphs option
JaroslavTulach Jun 22, 2022
45e51f0
Cross linking with runtime-guide.md
JaroslavTulach Jun 22, 2022
c49c766
Updating README according to Hubert's review comments
JaroslavTulach Jun 22, 2022
1aab449
Correcting a simpler typo seems to fix the script for Windows 11
JaroslavTulach Jun 22, 2022
7d04edf
Merge branch 'develop' into wip/jtulach/enso4igv-182388312
mergify[bot] Jun 22, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/enso4igv.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# This workflow will build a Java project with Maven, and cache/restore any dependencies to improve the workflow execution time
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven

name: Enso Language Support for IGV

on:
push:
branches: [develop]
pull_request:
branches: [develop]
paths:
- "tools/enso4igv/**/*"
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

jobs:
build:
runs-on: ubuntu-20.04
strategy:
matrix:
java: ["8"]

steps:
- uses: actions/checkout@v2
- name: Set up Java
uses: actions/setup-java@v2
with:
distribution: "zulu"
java-version: ${{ matrix.java }}
cache: maven

- name: Update project version
run: |
cd tools/enso4igv
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
mvn versions:set -DnewVersion=`mvn -q -DforceStdout help:evaluate -Dexpression=project.version | cut -f1 -d -`.$GITHUB_RUN_NUMBER
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

- name: Build with Maven
run: mvn -B package --file tools/enso4igv/pom.xml
- name: Archive NBM file
uses: actions/upload-artifact@v3
with:
name: Enso IGV Plugin
path: tools/enso4igv/target/*.nbm
133 changes: 133 additions & 0 deletions tools/enso4igv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Enso Language Support for NetBeans & Ideal Graph Visualizer
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

[![Enso Language Support for IGV](https://github.com/enso-org/enso/actions/workflows/enso4igv.yml/badge.svg)](https://github.com/enso-org/enso/actions/workflows/enso4igv.yml)

[Enso language](http://enso.org) runtime engine is built on top of
[GraalVM](http://graalvm.org) and its _Truffle framework_. Enso, as a good
citizen of the GraalVM ecosystem, benefits from polyglot capabilities of GraalVM
as well as its rich tooling offering. One of such tools is _IGV_ - the _Ideal
Graph Visualizer_ - an excellent tool to get insights into behavior of Graal
compiler.

This document shows how to use _IGV_ with the _Enso language_.

## Installation
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

Visit [GraalVM.org](http://graalvm.org) download page and continue towards
_enterprise edition_ option. There is an _Ideal Graph Visualizer_ option. After
clicking thru the confirmation dialogs you should get a ZIP - I've just got
`idealgraphvisualizer-22.1.0.zip` and then:
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

```bash
$ unzip idealgraphvisualizer-22.1.0.zip
$ ./idealgraphvisualizer/bin/idealgraphvisualizer --userdir /tmp/emptyuserdir
```

launches the _IGV_ application. One doesn't have to use the `--userdir` option,
but doing so ensures the newly running _IGV_ process is isolated from any
settings left around by previous usage of _IGV_.

Now download
[Enso Language Support module](https://github.com/enso-org/enso/actions/workflows/enso4igv.yml).
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
Follow the
[GitHub actions link](https://github.com/enso-org/enso/actions/workflows/enso4igv.yml)
and select a build (usually the latest one). The build summary page provides
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
various information as well as list of artifacts at the bottom. Download the
_Enso IGV Plugin_ ZIP file. Make sure you are logged into GitHub - artifacts are
only available to those logged in. Unzip it and get `enso*.nbm` file. This file
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
can be installed into _IGV_ (or any other [NetBeans](http://netbeans.apache.org)
based application). Go to _Tools_/_Plugins_/_Downloaded_ and install the NBM
file.
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

![Tools/Plugins/Downloaded](docs/tools_plugins_downloaded.png)
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

Proceed by clicking _Install_. You may be asked to download _TextMate Lexer_ - a
necessary dependency of the _Enso support_ module. Continue thru the wizard to
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
_finish_ the installation.

![Tools/Plugins/Downloaded](docs/installer.png)

## Using the IGV

Get an instance of the Enso runtime engine (see
[Running Enso](../../docs/CONTRIBUTING.md#running-enso)) and then launch it with
special Java options:
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

```bash
enso$ JAVA_OPTS="-Dpolyglot.engine.AllowExperimentalOptions=true\
-Dpolyglot.engine.TraceCompilation=true\
-Dpolyglot.engine.MultiTier=false\
-Dgraal.Dump=Truffle:1"\
./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run yourprogram.enso
```

When executed on [GraalVM 21.3.0](http://graalvm.org) these options instruct the
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
_Graal/Truffle compiler_ to dump files into `graal_dumps/_sometimestamp_`
directory. Generating these files takes a while - make sure `yourprogram.enso`
runs long enough for the system to warmup, compile the code and run at _full
speed_.

#### Sieve of Eratosthenes Example

As an example you can download
[sieve.enso](https://github.com/jtulach/sieve/blob/5b32450da35415322e683bb9769aa45f0d71f1df/enso/sieve.enso)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd upload the file and link here from this repo.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. It would be good to include sieve.enso and make it part of daily executed code. That way we know the syntax is correct and the engine can still run the file. Where should I put the sieve.enso file then? CCing @hubertp , @jdunkerley , @kustosz - thank you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tools/enso4igv/examples ? Maybe there is something simpler we could add as well/instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It cannot be too simple program, it has to warmup. It could be any of the traditional benchmarks like N-body, Funkuch, etc. However I doubt they are simpler.

Anyway let's address this later. I am already behind the schedule and properly solving this (e.g. making sure the file is tested on each change in engine) requires additional thoughts.

which computes hundred thousand of prime numbers repeatedly and measures time of
each round. Download the file and launch Enso with:

```bash
enso$ JAVA_OPTS="-Dpolyglot.engine.AllowExperimentalOptions=true\
-Dpolyglot.engine.TraceCompilation=true\
-Dpolyglot.engine.MultiTier=false\
-Dgraal.Dump=Truffle:1"\
./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run sieve.enso
```

Bunch of files in `graal_dumps/*` subdirectory is going to be generated:

```bash
enso$ ls graal_dumps/*/Truffle* | tail -n5
graal_dumps/2022.06.20.06.18.21.733/TruffleHotSpotCompilation-9889[argument<2>].bgv
graal_dumps/2022.06.20.06.18.21.733/TruffleHotSpotCompilation-9896[IfThenElseMethodGen@3af870b9_<split-62b6b4f3>]_1.bgv
graal_dumps/2022.06.20.06.18.21.733/TruffleHotSpotCompilation-9896[IfThenElseMethodGen@3af870b9_<split-62b6b4f3>].bgv
graal_dumps/2022.06.20.06.18.21.733/TruffleHotSpotCompilation-9935[Primes.next_<split-717d5bdf>]_1.bgv
graal_dumps/2022.06.20.06.18.21.733/TruffleHotSpotCompilation-9935[Primes.next_<split-717d5bdf>].bgv
```

Let's launch IGV with Enso integration and let's open graph for one of the
top-most functions: `TruffleHotSpotCompilation*Primes_next*.bgv`. Choose
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
compilation phase _"Before lowering"_:

![Before Lowering Graph](docs/igv_graph.png)

Now you can inspect the _compiler graphs_ the regular _IGV_ way. Let's locate
for example `LoadField#FunctionSchema.isFullyApplied` node and let's check how
it got _inlined_:
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

![Inlining Stacktrace](docs/igv_stacktrace.png)

The stack trace shows what methods of the Enso interpreter and Truffle runtime
are _"inlined on stack"_ when this node is being compiled. This is all regular
_IGV_ functionality, but now we can switch to _Enso view_:

![Enso Source](docs/igv_enso.png)

By choosing the _Enso language icon_ in front of the stack trace combo, the
source code of our `.enso` program is opened and we can analyze what _compiler
nodes_ refer to what lines in the our _Enso_ program. Click _Navigate to Source_
icon in the _Stack View_ to get from graph node to source. Select a drop down
widget in the editor toolbar to show you what compiler nodes as associated with
currently selected line.

## Building

The plugin can be rebuilt using [Apache Maven](http://maven.apache.org). Switch
to this directory and invoke:

```bash
enso/tools/enso4igv$ mvn clean install
enso/tools/enso4igv$ ls target/*.nbm
target/enso4igv-1.0-SNAPSHOT.nbm
```

an NBM file is generated which can be installed into IGV, NetBeans or any other
NetBeans based application.
Binary file added tools/enso4igv/docs/igv_enso.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tools/enso4igv/docs/igv_graph.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tools/enso4igv/docs/igv_stacktrace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tools/enso4igv/docs/installer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
111 changes: 111 additions & 0 deletions tools/enso4igv/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.enso.tools</groupId>
<artifactId>enso4igv</artifactId>
<packaging>nbm</packaging>
<name>Enso Language Support for NetBeans &amp; Ideal Graph Visualizer</name>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.netbeans.utilities</groupId>
<artifactId>nbm-maven-plugin</artifactId>
<extensions>true</extensions>
<version>4.7</version>
<configuration>
<useOSGiDependencies>true</useOSGiDependencies>
<author>Enso.org</author>
<licenseName>Apache 2.0</licenseName>
<licenseFile>../../LICENSE</licenseFile>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.2.2</version>
<configuration>
<archive>
<manifestFile>${project.build.outputDirectory}/META-INF/MANIFEST.MF</manifestFile>
</archive>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-netbeans-api-annotations-common</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-netbeans-modules-textmate-lexer</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-netbeans-api-templates</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-filesystems</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-loaders</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-nodes</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-util</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-util-lookup</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-util-ui</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-windows</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-text</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-netbeans-core-multiview</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-awt</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-openide-dialogs</artifactId>
<version>${netbeans.version}</version>
</dependency>
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<netbeans.version>RELEASE113</netbeans.version>
</properties>
</project>
Loading