Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid whole source reparsing when the IDE performs a simple edit #3508

Merged
merged 67 commits into from
Jun 16, 2022
Merged
Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
cecd735
IncrementalUpdatesTest is a copy of RuntimeServerTest written in Java
JaroslavTulach Jun 1, 2022
8532a9a
Recognize simple TextEdit change
JaroslavTulach Jun 2, 2022
9aeeb2e
Eliminating useless Scala <-> Java collection conversions
JaroslavTulach Jun 2, 2022
fd7f238
Use node counting instrument to fail when reparsing happens
JaroslavTulach Jun 3, 2022
ccce1fd
No need to create a dummy node. Return null
JaroslavTulach Jun 3, 2022
8bd50ab
Identify simple edit and apply it to right Truffle node
JaroslavTulach Jun 3, 2022
fab4189
Hack to treat IDE's simple+comment edit as a simple edit
JaroslavTulach Jun 3, 2022
467fb19
Applying javafmt
JaroslavTulach Jun 4, 2022
6303000
Associating TruffleFile, Rope, Source into an immutable record and up…
JaroslavTulach Jun 4, 2022
1a87a6d
Update SourceSections when performing simple edit
JaroslavTulach Jun 5, 2022
1164679
Survive when changed value isn't a number
JaroslavTulach Jun 7, 2022
7603648
Formatted by scalafmt
JaroslavTulach Jun 7, 2022
31648ad
Introducing patchable tag to mark nodes that know how to patch themse…
JaroslavTulach Jun 7, 2022
a66862b
Avoid iterating node subtrees that are clearly off
JaroslavTulach Jun 8, 2022
28a162a
Using mapExpressions to update the IR after simple edit
JaroslavTulach Jun 8, 2022
86ef968
Merge remote-tracking branch 'origin/develop' into wip/jtulach/Increm…
JaroslavTulach Jun 8, 2022
40eb24b
receiveNIgnoreStdLib method now requires two arguments
JaroslavTulach Jun 8, 2022
d7deb6c
Module maintains its latest Source as well as 'deltas' against latest…
JaroslavTulach Jun 8, 2022
03b58e4
Note in changelog and using a method reference
JaroslavTulach Jun 9, 2022
69a2f98
Merge remote-tracking branch 'origin/develop' into wip/jtulach/Increm…
JaroslavTulach Jun 9, 2022
45e7bb4
Applying proper formatting with javafmtAll
JaroslavTulach Jun 9, 2022
a785033
Generalizing the test to work also on text literals
JaroslavTulach Jun 9, 2022
6797f59
Handle simple edits of text literals
JaroslavTulach Jun 9, 2022
348c18d
Applying scalafmt formatting
JaroslavTulach Jun 10, 2022
dea5e92
Avoid modifications to ModuleScope
JaroslavTulach Jun 10, 2022
1e4e8a4
Representing interactive state by presence of patchedValues
JaroslavTulach Jun 10, 2022
b97bd2d
Merge remote-tracking branch 'origin/develop' into wip/jtulach/Increm…
JaroslavTulach Jun 10, 2022
48a6c32
Call Module.createSection to adjust to deltas provided by position ma…
JaroslavTulach Jun 12, 2022
f37250c
Associate UnwindException with a value and use it in the IdExecutionI…
JaroslavTulach Jun 12, 2022
eb865ea
Support inline sources different from those associated with a Module
JaroslavTulach Jun 13, 2022
b85e455
Root node without a source produces no source sections
JaroslavTulach Jun 13, 2022
dbe6548
Merging with develop branch
JaroslavTulach Jun 14, 2022
0bdfde9
Enabling use of Frgaal and Truffle in runtime-with-instruments tests
JaroslavTulach Jun 14, 2022
71ff2b6
Moving IncrementalUpdatesTest to runtime-with-instruments project
JaroslavTulach Jun 14, 2022
919ab9d
Working with WeakHashMap shall be done behind a boundary
JaroslavTulach Jun 14, 2022
d78f850
Adding withDebug commands to runtime-with-instruments
JaroslavTulach Jun 14, 2022
a84ae04
Use pattern matching to find SimpleUpdate case class
JaroslavTulach Jun 14, 2022
2cc359e
Proper formatting
JaroslavTulach Jun 14, 2022
d754e16
Update the Truffle AST from a provided IR.Expression
JaroslavTulach Jun 14, 2022
c80567a
Share conversion of number literal to runtime value between parsing a…
JaroslavTulach Jun 14, 2022
c156570
Subsequent updates scan the previously modified nodes for a match
JaroslavTulach Jun 14, 2022
a680c25
Uses SlowToInstrumentTag. RuntimeInstrumentTest 8:3 with should not i…
JaroslavTulach Jun 15, 2022
56131f3
Trading one again failing test for three passing. Score is 10:1
JaroslavTulach Jun 15, 2022
4c039ff
All 11 tests of RuntimeInstrumentTest is passing
JaroslavTulach Jun 15, 2022
41c81e0
Removing LocationFilter
JaroslavTulach Jun 15, 2022
3a10046
Always expect ClosureRootNode from the entryCallTarget
JaroslavTulach Jun 15, 2022
cc94d61
Renamed to AvoidIdInstrumentationTag
JaroslavTulach Jun 15, 2022
4891467
No changes in test needed
JaroslavTulach Jun 15, 2022
03ddb48
Documenting the purpose of PatchedModuleValues
JaroslavTulach Jun 15, 2022
162ed20
Merge branch 'jtulach/RewriteLocationFilter' into wip/jtulach/Increme…
JaroslavTulach Jun 15, 2022
2d9e4ef
Documentation for ModuleSources
JaroslavTulach Jun 15, 2022
8959ba2
Clearing debris and unused imports
JaroslavTulach Jun 15, 2022
ba152bc
Compute simpleUpdate with the help of map and filter
JaroslavTulach Jun 16, 2022
08dbd6f
Javadoc clarification
JaroslavTulach Jun 16, 2022
c8f0228
Javadoc clarification
JaroslavTulach Jun 16, 2022
6c81820
Addressing another round of comments
JaroslavTulach Jun 16, 2022
9240bda
Merge branch 'develop' into wip/jtulach/IncrementalUpdates-182323784
mergify[bot] Jun 16, 2022
52630ed
Connecting documentation of setLiteralSource with PatchedModuleValues
JaroslavTulach Jun 16, 2022
fd67025
Explaining meaning of IndexRange.between(firstFunctionLine, afterFunc…
JaroslavTulach Jun 16, 2022
5d280d5
More information in Javadoc
JaroslavTulach Jun 16, 2022
0a573ed
newIR.map
JaroslavTulach Jun 16, 2022
ec417c8
newIR.map 2
JaroslavTulach Jun 16, 2022
f088163
Underscore
JaroslavTulach Jun 16, 2022
f54d387
_._2.size
JaroslavTulach Jun 16, 2022
7144dbc
Ensure only IR.Literal is returned
JaroslavTulach Jun 16, 2022
a5db906
Throw CompilerError rather than returning a string with error message
JaroslavTulach Jun 16, 2022
68a9edf
Merge branch 'develop' into wip/jtulach/IncrementalUpdates-182323784
JaroslavTulach Jun 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,7 @@
- [Fixed compiler issue related to module cache.][3367]
- [Fixed execution of defaulted arguments of Atom Constructors][3358]
- [Converting Enso Date to java.time.LocalDate and back][3374]
- [Incremental Reparsing of a Simple Edits][3508]
- [Functions with all-defaulted arguments now execute automatically][3414]
- [Provide `tagValues` for function arguments in the language server][3422]
- [Delay construction of Truffle nodes to speed initialization][3429]
Expand All @@ -255,6 +256,7 @@
[3360]: https://github.com/enso-org/enso/pull/3360
[3367]: https://github.com/enso-org/enso/pull/3367
[3374]: https://github.com/enso-org/enso/pull/3374
[3508]: https://github.com/enso-org/enso/pull/3508
[3412]: https://github.com/enso-org/enso/pull/3412
[3414]: https://github.com/enso-org/enso/pull/3414
[3417]: https://github.com/enso-org/enso/pull/3417
Expand Down
4 changes: 4 additions & 0 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -1322,8 +1322,10 @@ lazy val `runtime-instrument-runtime-server` = (project in file("engine/runtime-
lazy val `runtime-with-instruments` = (project in file("engine/runtime-with-instruments"))
.configs(Benchmark)
.settings(
frgaalJavaCompilerSetting,
inConfig(Compile)(truffleRunOptionsSettings),
inConfig(Benchmark)(Defaults.testSettings),
commands += WithDebugCommand.withDebug,
Benchmark / javacOptions --= Seq("-source", frgaalSourceLevel),
Test / javaOptions ++= Seq(
"-Dgraalvm.locatorDisabled=true",
Expand All @@ -1335,6 +1337,8 @@ lazy val `runtime-with-instruments` = (project in file("engine/runtime-with-ins
),
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.graalvm.truffle" % "truffle-api" % graalVersion % Test,
"org.graalvm.truffle" % "truffle-dsl-processor" % graalVersion % Test,
),
// Note [Unmanaged Classpath]
Test / unmanagedClasspath += (baseDirectory.value / ".." / ".." / "app" / "gui" / "view" / "graph-editor" / "src" / "builtin" / "visualization" / "native" / "inc"),
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package org.enso.interpreter.instrument;

import com.oracle.truffle.api.CallTarget;
import com.oracle.truffle.api.RootCallTarget;
import com.oracle.truffle.api.Truffle;
import com.oracle.truffle.api.frame.FrameInstance;
import com.oracle.truffle.api.frame.FrameInstanceVisitor;
Expand All @@ -9,25 +10,25 @@
import com.oracle.truffle.api.interop.InteropException;
import com.oracle.truffle.api.interop.InteropLibrary;
import com.oracle.truffle.api.nodes.Node;
import com.oracle.truffle.api.nodes.RootNode;
import org.enso.interpreter.instrument.execution.LocationFilter;
import com.oracle.truffle.api.source.SourceSection;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import org.enso.interpreter.instrument.execution.Timer;
import org.enso.interpreter.instrument.profiling.ExecutionTime;
import org.enso.interpreter.instrument.profiling.ProfilingInfo;
import org.enso.interpreter.node.EnsoRootNode;
import org.enso.interpreter.node.ExpressionNode;
import org.enso.interpreter.node.MethodRootNode;
import org.enso.interpreter.node.callable.FunctionCallInstrumentationNode;
import org.enso.interpreter.runtime.control.TailCallException;
import org.enso.interpreter.runtime.error.PanicException;
import org.enso.interpreter.runtime.error.PanicSentinel;
import org.enso.interpreter.runtime.tag.IdentifiedTag;
import org.enso.interpreter.runtime.type.Types;
import org.enso.logger.masking.MaskedString;
import org.enso.pkg.QualifiedName;
import org.enso.interpreter.runtime.Module;

import java.util.*;
import java.util.function.Consumer;
import org.enso.interpreter.node.ClosureRootNode;
import org.enso.interpreter.runtime.tag.AvoidIdInstrumentationTag;

/** An instrument for getting values from AST-identified expressions. */
@TruffleInstrument.Registration(
Expand Down Expand Up @@ -231,6 +232,8 @@ public void onReturnExceptional(EventContext context, VirtualFrame frame, Throwa
context, frame, new PanicSentinel(panicException, context.getInstrumentedNode()));
} else if (exception instanceof PanicSentinel) {
onReturnValue(context, frame, exception);
} else if (UNWIND_HELPER.patchedValue(exception) != null) {
onReturnValue(context, frame, UNWIND_HELPER.patchedValue(exception));
}
}

Expand Down Expand Up @@ -278,8 +281,8 @@ private UUID getNodeId(Node node) {
/**
* Attach a new listener to observe identified nodes within given function.
*
* @param module module that contains the code
* @param entryCallTarget the call target being observed.
* @param locationFilter the location filter.
* @param cache the precomputed expression values.
* @param methodCallsCache the storage tracking the executed method calls.
* @param syncState the synchronization state of runtime updates.
Expand All @@ -292,8 +295,8 @@ private UUID getNodeId(Node node) {
*/
@Override
public EventBinding<ExecutionEventListener> bind(
Module module,
CallTarget entryCallTarget,
LocationFilter locationFilter,
RuntimeCache cache,
MethodCallsCache methodCallsCache,
UpdatesSynchronizationState syncState,
Expand All @@ -302,12 +305,17 @@ public EventBinding<ExecutionEventListener> bind(
Consumer<IdExecutionInstrument.ExpressionValue> onComputedCallback,
Consumer<IdExecutionInstrument.ExpressionValue> onCachedCallback,
Consumer<Exception> onExceptionalCallback) {
SourceSectionFilter filter =
SourceSectionFilter.newBuilder()
.tagIs(StandardTags.ExpressionTag.class, StandardTags.CallTag.class)
.tagIs(IdentifiedTag.class)
.sourceSectionEquals(locationFilter.getSections())
.build();
var builder = SourceSectionFilter.newBuilder()
.tagIs(StandardTags.ExpressionTag.class, StandardTags.CallTag.class)
.tagIs(IdentifiedTag.class)
.tagIsNot(AvoidIdInstrumentationTag.class)
.sourceIs(module::isModuleSource);

if (entryCallTarget instanceof RootCallTarget r && r.getRootNode() instanceof ClosureRootNode c && c.getSourceSection() != null) {
SourceSection s = c.getSourceSection();
builder.lineIn(SourceSectionFilter.IndexRange.between(s.getStartLine(), s.getEndLine() + 1));
}
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
SourceSectionFilter filter = builder.build();

return env.getInstrumenter()
.attachExecutionEventListener(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
package org.enso.interpreter.test;

import com.oracle.truffle.api.instrumentation.EventContext;
import com.oracle.truffle.api.instrumentation.ExecutionEventNode;
import com.oracle.truffle.api.instrumentation.ExecutionEventNodeFactory;
import com.oracle.truffle.api.instrumentation.SourceSectionFilter;
import com.oracle.truffle.api.instrumentation.TruffleInstrument;
import com.oracle.truffle.api.nodes.Node;
import com.oracle.truffle.api.source.SourceSection;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.function.Function;
import static org.junit.Assert.fail;

/** Testing instrument to control newly created nodes. */
@TruffleInstrument.Registration(
id = NodeCountingTestInstrument.INSTRUMENT_ID,
services = NodeCountingTestInstrument.class)
public class NodeCountingTestInstrument extends TruffleInstrument {
public static final String INSTRUMENT_ID = "node-count-test";
private Map<Node, Node> all = new ConcurrentHashMap<>();
private Map<Class, List<Node>> counter = new ConcurrentHashMap<>();
private Env env;

@Override
protected void onCreate(Env env) {
env.registerService(this);
this.env = env;
}

public void enable() {
this.env
.getInstrumenter()
.attachExecutionEventFactory(SourceSectionFilter.ANY, new CountingFactory());
}

public Map<Class, List<Node>> assertNewNodes(String msg, int min, int max) {
Map<Class, List<Node>> prev = counter;
long value = prev.values().stream().mapToInt(List::size).sum();

Function<String, String> dump =
(txt) -> {
var sb = new StringBuilder(txt);
prev.values().stream()
.forEach(
(t) -> {
t.forEach(
(n) -> {
dumpNode("", n, sb);
});
});
return sb.toString();
};

if (value < min) {
fail(dump.apply(msg + ". Minimal size should be " + min + ", but was: " + value + " in"));
}
if (value > max) {
fail(dump.apply(msg + ". Maximal size should be " + max + ", but was: " + value + " in"));
}
counter = new ConcurrentHashMap<>();
return prev;
}

private void dumpNode(String indent, Node n, StringBuilder sb) {
sb.append("\n").append(indent);
sb.append(n.getClass().getName());
final SourceSection ss = n.getSourceSection();
if (ss != null) {
sb.append(" @ ").append(ss.getSource().getName()).append(":").append(ss.getStartLine());
}
}

private final class CountingFactory implements ExecutionEventNodeFactory {
@Override
public ExecutionEventNode create(EventContext context) {
final Node node = context.getInstrumentedNode();
if (all.put(node, node) == null) {
counter.computeIfAbsent(node.getClass(), (__) -> new CopyOnWriteArrayList<>()).add(node);
}
return null;
}
}
}
Loading