Unboxed atoms #3862

kustosz · 2022-11-09T14:11:49Z

Pull Request Description

Introduces unboxed (and arity-specialized) storage schemes for Atoms. It results in improvements both in memory consumption and runtime.
Memory wise: instead of using an array, we now use object fields. We also enable unboxing. This cuts a good few pointers in an unboxed object. E.g. a quadruple of integers is now 64 bytes (4x8 bytes for long fields + 16 bytes for layout and constructor pointers + 16 bytes for a class header). It used to be 168 bytes (4x24 bytes for boxed Longs + 16 bytes for array header + 32 bytes for array contents + 8 bytes for constructor ptr + 16 bytes for class header), so we're saving 104 bytes a piece. In the least impressive scenarios (all-boxed fields) we're saving 8 bytes per object (saving 16 bytes for array header, using 8 bytes for the new layout field). In the most-benchmarked case (list of longs), we save 32 bytes per cons-cell.
Time wise:
All list-summing benchmarks observe a ~2x speedup. List generation benchmarks get ~25x speedups, probably both due to less GC activity and better allocation characteristics (only allocating one object per Cons, rather than Cons + Object[] for fields). The "map-reverse" family gets a neat 10x speedup (part of the work is reading, which is 2x faster, the other is allocating, which is now 25x faster, we end up with 10x when combined).

Important Notes

Checklist

Please include the following checklist in your PR:

The documentation has been updated if necessary.
All code conforms to the
Scala,
Java,
and
Rust
style guides.
All code has been tested:
- Unit tests have been written where possible.
- If GUI codebase was changed: Enso GUI was tested when built using BOTH
  ./run ide build and ./run ide watch.

JaroslavTulach · 2022-11-10T08:58:08Z

engine/runtime/src/main/java/org/enso/interpreter/node/callable/IndirectInvokeCallableNode.java

-import org.enso.interpreter.runtime.callable.atom.Atom;
-import org.enso.interpreter.runtime.callable.atom.AtomConstructor;
+import org.enso.interpreter.runtime.data.struct.Struct;
+import org.enso.interpreter.runtime.data.struct.AtomConstructor;


More than sixty changed files! I was wondering why the change is so huge. Consider doing the rename of Atom to Struct as a separate PR.

JaroslavTulach

Introduction of AtomL0 seems like the right approach that we can expand on.

JaroslavTulach · 2022-12-05T15:38:59Z

engine/runtime/src/main/java/org/enso/interpreter/node/expression/atom/InstantiateNode.java

+    @Specialization(guards = "arity == 2")
+    Object do2(AtomConstructor constructor, Object[] arguments) {
+      if (arguments[0] instanceof Long) {
+        return new AtomLO(constructor, (long) arguments[0], arguments[1]);


This is the "demo" shortcut.

JaroslavTulach · 2022-12-05T15:40:18Z

engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/Atom.java

-/** A runtime representation of an Atom in Enso. */
+/**
+ * A runtime representation of an Atom in Enso.
+ */
 @ExportLibrary(InteropLibrary.class)


Is it of any use to export these libraries from here?

JaroslavTulach · 2022-12-05T15:42:54Z

engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/StructsLibrary.java

+import com.oracle.truffle.api.library.LibraryFactory;
+
+@GenerateLibrary
+public abstract class StructsLibrary extends Library {


I've heard Library isn't the most efficient (in terms of AST size) concept for use inside of a language, but I think we are not that far to switch to other concepts yet.

Didn't Graal Team also mention that they want to eventually get rid of it as well?
I hope there will be an alternative.

I asked Christian Humer and he said: (Interop) Library is still useful and necessary for communication between different language implementations.

JaroslavTulach · 2022-12-05T15:48:32Z

engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/Atom.java

+    @ExportMessage
+    @ExplodeLoop
+    public boolean isMemberReadable(String member) {
+        for (int i = 0; i < constructor.getArity(); i++) {


This is not going to @ExplodeLoop I am afraid. constructor isn't compilation constant. You'd need some profile, I think.

# Conflicts: # engine/runtime/src/main/java/org/enso/interpreter/node/ExpressionNode.java # engine/runtime/src/main/java/org/enso/interpreter/node/expression/atom/InstantiateNode.java # engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/error/InvalidArrayIndexError.java # engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/error/PolyglotError.java # engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/number/decimal/CompareToNode.java # engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/text/util/TypeToDisplayTextNode.java # engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/Atom.java # engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/AtomConstructor.java # engine/runtime/src/main/java/org/enso/interpreter/runtime/type/TypesFromProxy.java

# Conflicts: # engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/meta/EqualsAnyNode.java # engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/Atom.java

hubertp

Can you update the description of the PR, please?

JaroslavTulach

Nice work. I am leaving few comments but all seems green and that's (almost) all that matters!

JaroslavTulach · 2023-01-24T03:29:01Z

engine/runtime/src/bench/scala/org/enso/interpreter/bench/fixtures/semantic/AtomFixtures.scala

@@ -121,10 +121,10 @@ class AtomFixtures extends DefaultInterpreterRunner {
      |    List.Cons h t -> @Tail_Call t.mapReverse f (List.Cons (f h) acc)
      |    _ -> acc
      |
-      |main = self -> list ->
+      |main = list ->


Shouldn't the JMH part of the benchmark check the returned value to make sure we execute some computation in the benchmark?

It probably should. I would suggest when we run the benchmarks in CI, we assert some non-trivial runtime (e.g. more than 1ms). Not a part of this PR tho.

JaroslavTulach · 2023-01-24T03:33:18Z

engine/runtime/src/main/java/org/enso/interpreter/instrument/HostObjectDebugWrapper.java

    if (object instanceof Atom atom) {
-      Object[] fields = atom.getFields();
+      Object[] fields = structs.getFields(atom);


Instead of atom.getFields() we'll have a StructsLibrary that will allow us to speculate on the shape of the atom. Would it make sense to have getter of long for an index another getter of double for an index and then generic getter of Object for an index?

Once the code is PE compiled, it makes no difference, but it would show some difference before compilation.

Yeah, this is possibly a good future improvement. I don't want to add this now, because we have no benchmarks that prove this is a problem at all, and it is significant additional complexity.

JaroslavTulach · 2023-01-24T03:36:20Z

engine/runtime/src/main/java/org/enso/interpreter/node/expression/atom/GetFieldNode.java

@@ -37,7 +41,7 @@ public GetFieldNode(TruffleLanguage<?> language, int index, Type type, String na
  public Object execute(VirtualFrame frame) {
    // this is safe, as only Atoms will ever get here through method dispatch.
    Atom atom = (Atom) Function.ArgumentsHelper.getPositionalArguments(frame.getArguments())[0];
-    return atom.getFields()[index];
+    return structs.getField(atom, index);


If we had getLongField, getDoubleField and getField we could speculate here possibly avoiding boxing before the code gets PEed.

JaroslavTulach · 2023-01-24T03:38:01Z

engine/runtime/src/main/java/org/enso/interpreter/node/expression/atom/GetFieldNode.java


 @NodeInfo(shortName = "get_field", description = "A base for auto-generated Atom getters.")
 public class GetFieldNode extends RootNode {
  private final int index;
  private final String name;
  private final Type type;

+  private @Child StructsLibrary structs = StructsLibrary.getFactory().createDispatched(10);


Speculating for 10 different shapes of atoms. Isn't that too much? Do we observe any change in performance when we go to three or even two?

We don't, but we don't have benchmarks that saturate this at all. Intuitively: I chose 10 because it is enough to handle all 2-field cases and it is much cheaper to go through the cache checks here 10 times than fire the uncached version even once.

JaroslavTulach · 2023-01-24T03:48:56Z

engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/Atom.java

   */
-  public Atom(AtomConstructor constructor, Object... fields) {
+  public Atom(AtomConstructor constructor) {


Make the constructor protected.

JaroslavTulach · 2023-01-24T04:15:59Z

engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/unboxing/Layout.java

+          if (layouts.length != this.unboxedLayouts.length) {
+            // Layouts changed since we last tried; Update & try again
+            updateFromConstructor();
+            return execute(arguments);


execute shall probably not be executed while holding the lock. Release the lock first, then call execute.

JaroslavTulach · 2023-01-24T04:19:55Z

engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/unboxing/Layout.java

+
+          // Layouts didn't change; just create a new one and register it
+          var newLayout = Layout.create(arity, flags);
+          constructor.addLayout(newLayout);


I am not very happy when I see an internal constructor logic being performed in _layout- I'd rather have it all at a single place. Exposinglock` and letting anyone invoke some operations that would rather be atomic is fragile.

As a minimum use assert lock.isHeldByCurrentThread() (as suggested elsewhere in this review) at methods that shall only be accessed while holding the lock.

Hah good point, I wrote it quickly and forgot about it... The whole thing needs a rewrite tbh, the responsibilities are horrid.

JaroslavTulach · 2023-01-24T04:24:08Z

.../runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/unboxing/UnboxingAtom.java

+ *       {@link Double} fields before the {@link Long} fields, but this is not required or enforced
+ *       by this class.
+ *   <li>These design choices mean that to enable optimal storage of N-field atoms, we need N+1
+ *       different subclasses.


JaroslavTulach · 2023-01-24T04:28:19Z

.../runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/unboxing/UnboxingAtom.java

+   * The concrete subclass will also generate a method allowing to obtain a factory for these nodes
+   * based on dynamically-passed index, i.e. with the following signature: {@code public static
+   * NodeFactory<? extends FieldGetterNode> getFieldGetterNodeFactory(int storageIndex, boolean
+   * isDoubleIfUnboxed)}


JaroslavTulach · 2023-01-24T04:30:04Z

engine/runtime/src/main/java/org/enso/interpreter/runtime/type/TypesFromProxy.java

@@ -2,6 +2,7 @@

 import org.enso.compiler.exception.CompilerError;
 import org.enso.interpreter.runtime.builtin.Builtins;
+import org.enso.interpreter.runtime.callable.atom.Atom;


JaroslavTulach · 2023-01-24T04:35:55Z

Pull Request Description

TBD. Nothing works here, PRing not to lose it

Please update PR's description to match reality and share some performance numbers.

Slipped through review of #3862

JaroslavTulach reviewed Nov 10, 2022

View reviewed changes

start this

5c549cf

kustosz force-pushed the wip/mk/unbox-atoms branch from f3e4514 to 5c549cf Compare November 29, 2022 15:40

kustosz added 6 commits December 3, 2022 14:04

stuff

3d5d867

bring the tests back

ec91cb2

fix benchmarks

9633a6b

it works

0598c7f

fmt

e115493

undo rename

1b0860d

JaroslavTulach reviewed Dec 5, 2022

View reviewed changes

kustosz added 20 commits December 20, 2022 17:00

getters

6b2a1aa

bang my head against instantiation

dd0e161

finish up instantiation logic

565e680

refactor and get it to compile!

a295ef1

somewhat works

6c4b43b

all tests passing

4a363b9

stdlib tests too

8f8c017

GraalVM is like a Hollywood blockbuster: more explosions make it better

6f8d8b3

checkpoint on moving constructor into layout

3d23532

checkpoint: done cons in layout

5dd9753

undo cons in layout

760dbd6

uncoment the rest of benchmarks

5e536a3

unify atom benchmarks

c420a7e

checkpoint

8bfd94b

checkpoint

3668dbb

migrate to fully automated

0315948

Merge branch 'develop' into wip/mk/unbox-atoms

07d1c0b

start de-methodyfying

c5a82e8

remove all occurences of getFields

798049d

kustosz added 9 commits January 15, 2023 14:31

fmt

d7eb566

add setters

2d62205

fix a bug in debugger

7f0d32a

Merge branch 'develop' into wip/mk/unbox-atoms

f34e51a

# Conflicts: # engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/meta/EqualsAnyNode.java # engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/Atom.java

post-merge fix

c1e9a59

cleanup

683319a

fmt?

a04b7bd

changelog

7032728

add documentation

36ca9a2

kustosz marked this pull request as ready for review January 21, 2023 18:34

kustosz requested review from 4e6 and hubertp as code owners January 21, 2023 18:34

hubertp reviewed Jan 23, 2023

View reviewed changes

JaroslavTulach requested a review from Akirathan January 24, 2023 04:33

JaroslavTulach approved these changes Jan 24, 2023

View reviewed changes

kustosz added 2 commits January 24, 2023 12:56

CR feedback

6016a75

Merge branch 'develop' into wip/mk/unbox-atoms

f775ab4

kustosz added the CI: Ready to merge This PR is eligible for automatic merge label Jan 24, 2023

mergify bot merged commit 242bd52 into develop Jan 24, 2023

mergify bot deleted the wip/mk/unbox-atoms branch January 24, 2023 13:03

hubertp added a commit that referenced this pull request Jan 24, 2023

Eliminate various compiler warnings

9e495db

Slipped through review of #3862

hubertp mentioned this pull request Jan 24, 2023

Eliminate various compiler warnings #4079

Merged

mergify bot pushed a commit that referenced this pull request Jan 24, 2023

Eliminate various compiler warnings (#4079)

c85377f

Slipped through review of #3862

This was referenced Feb 6, 2023

Enable asserts when running engine tests #4918

Closed

New parser has issues with named and defaulted args #4746

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unboxed atoms #3862

Unboxed atoms #3862

kustosz commented Nov 9, 2022 •

edited

Loading

JaroslavTulach Nov 10, 2022

JaroslavTulach left a comment

JaroslavTulach Dec 5, 2022

JaroslavTulach Dec 5, 2022

JaroslavTulach Dec 5, 2022

hubertp Jan 23, 2023

JaroslavTulach Jan 24, 2023

JaroslavTulach Dec 5, 2022

hubertp left a comment

JaroslavTulach left a comment

JaroslavTulach Jan 24, 2023

kustosz Jan 24, 2023

JaroslavTulach Jan 24, 2023

kustosz Jan 24, 2023

JaroslavTulach Jan 24, 2023

JaroslavTulach Jan 24, 2023

kustosz Jan 24, 2023

JaroslavTulach Jan 24, 2023

JaroslavTulach Jan 24, 2023

JaroslavTulach Jan 24, 2023

kustosz Jan 24, 2023

JaroslavTulach Jan 24, 2023

JaroslavTulach Jan 24, 2023

JaroslavTulach Jan 24, 2023

JaroslavTulach commented Jan 24, 2023

Pull Request Description

Unboxed atoms #3862

Unboxed atoms #3862

Conversation

kustosz commented Nov 9, 2022 • edited Loading

Pull Request Description

Important Notes

Checklist

Choose a reason for hiding this comment

JaroslavTulach left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hubertp left a comment

Choose a reason for hiding this comment

JaroslavTulach left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JaroslavTulach commented Jan 24, 2023

Pull Request Description

kustosz commented Nov 9, 2022 •

edited

Loading