Skip to content

all/any builtins implemented using Truffle #239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 33 commits into from
Nov 16, 2021
Merged

all/any builtins implemented using Truffle #239

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
d559e88
Non functional Truffle implementation of "all" builtin - test
OctaveLarose Oct 8, 2021
334be4c
all() and any() builtin functions implemented in GraalPython. Only fo…
OctaveLarose Oct 11, 2021
2bdbfa6
Temporarily removed any(), all() implementation now functional on dic…
OctaveLarose Oct 11, 2021
d63e8b5
all() now throws correctly if given a non-iterable object
OctaveLarose Oct 11, 2021
0b3c21c
Functional all() implementation for every iterable (famous last words)
OctaveLarose Oct 12, 2021
a82ba4f
Added any() builtin, using very similar logic to all()
OctaveLarose Oct 12, 2021
73ac20d
Specializations on lists/sets/tuples/dicts for all/any
OctaveLarose Oct 12, 2021
3df7af0
Simply returns True when all() is given a dict
OctaveLarose Oct 13, 2021
70d07cb
Fixed dict behaviour (now checks keys instead of entries)
OctaveLarose Oct 18, 2021
cd5bd76
Using GetNextNode + removed warnings
OctaveLarose Oct 18, 2021
d2a29a3
Added parent class to AllNode/AnyNode + fixed a bug related to Specia…
OctaveLarose Oct 18, 2021
24c18fa
Removed fall-through bug + removed checkHashEntries()
OctaveLarose Oct 21, 2021
a5ab869
Using HashingStorageLibrary instead of PHashingCollection
OctaveLarose Oct 21, 2021
a432076
Optimization attempt by specializing on SequenceStorage subtypes
OctaveLarose Oct 21, 2021
ac14bc8
Utilizing LenNode for generic SequenceStorage objects
OctaveLarose Oct 21, 2021
8b6104a
Attempts to cache PyObjectIsTrueNode and LenNode, reduces code duplic…
OctaveLarose Oct 22, 2021
4a8c67f
Removed nodeType field
OctaveLarose Oct 22, 2021
3107467
Removed shouldStopIteration function by "inlining" it
OctaveLarose Oct 28, 2021
fb93d4c
AllOrAnySequenceStorageNode: Removed factory generation annotation + …
OctaveLarose Oct 28, 2021
784b19e
AllOrAnyNode: added child PyObjectIsTrueNode attribute
OctaveLarose Oct 28, 2021
1828e55
Removed a broken import and an unused one in BuiltinFunctions
OctaveLarose Oct 29, 2021
13a3b26
Removed unnecessary public qualifiers + changed AllOrAnyNode's parent…
OctaveLarose Nov 1, 2021
80059f8
mx checkstyle (had left a public keyword behind accidentally) + code …
OctaveLarose Nov 1, 2021
65bbacf
Shared annotation for AllOrAnyNode/GetClassNode instances
OctaveLarose Nov 1, 2021
0fb7459
Fixed all builtin behavior
OctaveLarose Nov 3, 2021
16b54b6
Ran Eclipse formatter for BuiltinFunctions changes
OctaveLarose Nov 3, 2021
aa7cde4
Removed public identifier on AllOrAnyNode + added static keyword to e…
OctaveLarose Nov 8, 2021
5dd75dc
Merged doSet and doDict specialization methods
OctaveLarose Nov 8, 2021
de15e38
Loop profiling for all/any related methods
OctaveLarose Nov 8, 2021
522e6ed
Fixed infinite loops + profiling for HashingStorage
OctaveLarose Nov 10, 2021
bceb24a
Fixed loop profiling for doObject in all/any implementations
OctaveLarose Nov 10, 2021
66661b4
loopConditionProfile.profileCounted() calls instead of just profile()
OctaveLarose Nov 10, 2021
fa07d30
Fixed doBoolSequence() skipping some iterable elements
OctaveLarose Nov 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
222 changes: 222 additions & 0 deletions ...om.oracle.graal.python/src/com/oracle/graal/python/builtins/modules/BuiltinFunctions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
import static com.oracle.graal.python.builtins.objects.PNone.NO_VALUE;
import static com.oracle.graal.python.builtins.objects.PNotImplemented.NOT_IMPLEMENTED;
import static com.oracle.graal.python.nodes.BuiltinNames.ABS;
import static com.oracle.graal.python.nodes.BuiltinNames.ALL;
import static com.oracle.graal.python.nodes.BuiltinNames.ANY;
import static com.oracle.graal.python.nodes.BuiltinNames.ASCII;
import static com.oracle.graal.python.nodes.BuiltinNames.BIN;
import static com.oracle.graal.python.nodes.BuiltinNames.BREAKPOINT;
Expand Down Expand Up @@ -102,7 +104,9 @@
import com.oracle.graal.python.builtins.objects.code.PCode;
import com.oracle.graal.python.builtins.objects.common.DynamicObjectStorage;
import com.oracle.graal.python.builtins.objects.common.HashingCollectionNodes;
import com.oracle.graal.python.builtins.objects.common.HashingStorage;
import com.oracle.graal.python.builtins.objects.common.HashingStorageLibrary;
import com.oracle.graal.python.builtins.objects.common.PHashingCollection;
import com.oracle.graal.python.builtins.objects.common.SequenceNodes.GetObjectArrayNode;
import com.oracle.graal.python.builtins.objects.common.SequenceNodesFactory.GetObjectArrayNodeGen;
import com.oracle.graal.python.builtins.objects.common.SequenceStorageNodes;
Expand All @@ -119,6 +123,7 @@
import com.oracle.graal.python.builtins.objects.module.PythonModule;
import com.oracle.graal.python.builtins.objects.object.ObjectNodes;
import com.oracle.graal.python.builtins.objects.object.PythonObject;
import com.oracle.graal.python.builtins.objects.set.PBaseSet;
import com.oracle.graal.python.builtins.objects.str.PString;
import com.oracle.graal.python.builtins.objects.tuple.PTuple;
import com.oracle.graal.python.builtins.objects.type.SpecialMethodSlot;
Expand All @@ -134,6 +139,7 @@
import com.oracle.graal.python.lib.PyObjectGetAttr;
import com.oracle.graal.python.lib.PyObjectGetIter;
import com.oracle.graal.python.lib.PyObjectHashNode;
import com.oracle.graal.python.lib.PyObjectIsTrueNode;
import com.oracle.graal.python.lib.PyObjectLookupAttr;
import com.oracle.graal.python.lib.PyObjectReprAsObjectNode;
import com.oracle.graal.python.lib.PyObjectSizeNode;
Expand All @@ -144,6 +150,7 @@
import com.oracle.graal.python.nodes.ErrorMessages;
import com.oracle.graal.python.nodes.GraalPythonTranslationErrorNode;
import com.oracle.graal.python.nodes.PGuards;
import com.oracle.graal.python.nodes.PNodeWithContext;
import com.oracle.graal.python.nodes.PNodeWithRaise;
import com.oracle.graal.python.nodes.PRaiseNode;
import com.oracle.graal.python.nodes.PRootNode;
Expand Down Expand Up @@ -204,6 +211,8 @@
import com.oracle.graal.python.runtime.exception.PException;
import com.oracle.graal.python.runtime.exception.PythonErrorType;
import com.oracle.graal.python.runtime.object.PythonObjectFactory;
import com.oracle.graal.python.runtime.sequence.storage.BoolSequenceStorage;
import com.oracle.graal.python.runtime.sequence.storage.IntSequenceStorage;
import com.oracle.graal.python.runtime.sequence.storage.SequenceStorage;
import com.oracle.graal.python.util.CharsetMapping;
import com.oracle.graal.python.util.PythonUtils;
Expand Down Expand Up @@ -232,11 +241,13 @@
import com.oracle.truffle.api.library.CachedLibrary;
import com.oracle.truffle.api.nodes.ExplodeLoop;
import com.oracle.truffle.api.nodes.ExplodeLoop.LoopExplosionKind;
import com.oracle.truffle.api.nodes.LoopNode;
import com.oracle.truffle.api.nodes.Node;
import com.oracle.truffle.api.nodes.RootNode;
import com.oracle.truffle.api.nodes.UnexpectedResultException;
import com.oracle.truffle.api.profiles.BranchProfile;
import com.oracle.truffle.api.profiles.ConditionProfile;
import com.oracle.truffle.api.profiles.LoopConditionProfile;
import com.oracle.truffle.api.source.Source;
import com.oracle.truffle.api.utilities.TriState;

Expand Down Expand Up @@ -286,6 +297,217 @@ public Object absObject(VirtualFrame frame, Object object,
}
}

/**
* Common class for all() and any() operations, as their logic and behaviors are very similar.
*/
abstract static class AllOrAnyNode extends PNodeWithContext {
enum NodeType {
ALL,
ANY
}

@Child private PyObjectIsTrueNode isTrueNode = PyObjectIsTrueNode.create();

final private LoopConditionProfile loopConditionProfile = LoopConditionProfile.create();

abstract boolean execute(Frame frame, Object storageObj, NodeType nodeType);

@Specialization
boolean doBoolSequence(VirtualFrame frame,
BoolSequenceStorage sequenceStorage,
NodeType nodeType) {
boolean[] internalArray = sequenceStorage.getInternalBoolArray();
int seqLength = sequenceStorage.length();

loopConditionProfile.profileCounted(seqLength);
for (int i = 0; loopConditionProfile.inject(i < seqLength); i++) {
if (nodeType == NodeType.ALL && !isTrueNode.execute(frame, internalArray[i])) {
return false;
} else if (nodeType == NodeType.ANY && isTrueNode.execute(frame, internalArray[i])) {
return true;
}
}

return nodeType == NodeType.ALL;
}

@Specialization
boolean doIntSequence(VirtualFrame frame,
IntSequenceStorage sequenceStorage,
NodeType nodeType) {
int[] internalArray = sequenceStorage.getInternalIntArray();
int seqLength = sequenceStorage.length();

loopConditionProfile.profileCounted(seqLength);
for (int i = 0; loopConditionProfile.inject(i < seqLength); i++) {
if (nodeType == NodeType.ALL && !isTrueNode.execute(frame, internalArray[i])) {
return false;
} else if (nodeType == NodeType.ANY && isTrueNode.execute(frame, internalArray[i])) {
return true;
}
}

return nodeType == NodeType.ALL;
}

@Specialization
boolean doGenericSequence(VirtualFrame frame,
SequenceStorage sequenceStorage,
NodeType nodeType,
@Cached SequenceStorageNodes.LenNode lenNode) {
Object[] internalArray = sequenceStorage.getInternalArray();
int seqLength = lenNode.execute(sequenceStorage);

loopConditionProfile.profileCounted(seqLength);
for (int i = 0; loopConditionProfile.inject(i < seqLength); i++) {
if (nodeType == NodeType.ALL && !isTrueNode.execute(frame, internalArray[i])) {
return false;
} else if (nodeType == NodeType.ANY && isTrueNode.execute(frame, internalArray[i])) {
return true;
}
}

return nodeType == NodeType.ALL;
}

@Specialization(limit = "3")
protected boolean doHashStorage(VirtualFrame frame,
HashingStorage hashingStorage,
NodeType nodeType,
@CachedLibrary("hashingStorage") HashingStorageLibrary hlib) {
HashingStorageLibrary.HashingStorageIterator<Object> keysIter = hlib.keys(hashingStorage).iterator();
int seqLength = hlib.length(hashingStorage);

loopConditionProfile.profileCounted(seqLength);
for (int i = 0; loopConditionProfile.inject(i < seqLength); i++) {
Object key = keysIter.next();
if (nodeType == NodeType.ALL) {
if (!isTrueNode.execute(frame, key)) {
return false;
}
} else if (nodeType == NodeType.ANY && isTrueNode.execute(frame, key)) {
return true;
}
}

return nodeType == NodeType.ALL;
}
}

@Builtin(name = ALL, minNumOfPositionalArgs = 1)
@GenerateNodeFactory
public abstract static class AllNode extends PythonUnaryBuiltinNode {

@Specialization(guards = "cannotBeOverridden(object, getClassNode)", limit = "1")
static boolean doList(VirtualFrame frame,
PList object,
@SuppressWarnings("unused") @Shared("getClassNode") @Cached GetClassNode getClassNode,
@Shared("allOrAnyNode") @Cached AllOrAnyNode allOrAnyNode) {
return allOrAnyNode.execute(frame, object.getSequenceStorage(), AllOrAnyNode.NodeType.ALL);
}

@Specialization(guards = "cannotBeOverridden(object, getClassNode)", limit = "1")
static boolean doTuple(VirtualFrame frame,
PTuple object,
@SuppressWarnings("unused") @Shared("getClassNode") @Cached GetClassNode getClassNode,
@Shared("allOrAnyNode") @Cached AllOrAnyNode allOrAnyNode) {
return allOrAnyNode.execute(frame, object.getSequenceStorage(), AllOrAnyNode.NodeType.ALL);
}

@Specialization(guards = "cannotBeOverridden(object, getClassNode)", limit = "1")
static boolean doHashColl(VirtualFrame frame,
PHashingCollection object,
@SuppressWarnings("unused") @Shared("getClassNode") @Cached GetClassNode getClassNode,
@Shared("allOrAnyNode") @Cached AllOrAnyNode allOrAnyNode) {
return allOrAnyNode.execute(frame, object.getDictStorage(), AllOrAnyNode.NodeType.ALL);
}

@Specialization
boolean doObject(VirtualFrame frame,
Object object,
@Cached PyObjectGetIter getIter,
@Cached GetNextNode nextNode,
@Cached IsBuiltinClassProfile errorProfile,
@Cached PyObjectIsTrueNode isTrueNode) {
Object iterator = getIter.execute(frame, object);
int nbrIter = 0;

while (true) {
try {
Object next = nextNode.execute(frame, iterator);
nbrIter++;
if (!isTrueNode.execute(frame, next)) {
return false;
}
} catch (PException e) {
e.expectStopIteration(errorProfile);
break;
} finally {
LoopNode.reportLoopCount(this, nbrIter);
}
}

return true;
}
}

@Builtin(name = ANY, minNumOfPositionalArgs = 1)
@GenerateNodeFactory
public abstract static class AnyNode extends PythonUnaryBuiltinNode {

@Specialization(guards = "cannotBeOverridden(object, getClassNode)", limit = "1")
static boolean doList(VirtualFrame frame,
PList object,
@SuppressWarnings("unused") @Shared("getClassNode") @Cached GetClassNode getClassNode,
@Shared("allOrAnyNode") @Cached AllOrAnyNode allOrAnyNode) {
return allOrAnyNode.execute(frame, object.getSequenceStorage(), AllOrAnyNode.NodeType.ANY);
}

@Specialization(guards = "cannotBeOverridden(object, getClassNode)", limit = "1")
static boolean doTuple(VirtualFrame frame,
PTuple object,
@SuppressWarnings("unused") @Shared("getClassNode") @Cached GetClassNode getClassNode,
@Shared("allOrAnyNode") @Cached AllOrAnyNode allOrAnyNode) {
return allOrAnyNode.execute(frame, object.getSequenceStorage(), AllOrAnyNode.NodeType.ANY);
}

@Specialization(guards = "cannotBeOverridden(object, getClassNode)", limit = "1")
static boolean doHashColl(VirtualFrame frame,
PHashingCollection object,
@SuppressWarnings("unused") @Shared("getClassNode") @Cached GetClassNode getClassNode,
@Shared("allOrAnyNode") @Cached AllOrAnyNode allOrAnyNode) {
return allOrAnyNode.execute(frame, object.getDictStorage(), AllOrAnyNode.NodeType.ANY);
}

@Specialization
boolean doObject(VirtualFrame frame,
Object object,
@Cached PyObjectGetIter getIter,
@Cached GetNextNode nextNode,
@Cached IsBuiltinClassProfile errorProfile,
@Cached PyObjectIsTrueNode isTrueNode) {
Object iterator = getIter.execute(frame, object);
int nbrIter = 0;

while (true) {
try {
Object next = nextNode.execute(frame, iterator);
nbrIter++;
if (isTrueNode.execute(frame, next)) {
return true;
}
} catch (PException e) {
e.expectStopIteration(errorProfile);
break;
} finally {
LoopNode.reportLoopCount(this, nbrIter);
}
}

return false;
}
}

// bin(object)
@Builtin(name = BIN, minNumOfPositionalArgs = 1)
@TypeSystemReference(PythonArithmeticTypes.class)
Expand Down
16 changes: 0 additions & 16 deletions graalpython/lib-graalpython/functions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,22 +43,6 @@ def hasattr(obj, key):
return getattr(obj, key, default) is not default


@__graalpython__.builtin
def any(iterable):
for i in iterable:
if i:
return True
return False


@__graalpython__.builtin
def all(iterable):
for i in iterable:
if not i:
return False
return True


from sys import _getframe as __getframe__


Expand Down