Repository navigation
Releases
0.7.2
Changes
The C that ppy emit c and ppy emit cpp write reads like hand-written C in every form, library included, and ppy build --library gives C callers a library they can link and errors they can handle.
Structured C
- Every function is written with
while,if/else,break,continue, andreturn. Loops with several exits, shared raise and clean-up paths, generator resumes, and regex matchers no longer fall back to labels andgoto; a failed check returns where it happens instead of jumping tofallback:. - Tests that share a target are written as one condition (
if (!seen || x > best)), and anifthat returns has noelse. - No function in
examples/falls back to labels in any form (library,--standalone,--unsafe, and C++). On 0.7.1, 27 of the 183 emitted files did.
Libraries
ppy build --librarystops withE1807when an exported function did not lower, naming the function and the reason, instead of leaving it out of the header.- The library links libm, so a C consumer needs only
pkg-config --libs <name>. - An export whose Python code raises returns 0, and
ppy_last_error()gives CPython's last line for the exception (IndexError: list index out of range), instead of killing the caller withSIGILL.ppy emit cexports print that line before they abort.
Fixed
- The safe C form no longer writes string constants that no function reads, which
-Wunused-variableflagged on Python 3.12 and 3.13.
Install or upgrade with pip install --upgrade "ppy-lang[llvm]==0.7.2".
The full repository gate passed on Python 3.12, 3.13, and 3.14, with the emitted C of the new tests run under AddressSanitizer and UndefinedBehaviorSanitizer.
0.7.0
Changes
Classes written without annotations run natively: fields take their types from every store in the project, and methods that edit linked lists, trees, and graphs in place write back to the caller's objects. The boundary reads lists, dicts, and sets in place instead of copying them. More shapes lower, and more parameters take types without annotations. Fuzzing and the new examples found wrong answers that 0.6.0 also gave, now fixed.
Objects Python passes to native code keep their native records between calls, and numbers and strings that may be None run natively.
On TheAlgorithms/Python, the functions Python calls natively went from 806 to 909 (846 to 982 without strict mode), and 387 of 400 scripts print the same as under CPython, with the other 13 skipped and none differing.
Objects written in place
- Fields without annotations take their type from every store in the project (
node.left = Node(k)makesleftaNode | None) and from what is appended into a field that starts empty. - Writes through a local that holds a field and through an object a call returns run natively, and come back to the caller's objects with their identity kept, new nodes included.
- Tuple assignment into fields (rotations) and conditional expressions of objects lower natively. A class that only satisfies a Protocol is no longer kept in Python.
Objects kept native between calls
- A project object that Python passes to native code a second time keeps its native record while the Python object lives, so later calls do not copy it and everything reachable from it. Python stays the source of truth through CPython's dict and type watchers.
findon a 10,000-node linked list went from 3.1 ms to 34 µs a call (CPython 63 µs).PPY_RESIDENT=0turns it off.
Numbers and strings that may be None
int | None,float | None,bool | None, andstr | Nonerun natively as locals, parameters, results, fields, and list and dict elements, with CPython'sis None, truth, printing, andTypeErrortext.
Reading containers in place
- Python calls of native functions that only read their lists, dicts, and sets no longer copy them; strings are borrowed and dicts are indexed only when looked up.
- A call that writes copies back only what it wrote. Drawing from
randomcosts about 60 ns a call instead of 1.6 µs. - The cost model counts native per-element costs, so functions that would be slower stay in Python.
More shapes and more types
- Implicit ends of
if/elif/elsechains, list unpacking and swaps,if not xs,*argsof numbers, string constants, nested functions reading cells, settled globals in methods,d.get(k)as number-or-None,max/minof objects,os.pathandtimeit. - Without strict mode, parameters take types from
argparse, int/float joins, signature-keeping decorators, operator dunders, barelistrefined by its calls, defaults, doctests, and body use. ppy explain module.Classlists each field's type and where it came from.
Fixed
- With
strict = false, a function behind a project decorator ran without the decorator (square(3)under a doubling decorator printed 9, not 18). - A
boolstored where anintis declared printed1in native code. - A native method writing through a local holding a field lost the write under
ppy run. - Releasing a chain of about 200,000 objects overflowed the C stack; returning a long chain to Python raised
RecursionError. - A global rebound under
globalcould be read as its first value; unpacking a natively held list of numbers bound strings;flag + flagcrashed compilation; passing a Python-only function as a value crashed compilation.
Examples
55_linked_structures(python 0.67 s,ppy run0.34 s) and56_options_and_doctests(1.18 s, 0.44 s).54_objects_and_gridswent from 0.58 s to 0.25 s.
Known limitations: classes with list or dict fields are still copied at every call from Python, and sorting or keying by values that may be None stays in Python. See the changelog for the full list.
Install or upgrade with pip install --upgrade "ppy-lang[llvm]==0.7.0".
The full repository gate passed: 2,591 tests, 51/51 examples on all three paths, 72/72 conversion claims, pylint on every example, and the documentation examples. Fuzzing ran clean across every domain, including the new --structures, --shapes, --inference, --decorators, --bools, --resident, and --optional.
0.6.0
Changes
More ordinary Python runs natively under ppy run: functions that print, read input, or call back into Python; functions that read module globals, take the project's objects, or are called with keywords; isinstance, chained comparisons, in, integer powers, and generators passed around as values; and random, math, heapq, bisect, itertools, Counter, defaultdict, and lru_cache. The Python boundary copies containers and objects in C. Without strict mode, unannotated parameters take their types from the project's calls, and ppy run no longer refuses a program CPython runs.
On TheAlgorithms/Python, the functions Python calls natively went from 231 to 806 (846 without strict mode), and 387 of 400 scripts print the same as under CPython, with the other 13 skipped as nondeterministic or slow and none differing.
Effects, globals, and objects
- Functions that print, read input, read and write text files, or call Python functions compile. Output is held per call, so a fallback never prints twice;
input(),flush=True, files, and calls into Python are barriers after which nothing falls back. - A function that reads a module global bound once runs natively. Objects of the project's classes, and dataclass values in containers, cross between Python and native code, with shared objects and cycles kept shared.
More of Python native
isinstance, chained comparisons,inover tuples, sets, andrange, integer**andpow, moreforloops, generators returned or passed on, ande.args.- Keyword arguments and defaults, results from Python functions that change nothing, and nested functions that share nothing with the function around them.
- A native function given an
intfor afloatparameter returns what CPython returns:f(0)is0.
The standard library
random(draw for draw with CPython, module-level andrandom.Randominstances),math,heapq,bisect,itertools,deque,defaultdict,Counter,OrderedDict,functools.cache/lru_cache/reduce/partial/cmp_to_key,operator, andstring.
The boundary
- Containers and objects cross in C: about 8 to 35 times faster than in 0.5.0. A native two-int call costs 29 ns against 30 ns in CPython, and a call with keywords or defaults about 40 ns, bound in C.
- The cost model is measured: a function gets a Python boundary only where the call is cheaper than CPython's.
Unannotated code and the checker
- Without strict mode, parameters take their types from calls, defaults, and doctests, and
ppy explainshows where each came from. A Python caller passing other types runs the Python body. --no-strictturns what the analysis cannot follow (unresolvable imports,eval,globals(), star imports, metaclasses) into warnings, and that code runs on CPython.typing.Self,Generic[T], named tuples,MutableSequence, and thecollectionstypes are accepted.ppy runwithout the LLVM extra runs the program on CPython with one warning.
Fixed
- Cached recursion ran without its cache under
ppy run; a string-keyed map copy asked for 64 GiB; a double free after a native call raised; a segfault past the recursion limit; a nested function reading a lent list crashed;rng.choiceover strings freed its result;d.get(k)without a default crashedppy run; temporaries leaked when a check raised insidetry; several Python-backend folds that hid an exception.
Examples
51_clinic,52_unannotated,53_caches_and_counters, and54_objects_and_grids.
See the changelog for the full list and the known limitations.
Install or upgrade with pip install --upgrade "ppy-lang[llvm]==0.6.0".
The full repository gate passed: 2,463 tests, 49/49 examples on all three paths, 70/70 conversion claims, pylint on every example, and the documentation examples. The emitted C and C++ of the new tests runs clean under AddressSanitizer with leak detection.
0.5.0
Changes
Ordinary Python goes native without a rewrite. Plain lists, dicts, and sets compile, and so do exceptions, generators, closures, lambdas, and functions passed as values. A warm ppy run is within noise of python or faster on every example, and a first run takes about half as long as in 0.4. ppy explain --summary shows what stays in Python and why. A differential fuzzer now checks every path against CPython, and the bugs it and a corpus of public programs found are fixed.
Lists, dicts, and sets
- Plain
list,dict, andsetlower to native code underppy run, in standalone builds, and in emitted C and C++, with no rewrite toppy.Vecorppy.HashMap. This covers:- displays and comprehensions;
- their methods, slicing, and negative indices;
in,del,sorted,min,max,sum, and set algebra.
- Results and error messages are CPython's, and
except KeyErrorand the like around them run natively. - A function that takes or returns a container can be called from Python natively. Arguments are copied in, and writes come back to the caller's object.
- On a benchmark of plain-container code (longest increasing subsequence, a dict histogram, a set of tuples, a list sieve), CPython takes 1042 ms,
ppy run587 ms, and a standalone binary 554 ms.
Exceptions and generators
raise,try/except/else/finally, andassertcompile to native code, and exceptions cross native calls.- Generator functions and generator expressions compile when a
for,next,sum,min,max,sorted,any,all, or a comprehension consumes them. - New example
50_errors_and_generators: python 2.1 s,ppy run0.43 s, standalone 0.29 s.
ppy run is never slower
- A warm
ppy runstarts in about 15 to 20 ms overpython, down from about 40 ms.- It checks only the program's own sources and runs before the command line is parsed.
- A program with no native entry point runs from a precompiled plan.
- The runtime decides to call native code from Python only when the work grows with what crosses the boundary.
- String work is faster: the strings example runs in about 1 s warm, against 2.4 s for
python. scripts/run_overhead.pymeasures every example, and the performance page shows the result.
Functions as values
- Nested functions,
lambda, andnonlocalcompile. They use CPython's late binding, so a closure sees the variable as it is when the closure runs. Callablevalues work as parameters, results, container elements, and fields.sorted/min/max/sorttake anykey=.mapandfiltercompile where they are consumed.
More of Python native
- Exception classes with fields, and
raise ... from. - Generators stepped with
nextandfor. - Sets of ints walked in CPython's order.
floatandbooldict keys.- Dataclass
==andrepr,str()/repr()of objects. str.formatand%formatting.
First runs and coverage
- A first
ppy runtakes a median of 0.38 s, down from 0.67 s. The C runtimes are compiled once per machine with optimization, which also makes warm runs of collection- and string-heavy programs faster. ppy explain --summary [PATH ...]reports how many functions and statements go native and ranks what keeps the rest in Python, with a hint and a guide link for each reason.- On TheAlgorithms/Python, 14% of functions compile today. The top blockers are calls with unknown effects, I/O, and writes to parameters.
- Running 400 of its scripts under CPython and
ppy runfound several wrong answers, now fixed. Two examples:row = []hoisted out of a loop, anda, b = [0]*n, [0]*nsharing one list.
Differential fuzzing and fixes
scripts/fuzz.pygenerates programs over the native subset and runs each on CPython, the Python backend,ppy run, a standalone binary, and C and C++ under AddressSanitizer. It reports any difference, andtests/test_fuzz.pyreplays every case it has found.- It found, and this release fixes:
- NaN comparisons and truth.
a if c else bevaluating both sides.and/orover numbers returning a bool.- A
rangeloop that rebinds its variable looping forever. - Several undefined behaviors in emitted C.
- A stale list length after
del,pop, orappend. min/maxreading memory before writing it.- An interned empty literal shadowing
"-inf". - Alias analysis taking gigabytes on nested loops.
- A warm
ppy runreusing an old build after a same-size edit made within one timestamp tick.
Known limitation: a native function with a float parameter given an int returns a float where CPython keeps the int (f(0) gives 0.0). A fix is planned for 0.6.0.
Install or upgrade with pip install --upgrade ppy-lang==0.5.0.
The full repository gate passed: 2,151 tests, 45/45 examples on all three paths, pylint on every example, and the documentation examples. The emitted C and C++ of the new tests runs clean under AddressSanitizer with leak detection.
0.4.2
Changes
- Fixed:
int(x)of a float in native code returned -263 on x86 for NaN, an infinity, or a value past 263, and was undefined behavior in emitted C. It now gives CPython's answer:
x |
ppy run |
standalone binary, emitted C/C++ |
|---|---|---|
| NaN | ValueError: cannot convert float NaN to integer |
the same, exit status 1 |
| ±infinity | OverflowError: cannot convert float infinity to integer |
the same, exit status 1 |
| past 2**63 | the exact integer | OverflowError: the result does not fit in a 64-bit integer, exit status 1 |
The checks are range checks, so --unsafe keeps them. Nothing saturates to the largest or smallest word, since Python's integers have no largest. A value that fits still converts natively, truncated toward zero.
Install or upgrade with pip install --upgrade ppy-lang==0.4.2.
The full repository gate passed: 1,917 tests, 44/44 examples on all three paths, and the documentation examples.
0.4.1
Changes
This release adds the two pieces 0.4.0 left out: objects that order, hash, and compare themselves inside collections, and generic classes in inheritance.
Objects as elements and keys
Heap[Task],Vec[Task].sort(),sorted(...),TreeMap[Task, V], andTreeSet[Task]order instances by the class's__lt__.HashMap[Point, V]andHashSet[Point]hash and compare them by__hash__and__eq__, or by identity where the class defines neither.- Native code calls the compiled methods from inside the collection runtime.
- A guard that fails inside one hands the call back to Python, and a standalone binary prints what CPython raises.
__eq__(self, other: object)works as Python's typing spells it.- A method a subclass overrides keeps the collection in Python.
- The checker takes a class as a key where Python could use it as one, and refuses an unhashable one (
E1305). A tree's keys are one key when neither is less than the other.
from ppy import HashMap, Heap
class Cell:
def __init__(self, row: int, col: int) -> None:
self.row: int = row
self.col: int = col
def __eq__(self, other: object) -> bool:
return isinstance(other, Cell) and self.row == other.row and self.col == other.col
def __hash__(self) -> int:
return self.row * 1000 + self.col
class Visit:
def __init__(self, cost: int, cell: Cell) -> None:
self.cost: int = cost
self.cell: Cell = cell
def __lt__(self, other: "Visit") -> bool:
return self.cost < other.costA search over Heap[Visit] and HashMap[Cell, int] compiles to native code with both collections.
Generic bases
class Counted[T](Stack[T]),class IntStack(Stack[int]), andclass Pair[A, B](Box[A])are checked and compiled.- The checker binds each base's parameters from what the class gives it, so
IntStack().pop()is anint, andsuper()is the base with those arguments. - Native code lays out, instantiates, and dispatches by the same bindings.
- The checker binds each base's parameters from what the class gives it, so
Fixes
TreeSet[str].pop_min()/pop_max()natively returned a string the tree had already freed.- A string or object made only to look a key up was freed in the wrong place when the lookup ran inside a loop.
Install or upgrade with pip install --upgrade ppy-lang==0.4.1.
The full repository gate passed, and the emitted C and C++ of the new tests runs clean under AddressSanitizer with leak detection.
0.4.0
Changes
Ordinary Python data now compiles: text, class hierarchies, and the common container operations. They used to send a function back to CPython. Memory is reclaimed the way CPython reclaims it, cycles included, and a standalone binary that fails prints what CPython would have printed.
Strings
strlowers to native code underppy run, in standalone binaries, and in emitted C and C++. What lowers:- literals,
+,*, and comparisons; - indexing and slicing with steps;
in, iteration,ord/chr, andint()/float()parsing;str()/repr()/format();- f-strings with full format specs;
split,join,strip,replace,find,startswith, theis...tests, the case methods, and more.
- literals,
- Strings are collection elements and keys (
HashMap[str, int],TreeSet[str]) and class fields. They cross the Python boundary and are read and printed by standalone binaries.
Classes
- Single inheritance with dynamic dispatch,
super(), andisinstance. ABase-typed value holds any subclass. - Operator methods (
__add__,__eq__,__lt__,__getitem__,__contains__, ...). - In-place writes like
points[i].x = 3, andfield(default_factory=...). - Type-argument inference:
Pair(1, 2.5),s: Stack[int] = Stack(),v: Vec[int] = Vec().
Collections
- Map and set API:
keys(),values(), anditems();setdefault,update, andpop(key, default);- set algebra (
|,&,-,^,issubset, ...).
- Sequence API:
- slicing,
insert,remove,index,count; sort(key=..., reverse=...);- construction from iterables.
- slicing,
enumerate,zip,sorted,min,max, andsumover collections are native.- Functions that take or return collections are called natively from Python. Writes through arguments reach the caller.
Memory and errors
- Reference cycles (doubly linked lists, parent pointers) are collected.
- A call that falls back to Python frees what it allocated.
- Standalone binaries print CPython's final exception line (
KeyError: (3, -3),IndexError: pop from an empty Deque) and exit with status 1.
Also
--no-strictreports downgraded findings asW2010warnings instead of dropping them.- The documentation site has a new design.
- New examples:
48_stringsand49_inheritance.
Install or upgrade with pip install --upgrade ppy-lang==0.4.0. See CHANGELOG.md for the full list.
The full repository gate passed: 1,878 tests, 44/44 examples on all three paths, pylint on every example, and the documentation examples. The emitted C of the string, class, collection, and memory tests runs clean under AddressSanitizer with leak detection.
0.3.7
Changes
- Standalone builds read
float, the fixed widths (ppy.i8throughppy.u64,ppy.f32,ppy.f64), tuples of them, and buffers and lists of numbers natively. A standaloneprintwrites a float the wayreprdoes. ppy.input[Model]()reads a line of JSON into a dataclass, a pydantic model, or aTypedDict;ppy.scan[Model]()reads a JSON value across lines. Mismatches name the path ($.items[2].price).- Collections that compile:
Vec,Deque,Heap,MaxHeap,LinkedList,HashMap,HashSet,TreeMap,TreeSet. They hold numbers, tuples, dataclasses, and other collections (Vec[Vec[int]],HashMap[tuple[int, int], int]), are reference counted, and lower underppy run, in standalone binaries, and in emitted C and C++. - Generic functions take, make, and return collections of their type parameter, with one native instance per type.
- Object classes lower to native code: trees and linked nodes (
left: "Node | None"), methods,is None,len(obj),if obj:. Reference cycles are not freed. - Generic classes (
class Stack[T]) are checked per type argument and instantiated per type argument natively. - The documentation site is redesigned and every page rewritten.
from ppy import Deque, Vec
def farthest(adjacent: Vec[Vec[int]]) -> int:
seen = Vec[int](len(adjacent))
queue = Deque[int]()
queue.push_back(0)
seen[0] = 1
best: int = 0
while queue:
node: int = queue.pop_front()
for other in adjacent[node]:
if seen[other] == 0:
seen[other] = seen[node] + 1
best = max(best, seen[other])
queue.push_back(other)
return best - 1Install or upgrade with pip install --upgrade ppy-lang==0.3.7.
The full repository gate passed: 1,811 tests, 42/42 examples on all three paths, and the documentation examples. The emitted C of the collection and object tests runs clean under AddressSanitizer with leak detection.
0.3.6
Changes
- Simplify integer
ppy.inputandppy.scanin unsafe standalone C/C++ output to plainscanf(...);calls, without conversion-failure branches. - Assume successful reads with values in the selected 32-bit or 64-bit range. Malformed input, premature EOF, and out-of-range values are outside this mode's input contract. Safe emission and builds retain input validation.
- Preserve integer types and formats, C++ namespaces, and explicit
print(..., flush=True). Omitted or falseflushemits nofflush. - Add regression coverage for C/C++, both integer widths and input APIs, safe validation, and flushing; update the CLI documentation.
ppy emit c --standalone --unsafe --int-width 32 --format app.ppy -o app.c
ppy emit cpp --standalone --unsafe --int-width 32 --format app.ppy -o app.cppInstall or upgrade with pip install --upgrade ppy-lang==0.3.6.
0.3.5
Changes
- Add
--int-width 32for unsafe standalone C/C++ output, using real 32-bit arithmetic, plainint, and%dinput/output. The default remains 64-bit. - Reject constants outside the selected range and fixed runtime ABIs that cannot be narrowed safely.
- Place callees before callers and local declarations near their first write; combine adjacent C++ namespaces.
- Remove redundant casts and unnecessary floor-division corrections using type and branch bounds, while retaining negative division semantics.
- Avoid intermediate overflow when correcting a remainder with a large positive divisor.
- Document the integer model, usage, and limitations.
ppy emit c --standalone --unsafe --int-width 32 --format app.ppy -o app.c
ppy emit cpp --standalone --unsafe --int-width 32 --format app.ppy -o app.cppInstall or upgrade with pip install --upgrade ppy-lang==0.3.5.