Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
nmlc is generally faster with pypy than python 3.8
However output to nfo is specifically about 5x slower with pypy compared to python 3.8.
This appears to be due to poor performance of StringIO, which is a known issue with pypy: https://bitbucket.org/pypy/pypy/pull-requests/132/speed-up-pickledumps-fixes-issue-979
I created a test substituting StringIO with simple list append / join, which performs much faster
For Iron Horse compile the difference is ~10s with pypy vs ~2s with python 3.8. In the slower version the extra 8s represents ~25% of total compile time for the grf, so it is not insignificant.
My patch is not a viable candidate for merging, but this would be a helpful issue to fix.
FLHerne has a patch which substantially reduces output time with pypy in Iron Horse and FIRS compiles http://www.flherne.uk/files/nicer-quick-output.diff
I ran some tests against http://www.flherne.uk/files/nicer-quick-output.diff using Iron Horse and nml at rev 660722b
Timings are purely for 'Writing output' step, tested over a couple of runs, with primed caches.
Without patch: ~1.5s
Without patch: ~6.0s
The specific nml rev used is chosen because the patch conflicts with the next rev f71103b
The 5s improvement for PyPy with this patch reduces a total compile time from 30s to 25s.
I am +/-0 on whether we should include this; it's a massive boost for PyPy, but a performance regression for Python 3.8.
For context though: