Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change: Use PyPy's StringBuilder to speed up the use of StringIO. #98

Closed
wants to merge 1 commit into from

Conversation

@Xaroth
Copy link
Contributor

Xaroth commented Apr 21, 2020

This is an alternative to #76 .

Inspiration was found from https://bitbucket.org/pypy/pypy/pull-requests/132/speed-up-pickledumps-fixes-issue-979/diff , where a similar fix was applied to speed up pickle.dumps for PyPy.

This only affects PyPy, normal CPython versions will still use io.StringIO, therefore not incurring any performance loss.

@andythenorth
Copy link
Contributor

andythenorth commented Apr 22, 2020

Some test results using this PR with PyPy 7.3.0 to compile Iron Horse nml

  • about 40% faster for an nml->newgrf compile compared to nml master rev 9fdd4bb
  • the Iron Horse makefile has an nfo step, for which the 'Writing nfo' output step is specifically faster by about 85%

Python 3.8 timings are unchanged, as expected.

@FLHerne
Copy link
Contributor

FLHerne commented Apr 22, 2020

Much neater than my attempt!

That said, Ijust discovered that PyPy upstream have fixed this in 7.3.1:

Improve performance of io.StringIO(). It should now be faster than CPython in common use cases.

(the changelog categories are confusing, but I've verified that the relevant commits appear in 7.3.1 and not 7.3.0).

That's done by internally using a StringBuffer somewhat like this approach...

Testing on Linux with pypy 7.3.1, there is no measurable difference in performance with this patch; the present code is as fast with 7.3.1 as the patched version in 7.3.0 and earlier.

@andythenorth
Copy link
Contributor

andythenorth commented Apr 22, 2020

I think we can rely on 7.3.1, so I'm going to close this - thanks for all the contributions!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.