Use PyPy for pure python code

[santagada]

On the current Readme it states:
"But there are also a number of libraries in CPython that are C extension modules that need to be rewritten. This includes re, itertools and others."

I think that at least itertools is implemented in app level python on PyPy, why not just use them instead of rewriting modules?

[ns-cweber]

It's MIT licensed, so that shouldn't be a concern. Seems like it would save a lot of work.

[trotterdylan]

Definitely support this but from my brief look at PyPy stdlibs it seemed like there was a fair bit of weirdness going on with some of the PyPy code, e.g. https://bitbucket.org/pypy/pypy/src/61294de6f130/pypy/module/itertools/?at=default

I have no problem begging, borrowing and stealing what we have to to get the stdlib functional.

[ns-cweber]

Yeah, that is strange. Definitely optimized for PyPy. That module in particular depends on PyPy's jit package.

[jacobh]

Another option is https://github.com/pybee/ouroboros

[trotterdylan]

Interesting I did not know about that project! I'm a little unclear about the origin of the libraries. Seems like some of it's from PyPy.

I'm concerned about the licensing. It looks like a non-standard license in addition to PSF: https://github.com/pybee/ouroboros/blob/master/LICENSE

Does anybody know if that project includes novel code or is it all from other places?

[tav]

@trotterdylan It looks like most of it is from PyPy/Python, so you can just grab the standard licensed files.

[ulope]

@trotterdylan You should ask @freakboy3742. He's the mastermind behind BeeWare (pybee)

[freakboy3742]

You can certainly use PyPy code as a starting point - but "Just Use PyPy" isn't a complete solution, because there will always be system native components. For example, you can't implement a socket library without being able to access sockets.

I've got similar needs to Grumpy on my projects pybee/batavia and pybee/voc, which is why I started pybee/ouroboros. Most of the code that is already there has originated from either PyPy or CPython.

I'd be delighted to collaborate on Ouroboros with anyone from the Grumpy team (and any other Python implementations, including PyPy) who shares the need of having a Python standard library that isn't derived from CPython.

[trotterdylan]

Thanks for sharing, Russel. A collaboration sounds really interesting. A common pure-Python subset of the standard library that could be shared by all the different interpreters would make implementing Python runtimes a lot less of a chore. It reminds me a bit of the old STLPort project that was so useful when every C++ compiler had its own STL quirks and limitations.

Things get really interesting when you start to think about how to pare down the native integrations to get platform-specific features working. Like, if a runtime implements and exposes a POSIX interface to Python then most stuff should "just work".

Another area for investigation is how to enable optimization. ISTM that the design of many of the CPython libraries was dictated by what needed to be written in C. The re library is the obvious example here. If there was a more generic and flexible way to specify what ends up being implemented natively, then maybe it'd be easier for many runtimes to share the same standard library.

Anyway, happy to chat about this more. It'd be really cool if Grumpy could just use Ouroboros more or less as-is. Ideally the Grumpy repo would not include the stdlib or it would include Ouroboros wholesale rather than piecemeal from CPython and PyPy.