Update hard-switching routines from greenlet

[ghost]

Originally reported by: RMTEW FULL NAME (Bitbucket: rmtew, GitHub: rmtew)

---

(originally reported in Trac by @rmtew on 2012-07-23 03:05:10)

Greenlet has updated versions of Stackless' hard switching routines. It would be worthwhile to update these for Stackless where suitable.

One way in which it may not be suitable, is that Stackless is under the PSF license. And greenlet is under some confused mix of the MIT and PSF license, due to the confused authorship of Stackless-sourced files within it.

Any that are obviously derived from Stackless versions, should be PSF licensed.

Kristjan, should we consider this for the 3.3 release?

---

• Bitbucket: https://bitbucket.org/stackless-dev/stackless/issue/10

[ghost]

Original comment by RMTEW FULL NAME (Bitbucket: rmtew, GitHub: rmtew):

---

@krisvale on 2013-01-30 11:40:24 said:

Interesting proposition.
The greenlet switching routines are upgrdes of (some) of the stackless switching routines, avoiding the use of static global memory and thus thread-safe.
These routines have found their ways into another library, tealets, which is a pure-c stack slicing library.
I have been working on this library on and off for the past year, and am interested in, when it is ready, replacing stackless switching with this. It introduces greenlet-style stack management, with just-in-time stack copying, which for example removes the "climb-stack-and-eval" nastiness currently in stackless.

IMO, it is better to wait to do that, rather than to use the greenlet switching routines directly.

[ghost]

Original comment by RMTEW FULL NAME (Bitbucket: rmtew, GitHub: rmtew):

---

@ctismer on 2013-01-30 11:53:08 said:

In fact, this is interesting.
I even went further in considerations:
Is it possible to split soft- and hardswitching so much apart, that it is
possible to factor the greenlet-like code out and use it as a plugin?
This way we would have a pure-cpython switching/pickling implementation in
Stackless, and the optional greenlet switching as an extension module.

On the tealets, which are meanwhile superseded by "continulet":
Would this really be a c-only implementation for Stackless? How that, based
on portable c code?

[ghost]

Original comment by RMTEW FULL NAME (Bitbucket: rmtew, GitHub: rmtew):

---

Where do we stand on this? Are we integrating tealet in 2.8? If I recall correctly there was a discussion about integrating this in 2.7 and it was veto'd by @akruis.

If tealet is not going in, reassign to me and I'll integrate the switching routines directly.

[ghost]

Original comment by Anselm Kruis (Bitbucket: akruis, GitHub: akruis):

---

Well I wouldn't call it a veto. The conclusion was, that we need a proven benefit for the users of 2.7-slp. A reduced maintenance effort alone is probably no argument for 2.7-slp, because 2.7-slp is already very stable and we don't have any open issues with hard switching.

Let's put it into 2.8 or 3.4 first. If it works well enough we can still convert 2.7 to tealets.

[akruis]

@kristjanvalur To answer your comment in PR #163

You wrote a good summary. For 2.7. it is probably not worth the effort. One more year and than 2.7 is history.

We suspended our discussion about this topic nearly 5 years ago, but issue #10 is still open. since the the situation changed a bit: the test suite is better and more complete, memory leaks are gone, git simplifies the development workflow. Therefore I'm willing to review and approve pull requests. But don't expect me to do all the work myself. I maintain Stackless in my leisure time and the preparation of new minor versions (like 3.7) takes a lot of time.

[kristjanvalur]

In the years since I left CCP Games, I have had very little to do with core python development, including Stackless. Pressures of work mean that I don't have much time to contribute. Once in a while I do revisit some toy projects, including the tealet project. But that hasn't been updated in a while. Also, I don't know what is the current states of low level switching routines. There is some mismatch between all three, stackless, greenlets, and tealets. The latter (Anselm Kruis' latest foray into this) removes the needs for static variables.
But there are other differences, for example the amount of state that is saved. I wrote the original x64 switching code for stackless, and was careful to save all processor state that the ABI didn't specify as volatile. This included some FPU state and SSE2, if I remember correctly. I think we should try to at least reconcile some of this code into some best of breed stuff.

[akruis]

Outdated by pull request #230 (stackman).