Add code to walk the list of thread states #13

eklitzke opened this Issue Aug 25, 2016 · 4 comments


None yet

3 participants


This is how you do it:

  • each interpreter has a field called tstate_head which is the head of a linked list of thread states
  • each thread state has a field called next which is a pointer to the next thread state
  • the last thread state has a null pointer for next


  • each thread state has a back reference to the interpreter state in a field called interp
  • there's a single linked list of interpreters whose head is the static symbol interp_head

In the single-threaded case there is one interpreter and one thread state.

In the generic case the way you enumerate the thread states is:

  • locate _PyThreadState_Current
  • follow interp up to the interpreter
  • follow tstate_head to the first thread state (which will always be the same as _PyThreadState_Current for a single-threaded program, but could be different for a multi-threaded program)
  • follow the next field until NULL is encountered

It should be fine to ignore the multiple interpreter thing, no one really uses that feature nowadays.

ogrisel commented Oct 18, 2016

Related: it would be great to have a way to generate one flamegraph svg file active thread in a multithreaded python program.


I gave a few more details about this in #41 .

This is probably the next major feature I'll add to Pyflame (after fixing #34, which has a PR by me out). However, I'm going to spend most of November traveling. If someone wants to surprise me with a PR to fix this in the interim that would be great, otherwise I'll probably get some time in December.

@eklitzke eklitzke added this to the Pyflame2 milestone Oct 31, 2016

Did some initial work on this at batterseapower@36ba9e1

Right now it doesn't seem to be able to find interp_head, which is necessary if we want to do anything interesting (PyThreadState_Current is NULL if the GIL is not held). This should be fixable because e.g. nm -a knows about interp_head.


Fixed that, but PyThreadState.frame seems to always be null in the cases I looked at.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment