Add code to walk the list of thread states #13

Open
eklitzke opened this Issue Aug 25, 2016 · 4 comments

Projects

None yet

3 participants

@eklitzke
Member

This is how you do it:

  • each interpreter has a field called tstate_head which is the head of a linked list of thread states
  • each thread state has a field called next which is a pointer to the next thread state
  • the last thread state has a null pointer for next

Likewise:

  • each thread state has a back reference to the interpreter state in a field called interp
  • there's a single linked list of interpreters whose head is the static symbol interp_head

In the single-threaded case there is one interpreter and one thread state.

In the generic case the way you enumerate the thread states is:

  • locate _PyThreadState_Current
  • follow interp up to the interpreter
  • follow tstate_head to the first thread state (which will always be the same as _PyThreadState_Current for a single-threaded program, but could be different for a multi-threaded program)
  • follow the next field until NULL is encountered

It should be fine to ignore the multiple interpreter thing, no one really uses that feature nowadays.

@ogrisel
ogrisel commented Oct 18, 2016

Related: it would be great to have a way to generate one flamegraph svg file active thread in a multithreaded python program.

@eklitzke
Member

I gave a few more details about this in #41 .

This is probably the next major feature I'll add to Pyflame (after fixing #34, which has a PR by me out). However, I'm going to spend most of November traveling. If someone wants to surprise me with a PR to fix this in the interim that would be great, otherwise I'll probably get some time in December.

@eklitzke eklitzke added this to the Pyflame2 milestone Oct 31, 2016
@batterseapower

Did some initial work on this at batterseapower@36ba9e1

Right now it doesn't seem to be able to find interp_head, which is necessary if we want to do anything interesting (PyThreadState_Current is NULL if the GIL is not held). This should be fixable because e.g. nm -a knows about interp_head.

@batterseapower

Fixed that, but PyThreadState.frame seems to always be null in the cases I looked at.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment