Get a list of threads from Linux debuggee #47

fabiim · 2020-07-26T09:30:12Z

Fixes #36

Add a Thread trait
Retrieves a list of threads in linux

Notes:

I dropped a couple of the suggestions in the ticket (#36) because:

I decided to use procfs because libproc-rs does not support getting the list of threads from linux right now. I imagine this crate being useful in the future anyhow.
Having a default associated type in the Threat trait is not stable.
Returning a slice of Threat references does not make much sense to me because the threats change over time and I feel it's more useful to get a fresh current view every time.

src/target/unix.rs

.gitignore

Cargo.toml

bjorn3 · 2020-07-26T15:16:40Z

src/target/thread.rs

+
+    fn name(&self) -> Option<String>;
+    fn thread_id(&self) -> Self::ThreadId;
+}


This doesn't feel like the right abstraction. Every thread has it's own register set that can be read and written. Also individual threads can be waiting on the debugger and be running. On Linux at least ptrace attaches to a single thread, not a single process:

https://man7.org/linux/man-pages/man2/ptrace.2.html

A tracee first needs to be attached to the tracer. Attachment and subsequent commands are per thread: in a multithreaded process, every thread can be individually attached to a (potentially different) tracer, or left not attached and thus not debugged. Therefore, "tracee" always means "(one) thread", never "a (possibly multithreaded) process".

This also means that you need to enable the PTRACE_O_TRACECLONE option to attach to spawned threads.

Agreed, it looks like Thread should be almost like a subset of Target.
What are your thoughts on this @fabiim, @bjorn3?

We could say that Target has one or more Thread's with one being the main thread. read_regs will then ve moved to Thread, but read and write remain part of Target. The unpause method should remain part of Target I think, at least for as long as we don't have an equivalent to gdb's non-stop mode in which individual threads are paused and continued.

Let me see if I get it:

Move the read/write registers methods to the thread trait

Add a way to retrieve a main thread from a target

If that is correct, I have a couple of questions:

What should be the abstraction for the registers structure be in the trait?

Should threads() return all threads (including the main thread?)

What should be the abstraction for the registers structure be in the trait?

Just type Registers: Debug; or something like that for now I think. It is possible to later add additional bounds as necessary. I think threads should return a concrete thread type though for that to work.

Should threads() return all threads (including the main thread?)

I don't have an opinion on this.

Should threads() return all threads (including the main thread?)

Yes, I believe so. At least, most (all?) major debuggers impose this behaviour - e.g., if you launch a single-threaded process in gdb and run the info threads command, the output will include the main thread.

src/target/linux.rs

bjorn3

nits

src/target/linux.rs

- introduces a Thread struct - retrieves a list of threads in linux Notes: - I decided to use procfs because libproc-rs does not support getting the list of threads from linux right now [1]. I imagine this crate being useful in the future anyhow. [1] https://github.com/andrewdavidmackenzie/libproc-rs/blob/123d1762727c9c5d4e81888d8ea1a7f9701ddb45/src/libproc/proc_pid.rs#L260

Fails to compile on OSX. It's not needed there anywhere, so this is better.

Co-authored-by: bjorn3 <bjorn3@users.noreply.github.com>

That is the only place it's being used.

src/target/linux.rs

- Was printing the wrong message. - Was not waiting for the thread to start before retrieving the list - of threads. Seen it failed on CI.

Adding to supress warning until it's used on a integration test or by something else.

It's the original type in the kernel.

And remove silly dead code supression :D

…hreads

nbaksalyar

Thank you for the PR! I left a few suggestions & comments.

src/target/linux.rs

nbaksalyar · 2020-07-27T21:28:53Z

src/target/thread.rs

+
+    fn name(&self) -> Option<String>;
+    fn thread_id(&self) -> Self::ThreadId;
+}


Agreed, it looks like Thread should be almost like a subset of Target.
What are your thoughts on this @fabiim, @bjorn3?

nbaksalyar · 2020-07-27T21:41:51Z

currently there's no way to get a number of test threads from inside the tests (reading the env var is inconsistent)

FYI, I've raised a new issue for Rust: rust-lang/rust#74845

Makes API more flexible. Co-authored-by: Nikita Baksalyar <nikita.baksalyar@gmail.com>

src/target/linux.rs

The aplly suggestion did remove the previous code?

Note that is inconsistent: rust-lang/rust#74845

nbaksalyar

LGTM, thank you! 👍
I think the questions re trait Thread can be addressed in separate issues/PRs.

bjorn3 · 2020-07-28T21:54:48Z

target::linux::tests::reads_threads stdout ----

Error: NotFound(Some("/proc/5167/task/5170/stat"))

fabiim · 2020-07-28T22:00:21Z

target::linux::tests::reads_threads stdout ----

Error: NotFound(Some("/proc/5167/task/5170/stat"))

Yep, there is a bug, it should tolerate tasks that have gone away . I imagine this hit that.
Fixing it...

- Banged the thread change - Had duplicate procfs entries

src/target/linux.rs

Some tasks might not be there between getting them from the iterator and actually trying to read their page.

fabiim · 2020-07-29T06:56:22Z

target::linux::tests::reads_threads stdout ----

Error: NotFound(Some("/proc/5167/task/5170/stat"))
Yep, there is a bug, it should tolerate tasks that have gone away . I imagine this hit that.
Fixing it...

Fixed. Now we just skip threads not found and/or not ready yet.

It does not make much sense. Threads come and go, we don't have any consistency guarantees when calling the threads method. If the thread info is not ready it can be caught in a subsequent call to threads

bjorn3 · 2020-07-29T09:18:46Z

src/target/linux.rs

+                    // ok to skip. Thread is gone or it's page is not complete yet.
+                }
+                Err(err) => return Err(Box::new(err)),
+            }


A thread could change it's own name I think. Maybe LinuxThread should only store pid and tid and name() would read the current name or return None when the thread doesn't exist anymore?

Ah... interesting. Ok will check this later today then.

Regarding keeping the pid and tid: as far as I can tell they are the same thing.
So we only need to keep one. Is this correct?

procfs::process::Task contains the pid of the whole process and the tid of the specific thread. You could also directly store that struct.

You are right, a thread can change the name. From man procfs:

A thread may modify its comm value, or that of any of other thread in the same thread group

Going with your idea of having the name method return the most recent: do you think it will be fine to swallow all other procfs::ProcError into a None? If so, the user wouldn't know something went wrong. Should we use a result on the name?

Also I'm not sure I understand the benefits of the change. If you are getting a list of threads you either have them all paused or as soon as you get your information it might already be outdated. So the user will always have this issue of seeing old data, even if we push it elsewhere. He will always need to poll the thread list to get up-to-date information.

Should we use a result on the name?

Yes

If you are getting a list of threads you either have them all paused or as soon as you get your information it might already be outdated.

I think once we support actually debugging threads, we would store a list of all active threads. When using PTRACE_O_TRACECLONE to automatically attach to new threads we have to manually continue the new thread anyway, so we could just as well store it in a list. Only when attaching to a new process do we need to list all active threads. Making the LinuxThread persistent also has the benefit that you could continue and pause individual threads rather than the whole process.

Done.

I think once we support actually debugging threads, we would store a list of all active threads. When using PTRACE_O_TRACECLONE to automatically attach to new threads we have to manually continue the new thread anyway, so we could just as well store it in a list. Only when attaching to a new process do we need to list all active threads. Making the LinuxThread persistent also has the benefit that you could continue and pause individual threads rather than the whole process.

Alright, I completely missed that the first time you mentioned. I think I get it now - thank you!. We are going to be in lockstep with threads getting notified through ptrace that a new thread was created and adding that to the list of the process.

fabiim · 2020-07-31T08:51:28Z

@nbaksalyar @bjorn3 Is this ok?

My understanding from the discussion is that 2 essential features are missing:

The Thread abstraction should have (to start with) the read registers method using a Register: Debug Trait
The thread list should be kept inside the process and we need to update it as threads come and go

But I think it might be better to handle them as separate PR's?

Keeping the task to retrieve the name forces us to change the API to be a bit more complex but it allows to capture thread names changes.

nbaksalyar · 2020-07-31T15:14:03Z

But I think it might be better to handle them as separate PR's?

Definitely. This PR underwent quite a number of changes & rebases already - thanks for your patience!

fabiim mentioned this pull request Jul 26, 2020

Get a list of process threads on macOS #37

Closed

fabiim changed the title ~~Get a list of threads from Linux debuggee~~ Get a list of threads from Linux debuggee - WIP (fixing tests) Jul 26, 2020

fabiim changed the title ~~Get a list of threads from Linux debuggee - WIP (fixing tests)~~ Get a list of threads from Linux debuggee Jul 26, 2020

JJendryka reviewed Jul 26, 2020

View changes

src/target/unix.rs Outdated Show resolved Hide resolved

bjorn3 requested changes Jul 26, 2020

View changes

fabiim force-pushed the list-threads branch from 1c4edc1 to dc4d549 Compare July 26, 2020 20:07

fabiim commented Jul 26, 2020

View changes

src/target/linux.rs Outdated Show resolved Hide resolved

bjorn3 reviewed Jul 26, 2020

View changes

src/target/linux.rs Outdated Show resolved Hide resolved

src/target/linux.rs Outdated Show resolved Hide resolved

src/target/linux.rs Outdated Show resolved Hide resolved

fabiim and others added 6 commits July 26, 2020 21:54

Compile procfs only on linux

8614be8

Fails to compile on OSX. It's not needed there anywhere, so this is better.

Move location of Linux dependencies

f0ee9be

Add trailing newline

dc4d549

Improve comments

370ebae

Co-authored-by: bjorn3 <bjorn3@users.noreply.github.com>

Move threads to LinuxTarget only

da9e791

That is the only place it's being used.

bjorn3 reviewed Jul 26, 2020

View changes

src/target/linux.rs Outdated Show resolved Hide resolved

src/target/linux.rs Outdated Show resolved Hide resolved

fabiim added 5 commits July 26, 2020 23:00

Fix threads test

009d9c0

- Was printing the wrong message. - Was not waiting for the thread to start before retrieving the list - of threads. Seen it failed on CI.

Supress warning on threads

9af1455

Adding to supress warning until it's used on a integration test or by something else.

Use i32 as thread id type

3e4f9b3

It's the original type in the kernel.

Make method public

f1ce896

And remove silly dead code supression :D

Merge branch 'list-threads' of github.com:fabiim/headcrab into list-t…

ed6215d

…hreads

nbaksalyar reviewed Jul 27, 2020

View changes

Use Into<String>

0471ff8

Makes API more flexible. Co-authored-by: Nikita Baksalyar <nikita.baksalyar@gmail.com>

bjorn3 reviewed Jul 28, 2020

View changes

src/target/linux.rs Outdated Show resolved Hide resolved

fabiim added 3 commits July 28, 2020 21:09

Fix suggestion commit error

65a29a7

The aplly suggestion did remove the previous code?

Use the environment to get number of threads

771f707

Note that is inconsistent: rust-lang/rust#74845

Merge branch 'master' into list-threads

9012e7a

nbaksalyar approved these changes Jul 28, 2020

View changes

nbaksalyar requested a review from bjorn3 July 28, 2020 21:12

bjorn3 approved these changes Jul 28, 2020

View changes

fabiim added 2 commits July 28, 2020 23:34

Merge branch 'master' into list-threads

51ed53c

Fix bad merge conflict resolution

53fe03e

- Banged the thread change - Had duplicate procfs entries

fabiim commented Jul 28, 2020

View changes

src/target/linux.rs Outdated Show resolved Hide resolved

Ignore tasks not found

1a0b66c

Some tasks might not be there between getting them from the iterator and actually trying to read their page.

Stop retrying when incomplete

1da2a2d

It does not make much sense. Threads come and go, we don't have any consistency guarantees when calling the threads method. If the thread info is not ready it can be caught in a subsequent call to threads

bjorn3 reviewed Jul 29, 2020

View changes

Merge branch 'master' into list-threads

0b76f15

bjorn3 approved these changes Jul 31, 2020

View changes

bjorn3 requested a review from nbaksalyar July 31, 2020 08:55

fabiim added 2 commits July 31, 2020 10:23

Retrieve the most recent name from thread

929613a

Keeping the task to retrieve the name forces us to change the API to be a bit more complex but it allows to capture thread names changes.

Merge branch 'master' into list-threads

acdcc6c

fabiim mentioned this pull request Jul 31, 2020

Keep list of threads inside Linux Target #65

Open

nbaksalyar approved these changes Jul 31, 2020

View changes

nbaksalyar merged commit 9a15ae1 into headcrab-rs:master Jul 31, 2020
1 check passed

Get a list of threads from Linux debuggee #47

Get a list of threads from Linux debuggee #47

fabiim commented Jul 26, 2020

bjorn3 Jul 26, 2020

nbaksalyar Jul 27, 2020

bjorn3 Jul 27, 2020

fabiim Jul 28, 2020 •

edited

bjorn3 Jul 28, 2020

nbaksalyar Jul 28, 2020

bjorn3 left a comment

nbaksalyar left a comment

nbaksalyar Jul 27, 2020

nbaksalyar commented Jul 27, 2020

nbaksalyar left a comment

bjorn3 commented Jul 28, 2020

fabiim commented Jul 28, 2020

fabiim commented Jul 29, 2020

bjorn3 Jul 29, 2020

fabiim Jul 29, 2020

bjorn3 Jul 29, 2020

fabiim Jul 29, 2020

bjorn3 Jul 29, 2020

fabiim Jul 31, 2020

fabiim commented Jul 31, 2020

nbaksalyar commented Jul 31, 2020

Get a list of threads from Linux debuggee #47

Get a list of threads from Linux debuggee #47

Conversation

fabiim commented Jul 26, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabiim Jul 28, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjorn3 left a comment

Choose a reason for hiding this comment

nbaksalyar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nbaksalyar commented Jul 27, 2020

nbaksalyar left a comment

Choose a reason for hiding this comment

bjorn3 commented Jul 28, 2020

fabiim commented Jul 28, 2020

fabiim commented Jul 29, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabiim commented Jul 31, 2020

nbaksalyar commented Jul 31, 2020

fabiim Jul 28, 2020 •

edited