Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FD to pathname #237

Open
brendangregg opened this issue Sep 20, 2015 · 15 comments
Open

FD to pathname #237

brendangregg opened this issue Sep 20, 2015 · 15 comments

Comments

@brendangregg
Copy link
Member

@brendangregg brendangregg commented Sep 20, 2015

Would like a macro or function for mapping a file descriptor to a pathname. I was trying something like (this may be wrong):

#include <uapi/linux/ptrace.h>
#include <linux/sched.h>
#include <linux/fdtable.h>

int kprobe__vfs_fstat(struct pt_regs *ctx, unsigned int fd)
{
        struct file *file = (struct file *)&current->files->fdt[fd];
        bpf_trace_printk("fstat file %d %s\\n", fd, file->f_path.dentry->d_iname);
        return 0;
}

and got:

error: couldn't allocate output register for constraint 'r' at line 2148418561
@4ast
Copy link
Member

@4ast 4ast commented Sep 21, 2015

accessing 'current' like this is the problem. One really ugly hack would be to cat /proc/kallsyms |grep current_task and bpf_probe_read that binary address to get 'current', but it's too ugly... may be easier to remember fd->name association in bpf program attached to sys_open, then fstat can read it from the map ? I guess we need a helper for current anyway.

@brendangregg
Copy link
Member Author

@brendangregg brendangregg commented Oct 6, 2015

David Smith has put some work into this problem for SystemTap: https://sourceware.org/bugzilla/show_bug.cgi?id=17920

@4ast
Copy link
Member

@4ast 4ast commented Oct 7, 2015

it's pretty much doing
current->files->fdt[fd]->f_path.dentry->d_iname
with few useless spin_locks and refcnts.
Few helper functions for pieces of this sequence could have been used to avoid exposing current() as a single helper, but current may be necessary for other cases, so yes, bpf_get_current() is pretty high on my todo list. Just need to get big ticket items resolved first.

@4ast
Copy link
Member

@4ast 4ast commented Oct 7, 2016

though bpf_get_current_task() is now available, fd to pathname may be too complicated to do via probe_reads? Do we still need a separate helper for it ?

@brendangregg
Copy link
Member Author

@brendangregg brendangregg commented Oct 7, 2016

Yes, I tried using bpf_get_current_task() but it gets pretty horrible. I got as far as this and it still wasn't working (no file output; not populated on entry to stat()?):

#!/usr/bin/env python

from bcc import BPF

# define BPF program
prog = """
#include <uapi/linux/ptrace.h>
#include <linux/sched.h>
#include <linux/fdtable.h>

int kprobe__vfs_fstat(struct pt_regs *ctx, unsigned int fd)
{
    struct files_struct *files = NULL;
    struct fdtable *fdt = NULL;
    struct file *f = NULL;
    struct dentry *de = NULL;
    struct qstr dn = {};
    struct task_struct *curr = (struct task_struct *)bpf_get_current_task();
    bpf_probe_read(&files, sizeof(files), &curr->files);
    bpf_probe_read(&fdt, sizeof(fdt), &files->fdt);
    bpf_probe_read(&f, sizeof(f), &fdt[fd]);
    bpf_probe_read(&de, sizeof(de), &f->f_path.dentry);
    bpf_probe_read(&dn, sizeof(dn), &de->d_name);
    bpf_trace_printk("fstat fd=%d file=%s\\n", fd, dn.name);
    return 0;
}
"""

# load BPF program
b = BPF(text=prog)

# header
print("%-18s %-16s %-6s %s" % ("TIME(s)", "COMM", "PID", "MESSAGE"))

# format output
while 1:
    try:
        (task, pid, cpu, flags, ts, msg) = b.trace_fields()
    except ValueError:
        continue
    print("%-18.9f %-16s %-6d %s" % (ts, task, pid, msg))

If we can make it work, then at least we can see what the current state is...

@4ast
Copy link
Member

@4ast 4ast commented Oct 13, 2016

managed to make it work:

bpf_probe_read(&f, sizeof(f), &fdt[fd]);

should be

struct file **_fd = NULL;
...
bpf_probe_read(&_fd, sizeof(_fd), &fdt->fd);
bpf_probe_read(&f, sizeof(f), &_fd[fd]);
...
bpf_trace_printk("fstat name1=%s\\n", de->d_iname);
bpf_trace_printk("fstat name2=%s\\n", dn.name);

tried few tests...
looks like both short and full name are populated,
so dn.name is probably good enough always.

@brendangregg
Copy link
Member Author

@brendangregg brendangregg commented Oct 13, 2016

Awesome, thanks! So I'm starting to believe that d_iname isn't reliable:

7538432.717158000  lsb_release      15415  fstat fd=3     dn.name=dist-packages
7538432.717160000  lsb_release      15415  fstat fd=3 de->d_iname=dist-packages
7538432.717340000  lsb_release      15415  fstat fd=3     dn.name=apport_python_hook.cpython-35.pyc
7538432.717342000  lsb_release      15415  fstat fd=3 de->d_iname=p"??????t
7538432.717354000  lsb_release      15415  fstat fd=3     dn.name=apport_python_hook.cpython-35.pyc
7538432.717356000  lsb_release      15415  fstat fd=3 de->d_iname=p"??????t

So I'm going to have to fix some of the exsting *slower tools to go use dn.name instead.

Maybe I'll write a tool that uses this code (statsnoop?), and put the bpf_probe_read()s in an fd2path() static function. I suppose we could eventually move it to somewhere like src/cc/export/helpers.h, and provide this functionality in bcc, at least to start with.

@brendangregg
Copy link
Member Author

@brendangregg brendangregg commented Oct 10, 2019

We've discussed this a number of times. Using the d_iname or d_name only shows the filename. We want a helper to show the full absolute path. e.g., for a FD to pathname helper, we want "/usr/local/bin/bash" and not "bash".

This can be done in at least one of two ways:

A) adding a BPF kernel helper for this function. We've suggested/discussed this at plumber's etc.
B) using the new bounded loops in 5.2, writing a BCC helper that uses loops to construct the path.

@ethercflow
Copy link
Contributor

@ethercflow ethercflow commented Oct 17, 2019

I have implemented this by A) and committed to http://patchwork.ozlabs.org/patch/1179287/ PTAL

@yonghong-song
Copy link
Collaborator

@yonghong-song yonghong-song commented Oct 18, 2019

Thanks @ethercflow Let us continue the discussion in the mailing list.

fengguang pushed a commit to 0day-ci/linux that referenced this issue Oct 27, 2019
When people want to identify which file system files are being opened,
read, and written to, they can use this helper with file descriptor as
input to achieve this goal. Other pseudo filesystems are also supported.

This requirement is mainly discussed here:

  iovisor/bcc#237

v2->v3:
- remove unnecessary LOCKDOWN_BPF_READ
- refactor error handling section for enhanced readability
- provide a test case in tools/testing/selftests/bpf

v1->v2:
- fix backward compatibility
- add this helper description
- fix signed-off name

Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
@thedracle
Copy link

@thedracle thedracle commented Oct 29, 2019

I managed to do this by following the dentries structure with the following function:

static int read_dentry_strings(
    struct dentry *dtryp, char buf[DEFAULT_SUB_BUF_LEN][DEFAULT_SUB_BUF_SIZE]) {
    struct dentry dtry;
    struct dentry *lastdtryp = dtryp;
    int nread = 0;
    int i = 0;
    if (buf) {
        bpf_probe_read(&dtry, sizeof(struct dentry), dtryp);
        bpf_probe_read_str(buf[i], DEFAULT_SUB_BUF_SIZE, dtry.d_name.name);
        nread++;
        for (i = 1; i < DEFAULT_SUB_BUF_LEN; i++) {
            if (dtry.d_parent != lastdtryp) {
                lastdtryp = dtry.d_parent;
                bpf_probe_read(&dtry, sizeof(struct dentry), dtry.d_parent);
                bpf_probe_read_str(buf[i], DEFAULT_SUB_BUF_SIZE, dtry.d_name.name);
                nread++;
            } else
                break;
        }
    }
    return nread;
}

This will get the full path and place an entry per dentry into buf, following up to root on the respective mount.

You can use the same thing to get the mount path via dentries->filp.f_path.mnt, but the mount* is hidden inside a wrapping structure on the vfsmount*, so you have to use 'container_of_in' to get the wrapping structure, then you can use read_dentries on rmount.mnt_mountpoint to reconstruct the full path.

I just pass the data up into userspace separated into this array structure, and then reconstruct it there.

fengguang pushed a commit to 0day-ci/linux that referenced this issue Oct 30, 2019
When people want to identify which file system files are being opened,
read, and written to, they can use this helper with file descriptor as
input to achieve this goal. Other pseudo filesystems are also supported.

This requirement is mainly discussed here:

  iovisor/bcc#237

v3->v4:
- fix missing fdput()
- move fd2path from kernel/bpf/trace.c to kernel/trace/bpf_trace.c
- move fd2path's test code to another patch

v2->v3:
- remove unnecessary LOCKDOWN_BPF_READ
- refactor error handling section for enhanced readability
- provide a test case in tools/testing/selftests/bpf

v1->v2:
- fix backward compatibility
- add this helper description
- fix signed-off name

Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
fengguang pushed a commit to 0day-ci/linux that referenced this issue Nov 3, 2019
…pathname

When people want to identify which file system files are being opened,
read, and written to, they can use this helper with file descriptor as
input to achieve this goal. Other pseudo filesystems are also supported.

This requirement is mainly discussed here:

  iovisor/bcc#237

v6->v7:
- fix missing signed-off-by line

v5->v6: addressed Andrii's feedback
- avoid unnecessary goto end by having two explicit returns

v4->v5: addressed Andrii and Daniel's feedback
- rename bpf_fd2path to bpf_get_file_path to be consistent with other
helper's names
- when fdget_raw fails, set ret to -EBADF instead of -EINVAL
- remove fdput from fdget_raw's error path
- use IS_ERR instead of IS_ERR_OR_NULL as d_path ether returns a pointer
into the buffer or an error code if the path was too long
- modify the normal path's return value to return copied string length
including NUL
- update this helper description's Return bits.

v3->v4: addressed Daniel's feedback
- fix missing fdput()
- move fd2path from kernel/bpf/trace.c to kernel/trace/bpf_trace.c
- move fd2path's test code to another patch
- add comment to explain why use fdget_raw instead of fdget

v2->v3: addressed Yonghong's feedback
- remove unnecessary LOCKDOWN_BPF_READ
- refactor error handling section for enhanced readability
- provide a test case in tools/testing/selftests/bpf

v1->v2: addressed Daniel's feedback
- fix backward compatibility
- add this helper description
- fix signed-off name

Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
fengguang pushed a commit to 0day-ci/linux that referenced this issue Nov 3, 2019
…pathname

When people want to identify which file system files are being opened,
read, and written to, they can use this helper with file descriptor as
input to achieve this goal. Other pseudo filesystems are also supported.

This requirement is mainly discussed here:

  iovisor/bcc#237

v6->v7:
- fix missing signed-off-by line

v5->v6: addressed Andrii's feedback
- avoid unnecessary goto end by having two explicit returns

v4->v5: addressed Andrii and Daniel's feedback
- rename bpf_fd2path to bpf_get_file_path to be consistent with other
helper's names
- when fdget_raw fails, set ret to -EBADF instead of -EINVAL
- remove fdput from fdget_raw's error path
- use IS_ERR instead of IS_ERR_OR_NULL as d_path ether returns a pointer
into the buffer or an error code if the path was too long
- modify the normal path's return value to return copied string length
including NUL
- update this helper description's Return bits.

v3->v4: addressed Daniel's feedback
- fix missing fdput()
- move fd2path from kernel/bpf/trace.c to kernel/trace/bpf_trace.c
- move fd2path's test code to another patch
- add comment to explain why use fdget_raw instead of fdget

v2->v3: addressed Yonghong's feedback
- remove unnecessary LOCKDOWN_BPF_READ
- refactor error handling section for enhanced readability
- provide a test case in tools/testing/selftests/bpf

v1->v2: addressed Daniel's feedback
- fix backward compatibility
- add this helper description
- fix signed-off name

Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
fengguang pushed a commit to 0day-ci/linux that referenced this issue Nov 15, 2019
…pathname

When people want to identify which file system files are being opened,
read, and written to, they can use this helper with file descriptor as
input to achieve this goal. Other pseudo filesystems are also supported.

This requirement is mainly discussed here:

  iovisor/bcc#237

v7->v8: addressed Alexei's feedback
- use fget_raw instead of fdget_raw, as fdget_raw is only used inside fs/
- ensure we're in user context which is safe fot the help to run
- filter unmountable pseudo filesystem, because they don't have real path
- supplement the description of this helper function

v6->v7:
- fix missing signed-off-by line

v5->v6: addressed Andrii's feedback
- avoid unnecessary goto end by having two explicit returns

v4->v5: addressed Andrii and Daniel's feedback
- rename bpf_fd2path to bpf_get_file_path to be consistent with other
helper's names
- when fdget_raw fails, set ret to -EBADF instead of -EINVAL
- remove fdput from fdget_raw's error path
- use IS_ERR instead of IS_ERR_OR_NULL as d_path ether returns a pointer
into the buffer or an error code if the path was too long
- modify the normal path's return value to return copied string length
including NUL
- update this helper description's Return bits.

v3->v4: addressed Daniel's feedback
- fix missing fdput()
- move fd2path from kernel/bpf/trace.c to kernel/trace/bpf_trace.c
- move fd2path's test code to another patch
- add comment to explain why use fdget_raw instead of fdget

v2->v3: addressed Yonghong's feedback
- remove unnecessary LOCKDOWN_BPF_READ
- refactor error handling section for enhanced readability
- provide a test case in tools/testing/selftests/bpf

v1->v2: addressed Daniel's feedback
- fix backward compatibility
- add this helper description
- fix signed-off name

Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
fengguang pushed a commit to 0day-ci/linux that referenced this issue Nov 15, 2019
…pathname

When people want to identify which file system files are being opened,
read, and written to, they can use this helper with file descriptor as
input to achieve this goal. Other pseudo filesystems are also supported.

This requirement is mainly discussed here:

  iovisor/bcc#237

v8->v9:
- format helper description

v7->v8: addressed Alexei's feedback
- use fget_raw instead of fdget_raw, as fdget_raw is only used inside fs/
- ensure we're in user context which is safe fot the help to run
- filter unmountable pseudo filesystem, because they don't have real path
- supplement the description of this helper function

v6->v7:
- fix missing signed-off-by line

v5->v6: addressed Andrii's feedback
- avoid unnecessary goto end by having two explicit returns

v4->v5: addressed Andrii and Daniel's feedback
- rename bpf_fd2path to bpf_get_file_path to be consistent with other
helper's names
- when fdget_raw fails, set ret to -EBADF instead of -EINVAL
- remove fdput from fdget_raw's error path
- use IS_ERR instead of IS_ERR_OR_NULL as d_path ether returns a pointer
into the buffer or an error code if the path was too long
- modify the normal path's return value to return copied string length
including NUL
- update this helper description's Return bits.

v3->v4: addressed Daniel's feedback
- fix missing fdput()
- move fd2path from kernel/bpf/trace.c to kernel/trace/bpf_trace.c
- move fd2path's test code to another patch
- add comment to explain why use fdget_raw instead of fdget

v2->v3: addressed Yonghong's feedback
- remove unnecessary LOCKDOWN_BPF_READ
- refactor error handling section for enhanced readability
- provide a test case in tools/testing/selftests/bpf

v1->v2: addressed Daniel's feedback
- fix backward compatibility
- add this helper description
- fix signed-off name

Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
fengguang pushed a commit to 0day-ci/linux that referenced this issue Nov 19, 2019
…pathname

When people want to identify which file system files are being opened,
read, and written to, they can use this helper with file descriptor as
input to achieve this goal. Other pseudo filesystems are also supported.

This requirement is mainly discussed here:

  iovisor/bcc#237

v9->v10: addressed Andrii's feedback
- send this patch together with the patch selftests as one patch series

v8->v9:
- format helper description

v7->v8: addressed Alexei's feedback
- use fget_raw instead of fdget_raw, as fdget_raw is only used inside fs/
- ensure we're in user context which is safe fot the help to run
- filter unmountable pseudo filesystem, because they don't have real path
- supplement the description of this helper function

v6->v7:
- fix missing signed-off-by line

v5->v6: addressed Andrii's feedback
- avoid unnecessary goto end by having two explicit returns

v4->v5: addressed Andrii and Daniel's feedback
- rename bpf_fd2path to bpf_get_file_path to be consistent with other
helper's names
- when fdget_raw fails, set ret to -EBADF instead of -EINVAL
- remove fdput from fdget_raw's error path
- use IS_ERR instead of IS_ERR_OR_NULL as d_path ether returns a pointer
into the buffer or an error code if the path was too long
- modify the normal path's return value to return copied string length
including NUL
- update this helper description's Return bits.

v3->v4: addressed Daniel's feedback
- fix missing fdput()
- move fd2path from kernel/bpf/trace.c to kernel/trace/bpf_trace.c
- move fd2path's test code to another patch
- add comment to explain why use fdget_raw instead of fdget

v2->v3: addressed Yonghong's feedback
- remove unnecessary LOCKDOWN_BPF_READ
- refactor error handling section for enhanced readability
- provide a test case in tools/testing/selftests/bpf

v1->v2: addressed Daniel's feedback
- fix backward compatibility
- add this helper description
- fix signed-off name

Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
fengguang pushed a commit to 0day-ci/linux that referenced this issue Dec 6, 2019
…pathname

When people want to identify which file system files are being opened,
read, and written to, they can use this helper with file descriptor as
input to achieve this goal. Other pseudo filesystems are also supported.

This requirement is mainly discussed here:

  iovisor/bcc#237

v10->v11: addressed Al and Alexei's feedback
- fix missing fput()

v9->v10: addressed Andrii's feedback
- send this patch together with the patch selftests as one patch series

v8->v9:
- format helper description

v7->v8: addressed Alexei's feedback
- use fget_raw instead of fdget_raw, as fdget_raw is only used inside fs/
- ensure we're in user context which is safe fot the help to run
- filter unmountable pseudo filesystem, because they don't have real path
- supplement the description of this helper function

v6->v7:
- fix missing signed-off-by line

v5->v6: addressed Andrii's feedback
- avoid unnecessary goto end by having two explicit returns

v4->v5: addressed Andrii and Daniel's feedback
- rename bpf_fd2path to bpf_get_file_path to be consistent with other
helper's names
- when fdget_raw fails, set ret to -EBADF instead of -EINVAL
- remove fdput from fdget_raw's error path
- use IS_ERR instead of IS_ERR_OR_NULL as d_path ether returns a pointer
into the buffer or an error code if the path was too long
- modify the normal path's return value to return copied string length
including NUL
- update this helper description's Return bits.

v3->v4: addressed Daniel's feedback
- fix missing fdput()
- move fd2path from kernel/bpf/trace.c to kernel/trace/bpf_trace.c
- move fd2path's test code to another patch
- add comment to explain why use fdget_raw instead of fdget

v2->v3: addressed Yonghong's feedback
- remove unnecessary LOCKDOWN_BPF_READ
- refactor error handling section for enhanced readability
- provide a test case in tools/testing/selftests/bpf

v1->v2: addressed Daniel's feedback
- fix backward compatibility
- add this helper description
- fix signed-off name

Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
fengguang pushed a commit to 0day-ci/linux that referenced this issue Dec 20, 2019
…thname

When people want to identify which file system files are being opened,
read, and written to, they can use this helper with file descriptor as
input to achieve this goal. Other pseudo filesystems are also supported.

This requirement is mainly discussed here:

  iovisor/bcc#237

v12->v13: addressed Gregg and Yonghong's feedback
- rename to get_fd_path
- refactor code & comment to be clearer and more compliant

v11->v12: addressed Alexei's feedback
- only allow tracepoints to make sure it won't dead lock

v10->v11: addressed Al and Alexei's feedback
- fix missing fput()

v9->v10: addressed Andrii's feedback
- send this patch together with the patch selftests as one patch series

v8->v9:
- format helper description

v7->v8: addressed Alexei's feedback
- use fget_raw instead of fdget_raw, as fdget_raw is only used inside fs/
- ensure we're in user context which is safe fot the help to run
- filter unmountable pseudo filesystem, because they don't have real path
- supplement the description of this helper function

v6->v7:
- fix missing signed-off-by line

v5->v6: addressed Andrii's feedback
- avoid unnecessary goto end by having two explicit returns

v4->v5: addressed Andrii and Daniel's feedback
- rename bpf_fd2path to bpf_get_file_path to be consistent with other
helper's names
- when fdget_raw fails, set ret to -EBADF instead of -EINVAL
- remove fdput from fdget_raw's error path
- use IS_ERR instead of IS_ERR_OR_NULL as d_path ether returns a pointer
into the buffer or an error code if the path was too long
- modify the normal path's return value to return copied string length
including NUL
- update this helper description's Return bits.

v3->v4: addressed Daniel's feedback
- fix missing fdput()
- move fd2path from kernel/bpf/trace.c to kernel/trace/bpf_trace.c
- move fd2path's test code to another patch
- add comment to explain why use fdget_raw instead of fdget

v2->v3: addressed Yonghong's feedback
- remove unnecessary LOCKDOWN_BPF_READ
- refactor error handling section for enhanced readability
- provide a test case in tools/testing/selftests/bpf

v1->v2: addressed Daniel's feedback
- fix backward compatibility
- add this helper description
- fix signed-off name

Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
@yzgyyang
Copy link
Contributor

@yzgyyang yzgyyang commented Mar 8, 2021

I have implemented this by A) and committed to http://patchwork.ozlabs.org/patch/1179287/ PTAL

@ethercflow @yonghong-song From https://lore.kernel.org/netdev/c27d3cc2-f846-8aa9-10fd-c2940e7605d1@iogearbox.net/#t, I'm curious if this is still stuck on review/waiting for anyone? Would love to see this merged. :)

@yonghong-song
Copy link
Collaborator

@yonghong-song yonghong-song commented Mar 9, 2021

@yzgyyang
Copy link
Contributor

@yzgyyang yzgyyang commented Apr 19, 2021

It takes a "path" (powered by btf) instead of fd, so it won't be available
to kprobe, etc.
but it is available to kfunc. The use case is in many cases, you actually
have
"struct file *" from which you can get "path" and feed it into the helper.

@yonghong-song Thanks for the reply! This is not particularly useful for our use case, since we want to use kprobes. I have, though, took @thedracle 's idea above and developed the full path functionality in a reverse-dentry-lookup way for now - will open a PR shortly for this.

@yonghong-song
Copy link
Collaborator

@yonghong-song yonghong-song commented Apr 19, 2021

Sounds good. Thanks!

@yzgyyang yzgyyang mentioned this issue Apr 21, 2021
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

6 participants