Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regex is slowing startup #526

Closed
brendangregg opened this issue Apr 9, 2019 · 1 comment
Closed

regex is slowing startup #526

brendangregg opened this issue Apr 9, 2019 · 1 comment
Labels
enhancement New feature or request, changes on existing features
Milestone

Comments

@brendangregg
Copy link
Contributor

I have a more complex example that takes a few seconds for bpftrace to start, which turns out to be in regex for tracepoint arguments. Here is a simple (and faster) example:

# time bpftrace -e 'BEGIN { exit(); } t:syscalls:sys_enter_execve { @[0, 0, 0]++ }'
Attaching 2 probes...

real	0m0.191s
user	0m0.056s
sys	0m0.038s

# time bpftrace -e 'BEGIN { exit(); } t:syscalls:sys_enter_execve { @[args->filename, args->argv, args->envp]++ }'
Attaching 2 probes...

real	0m0.401s
user	0m0.231s
sys	0m0.052s

Here is a CPU flamegraph where I've searched for regex:

cpuflamegraph-bpftrace-startup

87.4% of CPU time is in regex. You can see the codepaths: it's in sementic_analyser.cpp here:

void SemanticAnalyser::visit(FieldAccess &acc)
[...]
  if (type.is_tparg) {
    for (AttachPoint *attach_point : *probe_->attach_points) {
      assert(probetype(attach_point->provider) == ProbeType::tracepoint);

      std::set<std::string> matches = bpftrace_.find_wildcard_matches(
          attach_point->target, attach_point->func,
          "/sys/kernel/debug/tracing/available_events");
      for (auto &match : matches) {
        std::string tracepoint_struct =
            TracepointFormatParser::get_struct_name(attach_point->target,
                                                    match);
        structs[tracepoint_struct] = bpftrace_.structs_[tracepoint_struct].fields;
      }
    }

In this case, I'm not even using a wildcard -- I'm matching syscalls:sys_enter_execve.

Could someone please fix this, eg:

  • Do a direct match if no wildcard operators are used.
  • Cache results (this is called several times due to multiple passes through semantic analyser).
  • Something else :)
@mmarchini mmarchini added the enhancement New feature or request, changes on existing features label Apr 15, 2019
@mmarchini mmarchini added this to the 0.10 milestone Apr 15, 2019
mmarchini added a commit that referenced this issue Apr 18, 2019
mmarchini added a commit to mmarchini/bpftrace that referenced this issue Apr 18, 2019
Improved speed by not performing wildcard match on probes without a
wildcard. Speedup of complex scripts (especially those using tracepoints
with arguments) will be much faster now. As an example from our toos:

Before:

  # time bpftrace -d statsnoop.bt
  ...
  bpftrace -d statsnoop.bt  1.67s user 0.06s system 99% cpu 1.748 total

After:

  # time bpftrace -d statsnoop.bt
  ...
  bpftrace -d statsnoop.bt  0.06s user 0.02s system 80% cpu 0.098 total

Ref: bpftrace#526
mmarchini added a commit that referenced this issue Apr 18, 2019
mmarchini added a commit that referenced this issue Apr 23, 2019
mmarchini added a commit that referenced this issue Apr 23, 2019
mmarchini added a commit that referenced this issue Apr 23, 2019
mmarchini added a commit that referenced this issue Apr 23, 2019
mmarchini added a commit that referenced this issue Apr 25, 2019
@mmarchini
Copy link
Contributor

Fixed by #559 and #557.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request, changes on existing features
Projects
None yet
Development

No branches or pull requests

2 participants