Skip to content

benchmarking: mapping for processors is wrong #2498

@efd6

Description

@efd6

When running benchmarks, the mapping between the processors listed in the output and the actual source appears to be broken.

$ git rev-parse HEAD  
1595cc1388bf5054c47d5828535ff772944804df
$ pwd
…/github.com/elastic/integrations/packages/sentinel_one_cloud_funnel
$ elastic-package benchmark pipeline -d event
<snip>
Run pipeline benchmarks for the package
--- Benchmark results for package: sentinel_one_cloud_funnel - START ---
╭─────────────────────────╮
│ parameters              │
├──────────────────┬──────┤
│ source_doc_count │   16 │
│ doc_count        │ 1000 │
╰──────────────────┴──────╯
╭───────────────────────────╮
│ pipeline_performance      │
├─────────────────┬─────────┤
│ processing_time │   0.69s │
│ eps             │ 1440.92 │
╰─────────────────┴─────────╯
╭────────────────────────────────────────╮
│ procs_by_total_time                    │
├───────────────────────────────┬────────┤
│ dot_expander @ default.yml:52 │ 19.31% │
│ json @ default.yml:34         │  4.61% │
│ remove @ default.yml:2778     │  3.89% │
│ script @ default.yml:2849     │  1.73% │
│ date @ default.yml:87         │  1.44% │
│ date @ default.yml:533        │  1.44% │
│ date @ default.yml:1723       │  1.30% │
│ gsub @ default.yml:20         │  1.01% │
│ script @ default.yml:2770     │  1.01% │
│ script @ pipeline-dns.yml:21  │  0.72% │
╰───────────────────────────────┴────────╯
╭──────────────────────────────────────────╮
│ procs_by_avg_time_per_doc                │
├───────────────────────────────┬──────────┤
│ dot_expander @ default.yml:52 │    134µs │
│ json @ default.yml:34         │     32µs │
│ remove @ default.yml:2778     │     27µs │
│ script @ pipeline-dns.yml:21  │ 19.841µs │
│ script @ default.yml:2849     │     12µs │
│ date @ default.yml:533        │  10.66µs │
│ date @ default.yml:87         │     10µs │
│ date @ default.yml:1723       │      9µs │
│ script @ default.yml:2770     │  7.462µs │
│ gsub @ default.yml:20         │      7µs │
╰───────────────────────────────┴──────────╯

--- Benchmark results for package: sentinel_one_cloud_funnel - END   ---
Done

But to take one example, the dot_expander since it's unique and near the head of the file, it is not at line 52, it's at 44.

Internal discussion has raised the possibility that this is due to the presence of reroute rules in the package, something that was found to be an issue in the pipeline test runner (ref:#1647).

Another thing that would help here is to not only identify the processor by its line number, but to also use its tag if it exists.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Team:EcosystemLabel for the Packages Ecosystem team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions