-
Notifications
You must be signed in to change notification settings - Fork 127
Open
Labels
Team:EcosystemLabel for the Packages Ecosystem teamLabel for the Packages Ecosystem team
Description
When running benchmarks, the mapping between the processors listed in the output and the actual source appears to be broken.
$ git rev-parse HEAD
1595cc1388bf5054c47d5828535ff772944804df
$ pwd
…/github.com/elastic/integrations/packages/sentinel_one_cloud_funnel
$ elastic-package benchmark pipeline -d event
<snip>
Run pipeline benchmarks for the package
--- Benchmark results for package: sentinel_one_cloud_funnel - START ---
╭─────────────────────────╮
│ parameters │
├──────────────────┬──────┤
│ source_doc_count │ 16 │
│ doc_count │ 1000 │
╰──────────────────┴──────╯
╭───────────────────────────╮
│ pipeline_performance │
├─────────────────┬─────────┤
│ processing_time │ 0.69s │
│ eps │ 1440.92 │
╰─────────────────┴─────────╯
╭────────────────────────────────────────╮
│ procs_by_total_time │
├───────────────────────────────┬────────┤
│ dot_expander @ default.yml:52 │ 19.31% │
│ json @ default.yml:34 │ 4.61% │
│ remove @ default.yml:2778 │ 3.89% │
│ script @ default.yml:2849 │ 1.73% │
│ date @ default.yml:87 │ 1.44% │
│ date @ default.yml:533 │ 1.44% │
│ date @ default.yml:1723 │ 1.30% │
│ gsub @ default.yml:20 │ 1.01% │
│ script @ default.yml:2770 │ 1.01% │
│ script @ pipeline-dns.yml:21 │ 0.72% │
╰───────────────────────────────┴────────╯
╭──────────────────────────────────────────╮
│ procs_by_avg_time_per_doc │
├───────────────────────────────┬──────────┤
│ dot_expander @ default.yml:52 │ 134µs │
│ json @ default.yml:34 │ 32µs │
│ remove @ default.yml:2778 │ 27µs │
│ script @ pipeline-dns.yml:21 │ 19.841µs │
│ script @ default.yml:2849 │ 12µs │
│ date @ default.yml:533 │ 10.66µs │
│ date @ default.yml:87 │ 10µs │
│ date @ default.yml:1723 │ 9µs │
│ script @ default.yml:2770 │ 7.462µs │
│ gsub @ default.yml:20 │ 7µs │
╰───────────────────────────────┴──────────╯
--- Benchmark results for package: sentinel_one_cloud_funnel - END ---
Done
But to take one example, the dot_expander since it's unique and near the head of the file, it is not at line 52, it's at 44.
Internal discussion has raised the possibility that this is due to the presence of reroute rules in the package, something that was found to be an issue in the pipeline test runner (ref:#1647).
Another thing that would help here is to not only identify the processor by its line number, but to also use its tag if it exists.
Metadata
Metadata
Assignees
Labels
Team:EcosystemLabel for the Packages Ecosystem teamLabel for the Packages Ecosystem team