Skip to content

Conversation

LucaMarconato
Copy link
Member

I added some basic sets of benchmarks for measuring the time to produce plots. The correctness of the plots is currently not checked, we can modify it later.

The only thing missing from the PR is the integration with GitHub Actions, so that we will catch on time eventual future performance drops. To finalize that I need some permissions from scverse.

@codecov-commenter
Copy link

codecov-commenter commented Jul 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.34%. Comparing base (ac4f287) to head (ae80eec).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #295   +/-   ##
=======================================
  Coverage   80.34%   80.34%           
=======================================
  Files          11       11           
  Lines        1745     1745           
=======================================
  Hits         1402     1402           
  Misses        343      343           

@timtreis timtreis added enhancement New feature or request housekeeping 🧹 labels Jul 14, 2024
@melonora
Copy link
Contributor

you mean for the token?

Copy link

codspeed-hq bot commented Aug 14, 2024

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕 12 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

  • test_plot_can_render_large_circles[datashader] (318.2 ms)
  • test_plot_can_render_large_circles[matplotlib] (402.9 ms)
  • test_plot_can_render_large_image[image] (273.7 ms)
  • test_plot_can_render_large_image[multiscale_image] (298.1 ms)
  • test_plot_can_render_large_labels[labels] (431.3 ms)
  • test_plot_can_render_large_labels[multiscale_labels] (448.4 ms)
  • test_plot_can_render_large_multipolygons[datashader] (499 ms)
  • test_plot_can_render_large_multipolygons[matplotlib] (622.9 ms)
  • test_plot_can_render_large_points[datashader] (514.3 ms)
  • test_plot_can_render_large_points[matplotlib] (250.1 ms)
  • test_plot_can_render_large_polygons[datashader] (382.1 ms)
  • test_plot_can_render_large_polygons[matplotlib] (438.8 ms)

@LucaMarconato
Copy link
Member Author

LucaMarconato commented Aug 15, 2024

Good news I setup everything. Bad news, CodSpeed doesn't seem to be suitable for the benchmarking use cases of spatialdata-plot because it excludes system calls by design, and we need them when looking at plotting performance.

See for example this screenshot; the execution time for test_plot_can_render_large_image[image] is 54.2s, but CodSpeed reports 273.7 milliseconds because most of the time is spent on system calls:
image

I wrote to the devs to see if there is an option to get around this, but most likely I'll end up closing this PR and switching to asv; which btw @Czaki is setting up on napari-spatialdata scverse/napari-spatialdata#289 right today, good timing!

EDIT: gonna close the PR already, wait for an answer and unless we can go back to CodSpeed, we can proceed with airspeed velocity (asv).

@LucaMarconato
Copy link
Member Author

I archived the branch behind this PR here https://github.com/scverse/spatialdata-plot/releases/tag/archive%2Fcodspeed_benchmarks, I'll delete this branch.

@LucaMarconato LucaMarconato deleted the benchmark branch August 15, 2024 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request housekeeping 🧹
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants