Skip to content

dvc-bench: rework our framework#296

Merged
efiop merged 1 commit intomasterfrom
pytest
Aug 18, 2021
Merged

dvc-bench: rework our framework#296
efiop merged 1 commit intomasterfrom
pytest

Conversation

@efiop
Copy link
Copy Markdown
Contributor

@efiop efiop commented Aug 13, 2021 •
edited
Loading

Draft

  • introduced benchmark categories:

    • cli: should be able to run these with any dvc (rpm, deb, pypi, snap, etc) (could be used in dvc-test repo too)
      • commands: granular tests for individual commands. These should have a cached setup, so that we could use them during rapid development instead of our hand-written scripts. Every test could be run in a separate machine.
      • stories: multistage start-to-end benchmarks, useful for testing workflows (e.g. in documentation, see test_sharing inspired by sharing-data-and-models use-case. Every full story could be run in a separate machine.
    • api: for python api only.
      • methods: granular tests for individual methods (e.g. api.open/read). Same reasoning as in cli.commands
      • stories: same as cli.stories but for our api. E.g. imagine using our api with pandas or smth like that.

  • migrated to pytest-benchmark

    • using same familiar fixtures from dvc core repo

  • introduced creature comforts for developers:

    • using small dataset by default, could be configured with --size(e.g. tiny(20), small(2K), large(70K), etc). All datasets are based on the same list of files and only differ by the number of files, to leverage deduplication and provide better performance.
    • datasets are pulled dynamically to improve performance and comfort
    • using local remote by default, could be configured with --remote(e.g. s3, azure, etc)
    • now anyone is able to just run pytest and get quick-ish results without having tribal knowledge
    • dataset and make_dataset fixtures for quick test setup

    TODO:

    • cleanup dvc fixture copies
    • migrate leftover tests
    • pbar for dynamic pulling
    • generate html with histograms generated html with CLI output, seems like it is enough for now and will not use much bandwith. Generated PR comments are a bit ugly, but fine for a POC.
    • adjust gha to use this
    • move to dvc repo? create a package with dvc repo fixtures?

    Related to resurrection epic #292

@efiop efiop mentioned this pull request Aug 13, 2021
Closed
@efiop efiop force-pushed the pytest branch 2 times, most recently from 86e1dc4 to 67e37eb Compare August 13, 2021 22:35
@efiop efiop mentioned this pull request Aug 16, 2021
Closed
1 task
@efiop efiop force-pushed the pytest branch 22 times, most recently from ab77ff4 to 54ec10a Compare August 17, 2021 23:27
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 17, 2021

name min max mean stddev median iqr outliers ops rounds iterations
tests/benchmarks/cli/commands/test_add.py::test_add 8.441827803999956 8.441827803999956 8.441827803999956 0 8.441827803999956 0.0 0;0 0.11845775858235039 1 1
tests/benchmarks/cli/commands/test_add.py::test_add 12.864210421999985 12.864210421999985 12.864210421999985 0 12.864210421999985 0.0 0;0 0.07773504686224894 1 1
tests/benchmarks/cli/commands/test_diff.py::test_add 46.47587393700002 46.47587393700002 46.47587393700002 0 46.47587393700002 0.0 0;0 0.021516539987081074 1 1
tests/benchmarks/cli/commands/test_diff.py::test_add 4.846063861999994 4.846063861999994 4.846063861999994 0 4.846063861999994 0.0 0;0 0.20635303794516963 1 1
tests/benchmarks/cli/commands/test_checkout.py::test_checkout 3.3878940260000263 3.3878940260000263 3.3878940260000263 0 3.3878940260000263 0.0 0;0 0.2951686187128665 1 1
tests/benchmarks/cli/commands/test_checkout.py::test_checkout 3.2801123249999478 3.2801123249999478 3.2801123249999478 0 3.2801123249999478 0.0 0;0 0.30486760845911454 1 1
tests/benchmarks/cli/commands/test_gc.py::test_gc 0.47846341300001427 0.47846341300001427 0.47846341300001427 0 0.47846341300001427 0.0 0;0 2.090023965949451 1 1
tests/benchmarks/cli/commands/test_gc.py::test_gc 0.4761868330000141 0.4761868330000141 0.4761868330000141 0 0.4761868330000141 0.0 0;0 2.1000160665928584 1 1
tests/benchmarks/cli/commands/test_get.py::test_get 48.190255501999985 48.190255501999985 48.190255501999985 0 48.190255501999985 0.0 0;0 0.0207510831719599 1 1
tests/benchmarks/cli/commands/test_get.py::test_get 4.634129410000014 4.634129410000014 4.634129410000014 0 4.634129410000014 0.0 0;0 0.2157902621023259 1 1
tests/benchmarks/cli/commands/test_help.py::test_help 0.2224361270000088 0.2224361270000088 0.2224361270000088 0 0.2224361270000088 0.0 0;0 4.495672593687807 1 1
tests/benchmarks/cli/commands/test_help.py::test_help 0.31420090199998185 0.31420090199998185 0.31420090199998185 0 0.31420090199998185 0.0 0;0 3.1826770503671495 1 1
tests/benchmarks/cli/commands/test_import.py::test_import 8.875956008000031 8.875956008000031 8.875956008000031 0 8.875956008000031 0.0 0;0 0.11266392026939803 1 1
tests/benchmarks/cli/commands/test_import.py::test_import 14.424068219999981 14.424068219999981 14.424068219999981 0 14.424068219999981 0.0 0;0 0.06932856838637451 1 1
tests/benchmarks/cli/commands/test_init.py::test_init 0.5088682439999275 0.5088682439999275 0.5088682439999275 0 0.5088682439999275 0.0 0;0 1.9651452252936077 1 1
tests/benchmarks/cli/commands/test_init.py::test_init 0.5032650580000109 0.5032650580000109 0.5032650580000109 0 0.5032650580000109 0.0 0;0 1.9870244995232282 1 1
tests/benchmarks/cli/commands/test_pull.py::test_pull 7.7042442889999165 7.7042442889999165 7.7042442889999165 0 7.7042442889999165 0.0 0;0 0.12979858406460387 1 1
tests/benchmarks/cli/commands/test_pull.py::test_pull 8.571047072000056 8.571047072000056 8.571047072000056 0 8.571047072000056 0.0 0;0 0.11667185952890231 1 1
tests/benchmarks/cli/commands/test_push.py::test_push 4.271682398999985 4.271682398999985 4.271682398999985 0 4.271682398999985 0.0 0;0 0.2340998011074286 1 1
tests/benchmarks/cli/commands/test_push.py::test_push 5.178573964000009 5.178573964000009 5.178573964000009 0 5.178573964000009 0.0 0;0 0.19310335373245976 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 7.014496135999934 7.014496135999934 7.014496135999934 0 7.014496135999934 0.0 0;0 0.14256191472795607 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 10.704579694000017 10.704579694000017 10.704579694000017 0 10.704579694000017 0.0 0;0 0.09341796021757923 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 1.2241695790000904 1.2241695790000904 1.2241695790000904 0 1.2241695790000904 0.0 0;0 0.8168802894259196 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 1.156288520999965 1.156288520999965 1.156288520999965 0 1.156288520999965 0.0 0;0 0.8648360524544465 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 0.5423066140000401 0.5423066140000401 0.5423066140000401 0 0.5423066140000401 0.0 0;0 1.843975297708477 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 0.4845038129999466 0.4845038129999466 0.4845038129999466 0 0.4845038129999466 0.0 0;0 2.063967244774006 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 7.688608448000082 7.688608448000082 7.688608448000082 0 7.688608448000082 0.0 0;0 0.13006254730790906 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 8.557927853000024 8.557927853000024 8.557927853000024 0 8.557927853000024 0.0 0;0 0.1168507163389377 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 1.2026489180000226 1.2026489180000226 1.2026489180000226 0 1.2026489180000226 0.0 0;0 0.8314978586294136 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 0.5421106979999877 0.5421106979999877 0.5421106979999877 0 0.5421106979999877 0.0 0;0 1.844641700835837 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 4.049912914999936 4.049912914999936 4.049912914999936 0 4.049912914999936 0.0 0;0 0.2469188896127204 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 4.613341645000105 4.613341645000105 4.613341645000105 0 4.613341645000105 0.0 0;0 0.21676261524740756 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 0.4536469990000569 0.4536469990000569 0.4536469990000569 0 0.4536469990000569 0.0 0;0 2.2043571371666335 1 1
tests/benchmarks/cli/stories/use-cases/test_sharing.py::test_sharing 0.512326078000001 0.512326078000001 0.512326078000001 0 0.512326078000001 0.0 0;0 1.9518819028376653 1 1
tests/benchmarks/cli/commands/test_status.py::test_status 1.8221231489999923 1.8221231489999923 1.8221231489999923 0 1.8221231489999923 0.0 0;0 0.548810326321146 1 1
tests/benchmarks/cli/commands/test_status.py::test_status 3.434719801 3.434719801 3.434719801 0 3.434719801 0.0 0;0 0.2911445643131808 1 1

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 17, 2021 •
edited by efiop
Loading 

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_add-add': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 7.5500 (1.0) 7.5500 (1.0) 7.5500 (1.0) 0.0000 (1.0) 7.5500 (1.0) 0.0000 (1.0) 0;0 0.1324 (1.0) 1 1
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 11.0543 (1.46) 11.0543 (1.46) 11.0543 (1.46) 0.0000 (1.0) 11.0543 (1.46) 0.0000 (1.0) 0;0 0.0905 (0.68) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_add-diff': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 42.6513 (10.84) 42.6513 (10.84) 42.6513 (10.84) 0.0000 (1.0) 42.6513 (10.84) 0.0000 (1.0) 0;0 0.0234 (0.09) 1 1
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 3.9332 (1.0) 3.9332 (1.0) 3.9332 (1.0) 0.0000 (1.0) 3.9332 (1.0) 0.0000 (1.0) 0;0 0.2542 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0001_2.5.4) 3.2843 (1.06) 3.2843 (1.06) 3.2843 (1.06) 0.0000 (1.0) 3.2843 (1.06) 0.0000 (1.0) 0;0 0.3045 (0.94) 1 1
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0002_2.6.0) 3.0933 (1.0) 3.0933 (1.0) 3.0933 (1.0) 0.0000 (1.0) 3.0933 (1.0) 0.0000 (1.0) 0;0 0.3233 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_gc-gc': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0001_2.5.4) 482.9832 (1.01) 482.9832 (1.01) 482.9832 (1.01) 0.0000 (1.0) 482.9832 (1.01) 0.0000 (1.0) 0;0 2.0705 (0.99) 1 1
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0002_2.6.0) 477.2167 (1.0) 477.2167 (1.0) 477.2167 (1.0) 0.0000 (1.0) 477.2167 (1.0) 0.0000 (1.0) 0;0 2.0955 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_get-get': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0001_2.5.4) 48.9028 (9.12) 48.9028 (9.12) 48.9028 (9.12) 0.0000 (1.0) 48.9028 (9.12) 0.0000 (1.0) 0;0 0.0204 (0.11) 1 1
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0002_2.6.0) 5.3610 (1.0) 5.3610 (1.0) 5.3610 (1.0) 0.0000 (1.0) 5.3610 (1.0) 0.0000 (1.0) 0;0 0.1865 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_help---help': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0001_2.5.4) 222.3318 (1.0) 222.3318 (1.0) 222.3318 (1.0) 0.0000 (1.0) 222.3318 (1.0) 0.0000 (1.0) 0;0 4.4978 (1.0) 1 1
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0002_2.6.0) 305.5536 (1.37) 305.5536 (1.37) 305.5536 (1.37) 0.0000 (1.0) 305.5536 (1.37) 0.0000 (1.0) 0;0 3.2727 (0.73) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_import-import': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0001_2.5.4) 8.1231 (1.0) 8.1231 (1.0) 8.1231 (1.0) 0.0000 (1.0) 8.1231 (1.0) 0.0000 (1.0) 0;0 0.1231 (1.0) 1 1
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0002_2.6.0) 13.5845 (1.67) 13.5845 (1.67) 13.5845 (1.67) 0.0000 (1.0) 13.5845 (1.67) 0.0000 (1.0) 0;0 0.0736 (0.60) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_init-init': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0001_2.5.4) 775.9207 (1.73) 775.9207 (1.73) 775.9207 (1.73) 0.0000 (1.0) 775.9207 (1.73) 0.0000 (1.0) 0;0 1.2888 (0.58) 1 1
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0002_2.6.0) 447.8031 (1.0) 447.8031 (1.0) 447.8031 (1.0) 0.0000 (1.0) 447.8031 (1.0) 0.0000 (1.0) 0;0 2.2331 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_pull-pull': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0001_2.5.4) 7.8881 (1.0) 7.8881 (1.0) 7.8881 (1.0) 0.0000 (1.0) 7.8881 (1.0) 0.0000 (1.0) 0;0 0.1268 (1.0) 1 1
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0002_2.6.0) 8.4652 (1.07) 8.4652 (1.07) 8.4652 (1.07) 0.0000 (1.0) 8.4652 (1.07) 0.0000 (1.0) 0;0 0.1181 (0.93) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_push-push': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0001_2.5.4) 3.2216 (1.0) 3.2216 (1.0) 3.2216 (1.0) 0.0000 (1.0) 3.2216 (1.0) 0.0000 (1.0) 0;0 0.3104 (1.0) 1 1
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0002_2.6.0) 3.7173 (1.15) 3.7173 (1.15) 3.7173 (1.15) 0.0000 (1.0) 3.7173 (1.15) 0.0000 (1.0) 0;0 0.2690 (0.87) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-add': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 5.2304 (1.0) 5.2304 (1.0) 5.2304 (1.0) 0.0000 (1.0) 5.2304 (1.0) 0.0000 (1.0) 0;0 0.1912 (1.0) 1 1
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 8.0768 (1.54) 8.0768 (1.54) 8.0768 (1.54) 0.0000 (1.0) 8.0768 (1.54) 0.0000 (1.0) 0;0 0.1238 (0.65) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 1.0320 (1.07) 1.0320 (1.07) 1.0320 (1.07) 0.0000 (1.0) 1.0320 (1.07) 0.0000 (1.0) 0;0 0.9690 (0.93) 1 1
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 0.9623 (1.0) 0.9623 (1.0) 0.9623 (1.0) 0.0000 (1.0) 0.9623 (1.0) 0.0000 (1.0) 0;0 1.0392 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 441.8298 (1.05) 441.8298 (1.05) 441.8298 (1.05) 0.0000 (1.0) 441.8298 (1.05) 0.0000 (1.0) 0;0 2.2633 (0.95) 1 1
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 419.8354 (1.0) 419.8354 (1.0) 419.8354 (1.0) 0.0000 (1.0) 419.8354 (1.0) 0.0000 (1.0) 0;0 2.3819 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 5.8941 (1.0) 5.8941 (1.0) 5.8941 (1.0) 0.0000 (1.0) 5.8941 (1.0) 0.0000 (1.0) 0;0 0.1697 (1.0) 1 1
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 6.4100 (1.09) 6.4100 (1.09) 6.4100 (1.09) 0.0000 (1.0) 6.4100 (1.09) 0.0000 (1.0) 0;0 0.1560 (0.92) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 883.7586 (2.01) 883.7586 (2.01) 883.7586 (2.01) 0.0000 (1.0) 883.7586 (2.01) 0.0000 (1.0) 0;0 1.1315 (0.50) 1 1
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 439.3674 (1.0) 439.3674 (1.0) 439.3674 (1.0) 0.0000 (1.0) 439.3674 (1.0) 0.0000 (1.0) 0;0 2.2760 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 3.1435 (1.0) 3.1435 (1.0) 3.1435 (1.0) 0.0000 (1.0) 3.1435 (1.0) 0.0000 (1.0) 0;0 0.3181 (1.0) 1 1
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 3.6062 (1.15) 3.6062 (1.15) 3.6062 (1.15) 0.0000 (1.0) 3.6062 (1.15) 0.0000 (1.0) 0;0 0.2773 (0.87) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 378.9884 (1.0) 378.9884 (1.0) 378.9884 (1.0) 0.0000 (1.0) 378.9884 (1.0) 0.0000 (1.0) 0;0 2.6386 (1.0) 1 1
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 423.0687 (1.12) 423.0687 (1.12) 423.0687 (1.12) 0.0000 (1.0) 423.0687 (1.12) 0.0000 (1.0) 0;0 2.3637 (0.90) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_status-status': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0001_2.5.4) 2.8606 (1.0) 2.8606 (1.0) 2.8606 (1.0) 0.0000 (1.0) 2.8606 (1.0) 0.0000 (1.0) 0;0 0.3496 (1.0) 1 1
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0002_2.6.0) 4.6840 (1.64) 4.6840 (1.64) 4.6840 (1.64) 0.0000 (1.0) 4.6840 (1.64) 0.0000 (1.0) 0;0 0.2135 (0.61) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 18, 2021 

--------------------------------------------------------------------------------------------------------------------- benchmark 'test_add-add': 2 tests ----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 5.6571 (1.0) 5.6571 (1.0) 5.6571 (1.0) 0.0000 (1.0) 5.6571 (1.0) 0.0000 (1.0) 0;0 0.1768 (1.0) 1 1
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 8.9232 (1.58) 8.9232 (1.58) 8.9232 (1.58) 0.0000 (1.0) 8.9232 (1.58) 0.0000 (1.0) 0;0 0.1121 (0.63) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_add-diff': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 41.9268 (11.59) 41.9268 (11.59) 41.9268 (11.59) 0.0000 (1.0) 41.9268 (11.59) 0.0000 (1.0) 0;0 0.0239 (0.09) 1 1
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 3.6165 (1.0) 3.6165 (1.0) 3.6165 (1.0) 0.0000 (1.0) 3.6165 (1.0) 0.0000 (1.0) 0;0 0.2765 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0001_2.5.4) 2.7309 (1.01) 2.7309 (1.01) 2.7309 (1.01) 0.0000 (1.0) 2.7309 (1.01) 0.0000 (1.0) 0;0 0.3662 (0.99) 1 1
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0002_2.6.0) 2.7043 (1.0) 2.7043 (1.0) 2.7043 (1.0) 0.0000 (1.0) 2.7043 (1.0) 0.0000 (1.0) 0;0 0.3698 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_gc-gc': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0001_2.5.4) 534.4385 (1.03) 534.4385 (1.03) 534.4385 (1.03) 0.0000 (1.0) 534.4385 (1.03) 0.0000 (1.0) 0;0 1.8711 (0.97) 1 1
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0002_2.6.0) 518.9603 (1.0) 518.9603 (1.0) 518.9603 (1.0) 0.0000 (1.0) 518.9603 (1.0) 0.0000 (1.0) 0;0 1.9269 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_get-get': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0001_2.5.4) 43.9958 (8.91) 43.9958 (8.91) 43.9958 (8.91) 0.0000 (1.0) 43.9958 (8.91) 0.0000 (1.0) 0;0 0.0227 (0.11) 1 1
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0002_2.6.0) 4.9398 (1.0) 4.9398 (1.0) 4.9398 (1.0) 0.0000 (1.0) 4.9398 (1.0) 0.0000 (1.0) 0;0 0.2024 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_help---help': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0001_2.5.4) 207.3475 (1.0) 207.3475 (1.0) 207.3475 (1.0) 0.0000 (1.0) 207.3475 (1.0) 0.0000 (1.0) 0;0 4.8228 (1.0) 1 1
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0002_2.6.0) 274.5536 (1.32) 274.5536 (1.32) 274.5536 (1.32) 0.0000 (1.0) 274.5536 (1.32) 0.0000 (1.0) 0;0 3.6423 (0.76) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_import-import': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0001_2.5.4) 8.3774 (1.0) 8.3774 (1.0) 8.3774 (1.0) 0.0000 (1.0) 8.3774 (1.0) 0.0000 (1.0) 0;0 0.1194 (1.0) 1 1
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0002_2.6.0) 14.0985 (1.68) 14.0985 (1.68) 14.0985 (1.68) 0.0000 (1.0) 14.0985 (1.68) 0.0000 (1.0) 0;0 0.0709 (0.59) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_init-init': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0001_2.5.4) 499.7290 (1.08) 499.7290 (1.08) 499.7290 (1.08) 0.0000 (1.0) 499.7290 (1.08) 0.0000 (1.0) 0;0 2.0011 (0.92) 1 1
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0002_2.6.0) 460.8847 (1.0) 460.8847 (1.0) 460.8847 (1.0) 0.0000 (1.0) 460.8847 (1.0) 0.0000 (1.0) 0;0 2.1697 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_pull-pull': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0001_2.5.4) 7.7696 (1.0) 7.7696 (1.0) 7.7696 (1.0) 0.0000 (1.0) 7.7696 (1.0) 0.0000 (1.0) 0;0 0.1287 (1.0) 1 1
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0002_2.6.0) 9.4503 (1.22) 9.4503 (1.22) 9.4503 (1.22) 0.0000 (1.0) 9.4503 (1.22) 0.0000 (1.0) 0;0 0.1058 (0.82) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_push-push': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0001_2.5.4) 3.2337 (1.0) 3.2337 (1.0) 3.2337 (1.0) 0.0000 (1.0) 3.2337 (1.0) 0.0000 (1.0) 0;0 0.3092 (1.0) 1 1
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0002_2.6.0) 3.7048 (1.15) 3.7048 (1.15) 3.7048 (1.15) 0.0000 (1.0) 3.7048 (1.15) 0.0000 (1.0) 0;0 0.2699 (0.87) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-add': 2 tests --------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 7.8929 (1.0) 7.8929 (1.0) 7.8929 (1.0) 0.0000 (1.0) 7.8929 (1.0) 0.0000 (1.0) 0;0 0.1267 (1.0) 1 1
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 12.1495 (1.54) 12.1495 (1.54) 12.1495 (1.54) 0.0000 (1.0) 12.1495 (1.54) 0.0000 (1.0) 0;0 0.0823 (0.65) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 1.2922 (1.04) 1.2922 (1.04) 1.2922 (1.04) 0.0000 (1.0) 1.2922 (1.04) 0.0000 (1.0) 0;0 0.7739 (0.96) 1 1
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 1.2400 (1.0) 1.2400 (1.0) 1.2400 (1.0) 0.0000 (1.0) 1.2400 (1.0) 0.0000 (1.0) 0;0 0.8064 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 561.2812 (1.04) 561.2812 (1.04) 561.2812 (1.04) 0.0000 (1.0) 561.2812 (1.04) 0.0000 (1.0) 0;0 1.7816 (0.97) 1 1
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 542.2634 (1.0) 542.2634 (1.0) 542.2634 (1.0) 0.0000 (1.0) 542.2634 (1.0) 0.0000 (1.0) 0;0 1.8441 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 8.1067 (1.0) 8.1067 (1.0) 8.1067 (1.0) 0.0000 (1.0) 8.1067 (1.0) 0.0000 (1.0) 0;0 0.1234 (1.0) 1 1
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 8.7818 (1.08) 8.7818 (1.08) 8.7818 (1.08) 0.0000 (1.0) 8.7818 (1.08) 0.0000 (1.0) 0;0 0.1139 (0.92) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull-noop': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 1.2820 (2.20) 1.2820 (2.20) 1.2820 (2.20) 0.0000 (1.0) 1.2820 (2.20) 0.0000 (1.0) 0;0 0.7800 (0.45) 1 1
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 0.5830 (1.0) 0.5830 (1.0) 0.5830 (1.0) 0.0000 (1.0) 0.5830 (1.0) 0.0000 (1.0) 0;0 1.7153 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 4.4604 (1.0) 4.4604 (1.0) 4.4604 (1.0) 0.0000 (1.0) 4.4604 (1.0) 0.0000 (1.0) 0;0 0.2242 (1.0) 1 1
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 5.2079 (1.17) 5.2079 (1.17) 5.2079 (1.17) 0.0000 (1.0) 5.2079 (1.17) 0.0000 (1.0) 0;0 0.1920 (0.86) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 504.8620 (1.0) 504.8620 (1.0) 504.8620 (1.0) 0.0000 (1.0) 504.8620 (1.0) 0.0000 (1.0) 0;0 1.9807 (1.0) 1 1
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 558.8041 (1.11) 558.8041 (1.11) 558.8041 (1.11) 0.0000 (1.0) 558.8041 (1.11) 0.0000 (1.0) 0;0 1.7895 (0.90) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_status-status': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0001_2.5.4) 2.7246 (1.0) 2.7246 (1.0) 2.7246 (1.0) 0.0000 (1.0) 2.7246 (1.0) 0.0000 (1.0) 0;0 0.3670 (1.0) 1 1
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0002_2.6.0) 5.2733 (1.94) 5.2733 (1.94) 5.2733 (1.94) 0.0000 (1.0) 5.2733 (1.94) 0.0000 (1.0) 0;0 0.1896 (0.52) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 18, 2021 

--------------------------------------------------------------------------------------------------------------------- benchmark 'test_add-add': 2 tests ----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 6.4953 (1.0) 6.4953 (1.0) 6.4953 (1.0) 0.0000 (1.0) 6.4953 (1.0) 0.0000 (1.0) 0;0 0.1540 (1.0) 1 1
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 9.4510 (1.46) 9.4510 (1.46) 9.4510 (1.46) 0.0000 (1.0) 9.4510 (1.46) 0.0000 (1.0) 0;0 0.1058 (0.69) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_add-diff': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 48.6462 (9.75) 48.6462 (9.75) 48.6462 (9.75) 0.0000 (1.0) 48.6462 (9.75) 0.0000 (1.0) 0;0 0.0206 (0.10) 1 1
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 4.9906 (1.0) 4.9906 (1.0) 4.9906 (1.0) 0.0000 (1.0) 4.9906 (1.0) 0.0000 (1.0) 0;0 0.2004 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0001_2.5.4) 3.4697 (1.01) 3.4697 (1.01) 3.4697 (1.01) 0.0000 (1.0) 3.4697 (1.01) 0.0000 (1.0) 0;0 0.2882 (0.99) 1 1
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0002_2.6.0) 3.4247 (1.0) 3.4247 (1.0) 3.4247 (1.0) 0.0000 (1.0) 3.4247 (1.0) 0.0000 (1.0) 0;0 0.2920 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_gc-gc': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0001_2.5.4) 565.0407 (1.03) 565.0407 (1.03) 565.0407 (1.03) 0.0000 (1.0) 565.0407 (1.03) 0.0000 (1.0) 0;0 1.7698 (0.97) 1 1
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0002_2.6.0) 549.9220 (1.0) 549.9220 (1.0) 549.9220 (1.0) 0.0000 (1.0) 549.9220 (1.0) 0.0000 (1.0) 0;0 1.8184 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_get-get': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0001_2.5.4) 53.4688 (9.49) 53.4688 (9.49) 53.4688 (9.49) 0.0000 (1.0) 53.4688 (9.49) 0.0000 (1.0) 0;0 0.0187 (0.11) 1 1
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0002_2.6.0) 5.6317 (1.0) 5.6317 (1.0) 5.6317 (1.0) 0.0000 (1.0) 5.6317 (1.0) 0.0000 (1.0) 0;0 0.1776 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_help---help': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0001_2.5.4) 232.2137 (1.0) 232.2137 (1.0) 232.2137 (1.0) 0.0000 (1.0) 232.2137 (1.0) 0.0000 (1.0) 0;0 4.3064 (1.0) 1 1
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0002_2.6.0) 307.3658 (1.32) 307.3658 (1.32) 307.3658 (1.32) 0.0000 (1.0) 307.3658 (1.32) 0.0000 (1.0) 0;0 3.2535 (0.76) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_import-import': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0001_2.5.4) 10.4949 (1.0) 10.4949 (1.0) 10.4949 (1.0) 0.0000 (1.0) 10.4949 (1.0) 0.0000 (1.0) 0;0 0.0953 (1.0) 1 1
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0002_2.6.0) 16.5860 (1.58) 16.5860 (1.58) 16.5860 (1.58) 0.0000 (1.0) 16.5860 (1.58) 0.0000 (1.0) 0;0 0.0603 (0.63) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_init-init': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0001_2.5.4) 520.8480 (1.09) 520.8480 (1.09) 520.8480 (1.09) 0.0000 (1.0) 520.8480 (1.09) 0.0000 (1.0) 0;0 1.9199 (0.92) 1 1
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0002_2.6.0) 476.8093 (1.0) 476.8093 (1.0) 476.8093 (1.0) 0.0000 (1.0) 476.8093 (1.0) 0.0000 (1.0) 0;0 2.0973 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_pull-pull': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0001_2.5.4) 7.1864 (1.0) 7.1864 (1.0) 7.1864 (1.0) 0.0000 (1.0) 7.1864 (1.0) 0.0000 (1.0) 0;0 0.1392 (1.0) 1 1
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0002_2.6.0) 7.9926 (1.11) 7.9926 (1.11) 7.9926 (1.11) 0.0000 (1.0) 7.9926 (1.11) 0.0000 (1.0) 0;0 0.1251 (0.90) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_push-push': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0001_2.5.4) 3.6907 (1.0) 3.6907 (1.0) 3.6907 (1.0) 0.0000 (1.0) 3.6907 (1.0) 0.0000 (1.0) 0;0 0.2710 (1.0) 1 1
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0002_2.6.0) 4.2186 (1.14) 4.2186 (1.14) 4.2186 (1.14) 0.0000 (1.0) 4.2186 (1.14) 0.0000 (1.0) 0;0 0.2370 (0.87) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-add': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 5.6807 (1.0) 5.6807 (1.0) 5.6807 (1.0) 0.0000 (1.0) 5.6807 (1.0) 0.0000 (1.0) 0;0 0.1760 (1.0) 1 1
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 7.9595 (1.40) 7.9595 (1.40) 7.9595 (1.40) 0.0000 (1.0) 7.9595 (1.40) 0.0000 (1.0) 0;0 0.1256 (0.71) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 1.0336 (1.04) 1.0336 (1.04) 1.0336 (1.04) 0.0000 (1.0) 1.0336 (1.04) 0.0000 (1.0) 0;0 0.9675 (0.96) 1 1
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 0.9894 (1.0) 0.9894 (1.0) 0.9894 (1.0) 0.0000 (1.0) 0.9894 (1.0) 0.0000 (1.0) 0;0 1.0107 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 427.5793 (1.0) 427.5793 (1.0) 427.5793 (1.0) 0.0000 (1.0) 427.5793 (1.0) 0.0000 (1.0) 0;0 2.3387 (1.0) 1 1
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 441.7338 (1.03) 441.7338 (1.03) 441.7338 (1.03) 0.0000 (1.0) 441.7338 (1.03) 0.0000 (1.0) 0;0 2.2638 (0.97) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 5.8366 (1.0) 5.8366 (1.0) 5.8366 (1.0) 0.0000 (1.0) 5.8366 (1.0) 0.0000 (1.0) 0;0 0.1713 (1.0) 1 1
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 6.2165 (1.07) 6.2165 (1.07) 6.2165 (1.07) 0.0000 (1.0) 6.2165 (1.07) 0.0000 (1.0) 0;0 0.1609 (0.94) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 914.2284 (2.12) 914.2284 (2.12) 914.2284 (2.12) 0.0000 (1.0) 914.2284 (2.12) 0.0000 (1.0) 0;0 1.0938 (0.47) 1 1
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 430.4053 (1.0) 430.4053 (1.0) 430.4053 (1.0) 0.0000 (1.0) 430.4053 (1.0) 0.0000 (1.0) 0;0 2.3234 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 3.0721 (1.0) 3.0721 (1.0) 3.0721 (1.0) 0.0000 (1.0) 3.0721 (1.0) 0.0000 (1.0) 0;0 0.3255 (1.0) 1 1
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 3.5440 (1.15) 3.5440 (1.15) 3.5440 (1.15) 0.0000 (1.0) 3.5440 (1.15) 0.0000 (1.0) 0;0 0.2822 (0.87) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 373.4741 (1.0) 373.4741 (1.0) 373.4741 (1.0) 0.0000 (1.0) 373.4741 (1.0) 0.0000 (1.0) 0;0 2.6776 (1.0) 1 1
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 416.4951 (1.12) 416.4951 (1.12) 416.4951 (1.12) 0.0000 (1.0) 416.4951 (1.12) 0.0000 (1.0) 0;0 2.4010 (0.90) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_status-status': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0001_2.5.4) 2.6823 (1.0) 2.6823 (1.0) 2.6823 (1.0) 0.0000 (1.0) 2.6823 (1.0) 0.0000 (1.0) 0;0 0.3728 (1.0) 1 1
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0002_2.6.0) 4.7948 (1.79) 4.7948 (1.79) 4.7948 (1.79) 0.0000 (1.0) 4.7948 (1.79) 0.0000 (1.0) 0;0 0.2086 (0.56) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 18, 2021 

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_add-add': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 7.6051 (1.0) 7.6051 (1.0) 7.6051 (1.0) 0.0000 (1.0) 7.6051 (1.0) 0.0000 (1.0) 0;0 0.1315 (1.0) 1 1
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 11.6401 (1.53) 11.6401 (1.53) 11.6401 (1.53) 0.0000 (1.0) 11.6401 (1.53) 0.0000 (1.0) 0;0 0.0859 (0.65) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_add-diff': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 41.3096 (9.34) 41.3096 (9.34) 41.3096 (9.34) 0.0000 (1.0) 41.3096 (9.34) 0.0000 (1.0) 0;0 0.0242 (0.11) 1 1
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 4.4225 (1.0) 4.4225 (1.0) 4.4225 (1.0) 0.0000 (1.0) 4.4225 (1.0) 0.0000 (1.0) 0;0 0.2261 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0001_2.5.4) 2.4285 (1.02) 2.4285 (1.02) 2.4285 (1.02) 0.0000 (1.0) 2.4285 (1.02) 0.0000 (1.0) 0;0 0.4118 (0.98) 1 1
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0002_2.6.0) 2.3694 (1.0) 2.3694 (1.0) 2.3694 (1.0) 0.0000 (1.0) 2.3694 (1.0) 0.0000 (1.0) 0;0 0.4220 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_gc-gc': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0001_2.5.4) 479.3766 (1.05) 479.3766 (1.05) 479.3766 (1.05) 0.0000 (1.0) 479.3766 (1.05) 0.0000 (1.0) 0;0 2.0860 (0.95) 1 1
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0002_2.6.0) 457.3424 (1.0) 457.3424 (1.0) 457.3424 (1.0) 0.0000 (1.0) 457.3424 (1.0) 0.0000 (1.0) 0;0 2.1865 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_get-get': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0001_2.5.4) 57.7434 (9.07) 57.7434 (9.07) 57.7434 (9.07) 0.0000 (1.0) 57.7434 (9.07) 0.0000 (1.0) 0;0 0.0173 (0.11) 1 1
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0002_2.6.0) 6.3696 (1.0) 6.3696 (1.0) 6.3696 (1.0) 0.0000 (1.0) 6.3696 (1.0) 0.0000 (1.0) 0;0 0.1570 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_help---help': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0001_2.5.4) 205.5809 (1.0) 205.5809 (1.0) 205.5809 (1.0) 0.0000 (1.0) 205.5809 (1.0) 0.0000 (1.0) 0;0 4.8643 (1.0) 1 1
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0002_2.6.0) 273.5963 (1.33) 273.5963 (1.33) 273.5963 (1.33) 0.0000 (1.0) 273.5963 (1.33) 0.0000 (1.0) 0;0 3.6550 (0.75) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_import-import': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0001_2.5.4) 6.8291 (1.0) 6.8291 (1.0) 6.8291 (1.0) 0.0000 (1.0) 6.8291 (1.0) 0.0000 (1.0) 0;0 0.1464 (1.0) 1 1
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0002_2.6.0) 11.9041 (1.74) 11.9041 (1.74) 11.9041 (1.74) 0.0000 (1.0) 11.9041 (1.74) 0.0000 (1.0) 0;0 0.0840 (0.57) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_init-init': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0001_2.5.4) 541.4169 (1.04) 541.4169 (1.04) 541.4169 (1.04) 0.0000 (1.0) 541.4169 (1.04) 0.0000 (1.0) 0;0 1.8470 (0.96) 1 1
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0002_2.6.0) 519.4303 (1.0) 519.4303 (1.0) 519.4303 (1.0) 0.0000 (1.0) 519.4303 (1.0) 0.0000 (1.0) 0;0 1.9252 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_pull-pull': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0001_2.5.4) 7.8007 (1.0) 7.8007 (1.0) 7.8007 (1.0) 0.0000 (1.0) 7.8007 (1.0) 0.0000 (1.0) 0;0 0.1282 (1.0) 1 1
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0002_2.6.0) 9.0538 (1.16) 9.0538 (1.16) 9.0538 (1.16) 0.0000 (1.0) 9.0538 (1.16) 0.0000 (1.0) 0;0 0.1105 (0.86) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_push-push': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0001_2.5.4) 4.2416 (1.0) 4.2416 (1.0) 4.2416 (1.0) 0.0000 (1.0) 4.2416 (1.0) 0.0000 (1.0) 0;0 0.2358 (1.0) 1 1
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0002_2.6.0) 4.8511 (1.14) 4.8511 (1.14) 4.8511 (1.14) 0.0000 (1.0) 4.8511 (1.14) 0.0000 (1.0) 0;0 0.2061 (0.87) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-add': 2 tests --------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 6.5611 (1.0) 6.5611 (1.0) 6.5611 (1.0) 0.0000 (1.0) 6.5611 (1.0) 0.0000 (1.0) 0;0 0.1524 (1.0) 1 1
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 10.2157 (1.56) 10.2157 (1.56) 10.2157 (1.56) 0.0000 (1.0) 10.2157 (1.56) 0.0000 (1.0) 0;0 0.0979 (0.64) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 1.1595 (1.06) 1.1595 (1.06) 1.1595 (1.06) 0.0000 (1.0) 1.1595 (1.06) 0.0000 (1.0) 0;0 0.8625 (0.95) 1 1
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 1.0976 (1.0) 1.0976 (1.0) 1.0976 (1.0) 0.0000 (1.0) 1.0976 (1.0) 0.0000 (1.0) 0;0 0.9111 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 512.7340 (1.09) 512.7340 (1.09) 512.7340 (1.09) 0.0000 (1.0) 512.7340 (1.09) 0.0000 (1.0) 0;0 1.9503 (0.92) 1 1
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 469.3176 (1.0) 469.3176 (1.0) 469.3176 (1.0) 0.0000 (1.0) 469.3176 (1.0) 0.0000 (1.0) 0;0 2.1308 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 6.6805 (1.0) 6.6805 (1.0) 6.6805 (1.0) 0.0000 (1.0) 6.6805 (1.0) 0.0000 (1.0) 0;0 0.1497 (1.0) 1 1
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 7.1826 (1.08) 7.1826 (1.08) 7.1826 (1.08) 0.0000 (1.0) 7.1826 (1.08) 0.0000 (1.0) 0;0 0.1392 (0.93) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull-noop': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 1.0311 (2.06) 1.0311 (2.06) 1.0311 (2.06) 0.0000 (1.0) 1.0311 (2.06) 0.0000 (1.0) 0;0 0.9699 (0.48) 1 1
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 0.5000 (1.0) 0.5000 (1.0) 0.5000 (1.0) 0.0000 (1.0) 0.5000 (1.0) 0.0000 (1.0) 0;0 2.0001 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 3.6179 (1.0) 3.6179 (1.0) 3.6179 (1.0) 0.0000 (1.0) 3.6179 (1.0) 0.0000 (1.0) 0;0 0.2764 (1.0) 1 1
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 4.4449 (1.23) 4.4449 (1.23) 4.4449 (1.23) 0.0000 (1.0) 4.4449 (1.23) 0.0000 (1.0) 0;0 0.2250 (0.81) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 433.9024 (1.0) 433.9024 (1.0) 433.9024 (1.0) 0.0000 (1.0) 433.9024 (1.0) 0.0000 (1.0) 0;0 2.3047 (1.0) 1 1
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 491.2965 (1.13) 491.2965 (1.13) 491.2965 (1.13) 0.0000 (1.0) 491.2965 (1.13) 0.0000 (1.0) 0;0 2.0354 (0.88) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_status-status': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0001_2.5.4) 2.6189 (1.0) 2.6189 (1.0) 2.6189 (1.0) 0.0000 (1.0) 2.6189 (1.0) 0.0000 (1.0) 0;0 0.3818 (1.0) 1 1
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0002_2.6.0) 4.5830 (1.75) 4.5830 (1.75) 4.5830 (1.75) 0.0000 (1.0) 4.5830 (1.75) 0.0000 (1.0) 0;0 0.2182 (0.57) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 18, 2021 

--------------------------------------------------------------------------------------------------------------------- benchmark 'test_add-add': 2 tests ----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 6.6417 (1.0) 6.6417 (1.0) 6.6417 (1.0) 0.0000 (1.0) 6.6417 (1.0) 0.0000 (1.0) 0;0 0.1506 (1.0) 1 1
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 9.9599 (1.50) 9.9599 (1.50) 9.9599 (1.50) 0.0000 (1.0) 9.9599 (1.50) 0.0000 (1.0) 0;0 0.1004 (0.67) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_add-diff': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0001_2.5.4) 43.2247 (9.56) 43.2247 (9.56) 43.2247 (9.56) 0.0000 (1.0) 43.2247 (9.56) 0.0000 (1.0) 0;0 0.0231 (0.10) 1 1
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 4.5212 (1.0) 4.5212 (1.0) 4.5212 (1.0) 0.0000 (1.0) 4.5212 (1.0) 0.0000 (1.0) 0;0 0.2212 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0001_2.5.4) 2.3997 (1.01) 2.3997 (1.01) 2.3997 (1.01) 0.0000 (1.0) 2.3997 (1.01) 0.0000 (1.0) 0;0 0.4167 (0.99) 1 1
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0002_2.6.0) 2.3688 (1.0) 2.3688 (1.0) 2.3688 (1.0) 0.0000 (1.0) 2.3688 (1.0) 0.0000 (1.0) 0;0 0.4222 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_gc-gc': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0001_2.5.4) 652.2032 (1.03) 652.2032 (1.03) 652.2032 (1.03) 0.0000 (1.0) 652.2032 (1.03) 0.0000 (1.0) 0;0 1.5333 (0.98) 1 1
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0002_2.6.0) 636.2317 (1.0) 636.2317 (1.0) 636.2317 (1.0) 0.0000 (1.0) 636.2317 (1.0) 0.0000 (1.0) 0;0 1.5718 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_get-get': 2 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0001_2.5.4) 53.7950 (9.35) 53.7950 (9.35) 53.7950 (9.35) 0.0000 (1.0) 53.7950 (9.35) 0.0000 (1.0) 0;0 0.0186 (0.11) 1 1
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0002_2.6.0) 5.7511 (1.0) 5.7511 (1.0) 5.7511 (1.0) 0.0000 (1.0) 5.7511 (1.0) 0.0000 (1.0) 0;0 0.1739 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_help---help': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0001_2.5.4) 199.7678 (1.0) 199.7678 (1.0) 199.7678 (1.0) 0.0000 (1.0) 199.7678 (1.0) 0.0000 (1.0) 0;0 5.0058 (1.0) 1 1
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0002_2.6.0) 265.6996 (1.33) 265.6996 (1.33) 265.6996 (1.33) 0.0000 (1.0) 265.6996 (1.33) 0.0000 (1.0) 0;0 3.7636 (0.75) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_import-import': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0001_2.5.4) 6.8318 (1.0) 6.8318 (1.0) 6.8318 (1.0) 0.0000 (1.0) 6.8318 (1.0) 0.0000 (1.0) 0;0 0.1464 (1.0) 1 1
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0002_2.6.0) 11.2924 (1.65) 11.2924 (1.65) 11.2924 (1.65) 0.0000 (1.0) 11.2924 (1.65) 0.0000 (1.0) 0;0 0.0886 (0.60) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_init-init': 2 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0001_2.5.4) 409.9836 (1.05) 409.9836 (1.05) 409.9836 (1.05) 0.0000 (1.0) 409.9836 (1.05) 0.0000 (1.0) 0;0 2.4391 (0.95) 1 1
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0002_2.6.0) 389.9706 (1.0) 389.9706 (1.0) 389.9706 (1.0) 0.0000 (1.0) 389.9706 (1.0) 0.0000 (1.0) 0;0 2.5643 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_pull-pull': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0001_2.5.4) 6.6974 (1.0) 6.6974 (1.0) 6.6974 (1.0) 0.0000 (1.0) 6.6974 (1.0) 0.0000 (1.0) 0;0 0.1493 (1.0) 1 1
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0002_2.6.0) 8.2515 (1.23) 8.2515 (1.23) 8.2515 (1.23) 0.0000 (1.0) 8.2515 (1.23) 0.0000 (1.0) 0;0 0.1212 (0.81) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_push-push': 2 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0001_2.5.4) 4.1096 (1.0) 4.1096 (1.0) 4.1096 (1.0) 0.0000 (1.0) 4.1096 (1.0) 0.0000 (1.0) 0;0 0.2433 (1.0) 1 1
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0002_2.6.0) 4.8886 (1.19) 4.8886 (1.19) 4.8886 (1.19) 0.0000 (1.0) 4.8886 (1.19) 0.0000 (1.0) 0;0 0.2046 (0.84) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-add': 2 tests --------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 7.1945 (1.0) 7.1945 (1.0) 7.1945 (1.0) 0.0000 (1.0) 7.1945 (1.0) 0.0000 (1.0) 0;0 0.1390 (1.0) 1 1
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 11.0852 (1.54) 11.0852 (1.54) 11.0852 (1.54) 0.0000 (1.0) 11.0852 (1.54) 0.0000 (1.0) 0;0 0.0902 (0.65) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 1.3211 (1.09) 1.3211 (1.09) 1.3211 (1.09) 0.0000 (1.0) 1.3211 (1.09) 0.0000 (1.0) 0;0 0.7569 (0.92) 1 1
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 1.2117 (1.0) 1.2117 (1.0) 1.2117 (1.0) 0.0000 (1.0) 1.2117 (1.0) 0.0000 (1.0) 0;0 0.8253 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 522.0221 (1.01) 522.0221 (1.01) 522.0221 (1.01) 0.0000 (1.0) 522.0221 (1.01) 0.0000 (1.0) 0;0 1.9156 (0.99) 1 1
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 516.9437 (1.0) 516.9437 (1.0) 516.9437 (1.0) 0.0000 (1.0) 516.9437 (1.0) 0.0000 (1.0) 0;0 1.9344 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 7.5596 (1.0) 7.5596 (1.0) 7.5596 (1.0) 0.0000 (1.0) 7.5596 (1.0) 0.0000 (1.0) 0;0 0.1323 (1.0) 1 1
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 7.7947 (1.03) 7.7947 (1.03) 7.7947 (1.03) 0.0000 (1.0) 7.7947 (1.03) 0.0000 (1.0) 0;0 0.1283 (0.97) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull-noop': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 1.0587 (2.00) 1.0587 (2.00) 1.0587 (2.00) 0.0000 (1.0) 1.0587 (2.00) 0.0000 (1.0) 0;0 0.9445 (0.50) 1 1
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 0.5284 (1.0) 0.5284 (1.0) 0.5284 (1.0) 0.0000 (1.0) 0.5284 (1.0) 0.0000 (1.0) 0;0 1.8926 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 2 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 4.0193 (1.0) 4.0193 (1.0) 4.0193 (1.0) 0.0000 (1.0) 4.0193 (1.0) 0.0000 (1.0) 0;0 0.2488 (1.0) 1 1
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 4.7479 (1.18) 4.7479 (1.18) 4.7479 (1.18) 0.0000 (1.0) 4.7479 (1.18) 0.0000 (1.0) 0;0 0.2106 (0.85) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 2 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.5.4) 480.5870 (1.0) 480.5870 (1.0) 480.5870 (1.0) 0.0000 (1.0) 480.5870 (1.0) 0.0000 (1.0) 0;0 2.0808 (1.0) 1 1
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 540.9338 (1.13) 540.9338 (1.13) 540.9338 (1.13) 0.0000 (1.0) 540.9338 (1.13) 0.0000 (1.0) 0;0 1.8487 (0.89) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_status-status': 2 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0001_2.5.4) 2.5629 (1.0) 2.5629 (1.0) 2.5629 (1.0) 0.0000 (1.0) 2.5629 (1.0) 0.0000 (1.0) 0;0 0.3902 (1.0) 1 1
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0002_2.6.0) 4.5022 (1.76) 4.5022 (1.76) 4.5022 (1.76) 0.0000 (1.0) 4.5022 (1.76) 0.0000 (1.0) 0;0 0.2221 (0.57) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```

@efiop efiop force-pushed the pytest branch 2 times, most recently from 06e2788 to e27c10d Compare August 18, 2021 02:07
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 18, 2021 

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_add-add': 3 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0001_2.6.3) 11.3925 (1.57) 11.3925 (1.57) 11.3925 (1.57) 0.0000 (1.0) 11.3925 (1.57) 0.0000 (1.0) 0;0 0.0878 (0.64) 1 1
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 10.9003 (1.50) 10.9003 (1.50) 10.9003 (1.50) 0.0000 (1.0) 10.9003 (1.50) 0.0000 (1.0) 0;0 0.0917 (0.66) 1 1
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0003_2.5.4) 7.2472 (1.0) 7.2472 (1.0) 7.2472 (1.0) 0.0000 (1.0) 7.2472 (1.0) 0.0000 (1.0) 0;0 0.1380 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_add-diff': 3 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0001_2.6.3) 4.1816 (1.0) 4.1816 (1.0) 4.1816 (1.0) 0.0000 (1.0) 4.1816 (1.0) 0.0000 (1.0) 0;0 0.2391 (1.0) 1 1
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 4.4710 (1.07) 4.4710 (1.07) 4.4710 (1.07) 0.0000 (1.0) 4.4710 (1.07) 0.0000 (1.0) 0;0 0.2237 (0.94) 1 1
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0003_2.5.4) 40.2071 (9.62) 40.2071 (9.62) 40.2071 (9.62) 0.0000 (1.0) 40.2071 (9.62) 0.0000 (1.0) 0;0 0.0249 (0.10) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 3 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0001_2.6.3) 3.3415 (1.06) 3.3415 (1.06) 3.3415 (1.06) 0.0000 (1.0) 3.3415 (1.06) 0.0000 (1.0) 0;0 0.2993 (0.94) 1 1
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0002_2.6.0) 3.1451 (1.0) 3.1451 (1.0) 3.1451 (1.0) 0.0000 (1.0) 3.1451 (1.0) 0.0000 (1.0) 0;0 0.3180 (1.0) 1 1
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0003_2.5.4) 3.8622 (1.23) 3.8622 (1.23) 3.8622 (1.23) 0.0000 (1.0) 3.8622 (1.23) 0.0000 (1.0) 0;0 0.2589 (0.81) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_gc-gc': 3 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0001_2.6.3) 661.7303 (1.13) 661.7303 (1.13) 661.7303 (1.13) 0.0000 (1.0) 661.7303 (1.13) 0.0000 (1.0) 0;0 1.5112 (0.88) 1 1
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0002_2.6.0) 603.6646 (1.03) 603.6646 (1.03) 603.6646 (1.03) 0.0000 (1.0) 603.6646 (1.03) 0.0000 (1.0) 0;0 1.6565 (0.97) 1 1
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0003_2.5.4) 584.7039 (1.0) 584.7039 (1.0) 584.7039 (1.0) 0.0000 (1.0) 584.7039 (1.0) 0.0000 (1.0) 0;0 1.7103 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_get-get': 3 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0001_2.6.3) 4.8312 (1.0) 4.8312 (1.0) 4.8312 (1.0) 0.0000 (1.0) 4.8312 (1.0) 0.0000 (1.0) 0;0 0.2070 (1.0) 1 1
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0002_2.6.0) 5.3339 (1.10) 5.3339 (1.10) 5.3339 (1.10) 0.0000 (1.0) 5.3339 (1.10) 0.0000 (1.0) 0;0 0.1875 (0.91) 1 1
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0003_2.5.4) 44.7231 (9.26) 44.7231 (9.26) 44.7231 (9.26) 0.0000 (1.0) 44.7231 (9.26) 0.0000 (1.0) 0;0 0.0224 (0.11) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_help---help': 3 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0001_2.6.3) 341.1441 (1.66) 341.1441 (1.66) 341.1441 (1.66) 0.0000 (1.0) 341.1441 (1.66) 0.0000 (1.0) 0;0 2.9313 (0.60) 1 1
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0002_2.6.0) 301.4364 (1.47) 301.4364 (1.47) 301.4364 (1.47) 0.0000 (1.0) 301.4364 (1.47) 0.0000 (1.0) 0;0 3.3174 (0.68) 1 1
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0003_2.5.4) 204.9798 (1.0) 204.9798 (1.0) 204.9798 (1.0) 0.0000 (1.0) 204.9798 (1.0) 0.0000 (1.0) 0;0 4.8785 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_import-import': 3 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0001_2.6.3) 14.4332 (1.57) 14.4332 (1.57) 14.4332 (1.57) 0.0000 (1.0) 14.4332 (1.57) 0.0000 (1.0) 0;0 0.0693 (0.64) 1 1
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0002_2.6.0) 15.1159 (1.65) 15.1159 (1.65) 15.1159 (1.65) 0.0000 (1.0) 15.1159 (1.65) 0.0000 (1.0) 0;0 0.0662 (0.61) 1 1
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0003_2.5.4) 9.1706 (1.0) 9.1706 (1.0) 9.1706 (1.0) 0.0000 (1.0) 9.1706 (1.0) 0.0000 (1.0) 0;0 0.1090 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_init-init': 3 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0001_2.6.3) 516.5441 (1.11) 516.5441 (1.11) 516.5441 (1.11) 0.0000 (1.0) 516.5441 (1.11) 0.0000 (1.0) 0;0 1.9359 (0.90) 1 1
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0002_2.6.0) 465.9558 (1.0) 465.9558 (1.0) 465.9558 (1.0) 0.0000 (1.0) 465.9558 (1.0) 0.0000 (1.0) 0;0 2.1461 (1.0) 1 1
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0003_2.5.4) 511.2961 (1.10) 511.2961 (1.10) 511.2961 (1.10) 0.0000 (1.0) 511.2961 (1.10) 0.0000 (1.0) 0;0 1.9558 (0.91) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_pull-pull': 3 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0001_2.6.3) 7.3708 (1.03) 7.3708 (1.03) 7.3708 (1.03) 0.0000 (1.0) 7.3708 (1.03) 0.0000 (1.0) 0;0 0.1357 (0.97) 1 1
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0002_2.6.0) 7.4631 (1.04) 7.4631 (1.04) 7.4631 (1.04) 0.0000 (1.0) 7.4631 (1.04) 0.0000 (1.0) 0;0 0.1340 (0.96) 1 1
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0003_2.5.4) 7.1801 (1.0) 7.1801 (1.0) 7.1801 (1.0) 0.0000 (1.0) 7.1801 (1.0) 0.0000 (1.0) 0;0 0.1393 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_push-push': 3 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0001_2.6.3) 6.3136 (1.05) 6.3136 (1.05) 6.3136 (1.05) 0.0000 (1.0) 6.3136 (1.05) 0.0000 (1.0) 0;0 0.1584 (0.96) 1 1
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0002_2.6.0) 6.1651 (1.02) 6.1651 (1.02) 6.1651 (1.02) 0.0000 (1.0) 6.1651 (1.02) 0.0000 (1.0) 0;0 0.1622 (0.98) 1 1
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0003_2.5.4) 6.0355 (1.0) 6.0355 (1.0) 6.0355 (1.0) 0.0000 (1.0) 6.0355 (1.0) 0.0000 (1.0) 0;0 0.1657 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-add': 3 tests --------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 11.9443 (1.49) 11.9443 (1.49) 11.9443 (1.49) 0.0000 (1.0) 11.9443 (1.49) 0.0000 (1.0) 0;0 0.0837 (0.67) 1 1
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 11.8710 (1.49) 11.8710 (1.49) 11.8710 (1.49) 0.0000 (1.0) 11.8710 (1.49) 0.0000 (1.0) 0;0 0.0842 (0.67) 1 1
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 7.9929 (1.0) 7.9929 (1.0) 7.9929 (1.0) 0.0000 (1.0) 7.9929 (1.0) 0.0000 (1.0) 0;0 0.1251 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 3 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 1.2143 (1.02) 1.2143 (1.02) 1.2143 (1.02) 0.0000 (1.0) 1.2143 (1.02) 0.0000 (1.0) 0;0 0.8235 (0.98) 1 1
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 1.1880 (1.0) 1.1880 (1.0) 1.1880 (1.0) 0.0000 (1.0) 1.1880 (1.0) 0.0000 (1.0) 0;0 0.8418 (1.0) 1 1
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 1.3034 (1.10) 1.3034 (1.10) 1.3034 (1.10) 0.0000 (1.0) 1.3034 (1.10) 0.0000 (1.0) 0;0 0.7672 (0.91) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 3 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 517.9664 (1.00) 517.9664 (1.00) 517.9664 (1.00) 0.0000 (1.0) 517.9664 (1.00) 0.0000 (1.0) 0;0 1.9306 (1.00) 1 1
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 517.2653 (1.0) 517.2653 (1.0) 517.2653 (1.0) 0.0000 (1.0) 517.2653 (1.0) 0.0000 (1.0) 0;0 1.9332 (1.0) 1 1
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 539.5161 (1.04) 539.5161 (1.04) 539.5161 (1.04) 0.0000 (1.0) 539.5161 (1.04) 0.0000 (1.0) 0;0 1.8535 (0.96) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 3 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 9.4405 (1.14) 9.4405 (1.14) 9.4405 (1.14) 0.0000 (1.0) 9.4405 (1.14) 0.0000 (1.0) 0;0 0.1059 (0.88) 1 1
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 9.6656 (1.16) 9.6656 (1.16) 9.6656 (1.16) 0.0000 (1.0) 9.6656 (1.16) 0.0000 (1.0) 0;0 0.1035 (0.86) 1 1
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 8.3069 (1.0) 8.3069 (1.0) 8.3069 (1.0) 0.0000 (1.0) 8.3069 (1.0) 0.0000 (1.0) 0;0 0.1204 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 3 tests --------------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 558.2692 (1.0) 558.2692 (1.0) 558.2692 (1.0) 0.0000 (1.0) 558.2692 (1.0) 0.0000 (1.0) 0;0 1.7913 (1.0) 1 1
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 573.7619 (1.03) 573.7619 (1.03) 573.7619 (1.03) 0.0000 (1.0) 573.7619 (1.03) 0.0000 (1.0) 0;0 1.7429 (0.97) 1 1
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 1,587.4082 (2.84) 1,587.4082 (2.84) 1,587.4082 (2.84) 0.0000 (1.0) 1,587.4082 (2.84) 0.0000 (1.0) 0;0 0.6300 (0.35) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 3 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 4.9745 (1.14) 4.9745 (1.14) 4.9745 (1.14) 0.0000 (1.0) 4.9745 (1.14) 0.0000 (1.0) 0;0 0.2010 (0.88) 1 1
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 5.3639 (1.23) 5.3639 (1.23) 5.3639 (1.23) 0.0000 (1.0) 5.3639 (1.23) 0.0000 (1.0) 0;0 0.1864 (0.81) 1 1
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 4.3571 (1.0) 4.3571 (1.0) 4.3571 (1.0) 0.0000 (1.0) 4.3571 (1.0) 0.0000 (1.0) 0;0 0.2295 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 3 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 554.1255 (1.16) 554.1255 (1.16) 554.1255 (1.16) 0.0000 (1.0) 554.1255 (1.16) 0.0000 (1.0) 0;0 1.8046 (0.86) 1 1
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 531.6801 (1.12) 531.6801 (1.12) 531.6801 (1.12) 0.0000 (1.0) 531.6801 (1.12) 0.0000 (1.0) 0;0 1.8808 (0.90) 1 1
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 476.1696 (1.0) 476.1696 (1.0) 476.1696 (1.0) 0.0000 (1.0) 476.1696 (1.0) 0.0000 (1.0) 0;0 2.1001 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_status-status': 3 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0001_2.6.3) 4.4081 (1.39) 4.4081 (1.39) 4.4081 (1.39) 0.0000 (1.0) 4.4081 (1.39) 0.0000 (1.0) 0;0 0.2269 (0.72) 1 1
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0002_2.6.0) 4.3641 (1.37) 4.3641 (1.37) 4.3641 (1.37) 0.0000 (1.0) 4.3641 (1.37) 0.0000 (1.0) 0;0 0.2291 (0.73) 1 1
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0003_2.5.4) 3.1779 (1.0) 3.1779 (1.0) 3.1779 (1.0) 0.0000 (1.0) 3.1779 (1.0) 0.0000 (1.0) 0;0 0.3147 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 18, 2021 

--------------------------------------------------------------------------------------------------------------------- benchmark 'test_add-add': 3 tests ----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0001_2.6.3) 8.4396 (1.39) 8.4396 (1.39) 8.4396 (1.39) 0.0000 (1.0) 8.4396 (1.39) 0.0000 (1.0) 0;0 0.1185 (0.72) 1 1
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 8.3386 (1.38) 8.3386 (1.38) 8.3386 (1.38) 0.0000 (1.0) 8.3386 (1.38) 0.0000 (1.0) 0;0 0.1199 (0.73) 1 1
test_add-add (benchmarks-tests-benchmarks-cli-commands-test_add.py-test_add/Linux-CPython-3.7-64bit/0003_2.5.4) 6.0531 (1.0) 6.0531 (1.0) 6.0531 (1.0) 0.0000 (1.0) 6.0531 (1.0) 0.0000 (1.0) 0;0 0.1652 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_add-diff': 3 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0001_2.6.3) 3.7265 (1.03) 3.7265 (1.03) 3.7265 (1.03) 0.0000 (1.0) 3.7265 (1.03) 0.0000 (1.0) 0;0 0.2683 (0.97) 1 1
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0002_2.6.0) 3.6222 (1.0) 3.6222 (1.0) 3.6222 (1.0) 0.0000 (1.0) 3.6222 (1.0) 0.0000 (1.0) 0;0 0.2761 (1.0) 1 1
test_add-diff (benchmarks-tests-benchmarks-cli-commands-test_diff.py-test_add/Linux-CPython-3.7-64bit/0003_2.5.4) 42.8081 (11.82) 42.8081 (11.82) 42.8081 (11.82) 0.0000 (1.0) 42.8081 (11.82) 0.0000 (1.0) 0;0 0.0234 (0.08) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 3 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0001_2.6.3) 2.8714 (1.06) 2.8714 (1.06) 2.8714 (1.06) 0.0000 (1.0) 2.8714 (1.06) 0.0000 (1.0) 0;0 0.3483 (0.95) 1 1
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0002_2.6.0) 2.7208 (1.0) 2.7208 (1.0) 2.7208 (1.0) 0.0000 (1.0) 2.7208 (1.0) 0.0000 (1.0) 0;0 0.3675 (1.0) 1 1
test_checkout-checkout (benchmarks-tests-benchmarks-cli-commands-test_checkout.py-test_checkout/Linux-CPython-3.7-64bit/0003_2.5.4) 2.8190 (1.04) 2.8190 (1.04) 2.8190 (1.04) 0.0000 (1.0) 2.8190 (1.04) 0.0000 (1.0) 0;0 0.3547 (0.97) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_gc-gc': 3 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0001_2.6.3) 624.8847 (1.06) 624.8847 (1.06) 624.8847 (1.06) 0.0000 (1.0) 624.8847 (1.06) 0.0000 (1.0) 0;0 1.6003 (0.95) 1 1
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0002_2.6.0) 591.1935 (1.0) 591.1935 (1.0) 591.1935 (1.0) 0.0000 (1.0) 591.1935 (1.0) 0.0000 (1.0) 0;0 1.6915 (1.0) 1 1
test_gc-gc (benchmarks-tests-benchmarks-cli-commands-test_gc.py-test_gc/Linux-CPython-3.7-64bit/0003_2.5.4) 610.4666 (1.03) 610.4666 (1.03) 610.4666 (1.03) 0.0000 (1.0) 610.4666 (1.03) 0.0000 (1.0) 0;0 1.6381 (0.97) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------- benchmark 'test_get-get': 3 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0001_2.6.3) 4.1428 (1.01) 4.1428 (1.01) 4.1428 (1.01) 0.0000 (1.0) 4.1428 (1.01) 0.0000 (1.0) 0;0 0.2414 (0.99) 1 1
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0002_2.6.0) 4.1145 (1.0) 4.1145 (1.0) 4.1145 (1.0) 0.0000 (1.0) 4.1145 (1.0) 0.0000 (1.0) 0;0 0.2430 (1.0) 1 1
test_get-get (benchmarks-tests-benchmarks-cli-commands-test_get.py-test_get/Linux-CPython-3.7-64bit/0003_2.5.4) 43.2549 (10.51) 43.2549 (10.51) 43.2549 (10.51) 0.0000 (1.0) 43.2549 (10.51) 0.0000 (1.0) 0;0 0.0231 (0.10) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------------- benchmark 'test_help---help': 3 tests ----------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0001_2.6.3) 305.7810 (1.58) 378.8344 (1.64) 324.8532 (1.55) 21.6970 (2.89) 316.6192 (1.54) 13.0295 (1.86) 1;1 3.0783 (0.64) 10 1
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0002_2.6.0) 288.1431 (1.49) 311.9795 (1.35) 297.2002 (1.42) 7.5201 (1.0) 295.6004 (1.43) 7.0071 (1.0) 3;1 3.3647 (0.70) 10 1
test_help---help (benchmarks-tests-benchmarks-cli-commands-test_help.py-test_help/Linux-CPython-3.7-64bit/0003_2.5.4) 193.8939 (1.0) 231.5034 (1.0) 209.0508 (1.0) 12.7794 (1.70) 206.0559 (1.0) 16.5650 (2.36) 4;0 4.7835 (1.0) 10 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_import-import': 3 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0001_2.6.3) 14.8759 (1.59) 14.8759 (1.59) 14.8759 (1.59) 0.0000 (1.0) 14.8759 (1.59) 0.0000 (1.0) 0;0 0.0672 (0.63) 1 1
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0002_2.6.0) 15.1097 (1.61) 15.1097 (1.61) 15.1097 (1.61) 0.0000 (1.0) 15.1097 (1.61) 0.0000 (1.0) 0;0 0.0662 (0.62) 1 1
test_import-import (benchmarks-tests-benchmarks-cli-commands-test_import.py-test_import/Linux-CPython-3.7-64bit/0003_2.5.4) 9.3690 (1.0) 9.3690 (1.0) 9.3690 (1.0) 0.0000 (1.0) 9.3690 (1.0) 0.0000 (1.0) 0;0 0.1067 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------- benchmark 'test_init-init': 3 tests ---------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0001_2.6.3) 513.8111 (1.23) 513.8111 (1.23) 513.8111 (1.23) 0.0000 (1.0) 513.8111 (1.23) 0.0000 (1.0) 0;0 1.9462 (0.81) 1 1
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0002_2.6.0) 416.8148 (1.0) 416.8148 (1.0) 416.8148 (1.0) 0.0000 (1.0) 416.8148 (1.0) 0.0000 (1.0) 0;0 2.3991 (1.0) 1 1
test_init-init (benchmarks-tests-benchmarks-cli-commands-test_init.py-test_init/Linux-CPython-3.7-64bit/0003_2.5.4) 431.2251 (1.03) 431.2251 (1.03) 431.2251 (1.03) 0.0000 (1.0) 431.2251 (1.03) 0.0000 (1.0) 0;0 2.3190 (0.97) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_pull-pull': 3 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0001_2.6.3) 9.1809 (1.13) 9.1809 (1.13) 9.1809 (1.13) 0.0000 (1.0) 9.1809 (1.13) 0.0000 (1.0) 0;0 0.1089 (0.89) 1 1
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0002_2.6.0) 9.0944 (1.12) 9.0944 (1.12) 9.0944 (1.12) 0.0000 (1.0) 9.0944 (1.12) 0.0000 (1.0) 0;0 0.1100 (0.90) 1 1
test_pull-pull (benchmarks-tests-benchmarks-cli-commands-test_pull.py-test_pull/Linux-CPython-3.7-64bit/0003_2.5.4) 8.1514 (1.0) 8.1514 (1.0) 8.1514 (1.0) 0.0000 (1.0) 8.1514 (1.0) 0.0000 (1.0) 0;0 0.1227 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------- benchmark 'test_push-push': 3 tests -----------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0001_2.6.3) 4.8936 (1.05) 4.8936 (1.05) 4.8936 (1.05) 0.0000 (1.0) 4.8936 (1.05) 0.0000 (1.0) 0;0 0.2043 (0.96) 1 1
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0002_2.6.0) 4.8440 (1.04) 4.8440 (1.04) 4.8440 (1.04) 0.0000 (1.0) 4.8440 (1.04) 0.0000 (1.0) 0;0 0.2064 (0.97) 1 1
test_push-push (benchmarks-tests-benchmarks-cli-commands-test_push.py-test_push/Linux-CPython-3.7-64bit/0003_2.5.4) 4.6745 (1.0) 4.6745 (1.0) 4.6745 (1.0) 0.0000 (1.0) 4.6745 (1.0) 0.0000 (1.0) 0;0 0.2139 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-add': 3 tests --------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 11.5279 (1.52) 11.5279 (1.52) 11.5279 (1.52) 0.0000 (1.0) 11.5279 (1.52) 0.0000 (1.0) 0;0 0.0867 (0.66) 1 1
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 11.6686 (1.54) 11.6686 (1.54) 11.6686 (1.54) 0.0000 (1.0) 11.6686 (1.54) 0.0000 (1.0) 0;0 0.0857 (0.65) 1 1
test_sharing-add (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 7.5913 (1.0) 7.5913 (1.0) 7.5913 (1.0) 0.0000 (1.0) 7.5913 (1.0) 0.0000 (1.0) 0;0 0.1317 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 3 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 1.1688 (1.0) 1.1688 (1.0) 1.1688 (1.0) 0.0000 (1.0) 1.1688 (1.0) 0.0000 (1.0) 0;0 0.8556 (1.0) 1 1
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 1.2122 (1.04) 1.2122 (1.04) 1.2122 (1.04) 0.0000 (1.0) 1.2122 (1.04) 0.0000 (1.0) 0;0 0.8249 (0.96) 1 1
test_sharing-add-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 1.2919 (1.11) 1.2919 (1.11) 1.2919 (1.11) 0.0000 (1.0) 1.2919 (1.11) 0.0000 (1.0) 0;0 0.7740 (0.90) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 3 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 535.8935 (1.05) 535.8935 (1.05) 535.8935 (1.05) 0.0000 (1.0) 535.8935 (1.05) 0.0000 (1.0) 0;0 1.8660 (0.95) 1 1
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 517.4070 (1.02) 517.4070 (1.02) 517.4070 (1.02) 0.0000 (1.0) 517.4070 (1.02) 0.0000 (1.0) 0;0 1.9327 (0.99) 1 1
test_sharing-checkout-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 509.7385 (1.0) 509.7385 (1.0) 509.7385 (1.0) 0.0000 (1.0) 509.7385 (1.0) 0.0000 (1.0) 0;0 1.9618 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 3 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 8.8509 (1.08) 8.8509 (1.08) 8.8509 (1.08) 0.0000 (1.0) 8.8509 (1.08) 0.0000 (1.0) 0;0 0.1130 (0.92) 1 1
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 9.1522 (1.12) 9.1522 (1.12) 9.1522 (1.12) 0.0000 (1.0) 9.1522 (1.12) 0.0000 (1.0) 0;0 0.1093 (0.89) 1 1
test_sharing-pull (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 8.1642 (1.0) 8.1642 (1.0) 8.1642 (1.0) 0.0000 (1.0) 8.1642 (1.0) 0.0000 (1.0) 0;0 0.1225 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 3 tests --------------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 539.6776 (1.0) 539.6776 (1.0) 539.6776 (1.0) 0.0000 (1.0) 539.6776 (1.0) 0.0000 (1.0) 0;0 1.8530 (1.0) 1 1
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 561.7202 (1.04) 561.7202 (1.04) 561.7202 (1.04) 0.0000 (1.0) 561.7202 (1.04) 0.0000 (1.0) 0;0 1.7802 (0.96) 1 1
test_sharing-pull-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 1,152.7430 (2.14) 1,152.7430 (2.14) 1,152.7430 (2.14) 0.0000 (1.0) 1,152.7430 (2.14) 0.0000 (1.0) 0;0 0.8675 (0.47) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 3 tests ------------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 4.9177 (1.17) 4.9177 (1.17) 4.9177 (1.17) 0.0000 (1.0) 4.9177 (1.17) 0.0000 (1.0) 0;0 0.2033 (0.86) 1 1
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 5.3201 (1.26) 5.3201 (1.26) 5.3201 (1.26) 0.0000 (1.0) 5.3201 (1.26) 0.0000 (1.0) 0;0 0.1880 (0.79) 1 1
test_sharing-push (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 4.2183 (1.0) 4.2183 (1.0) 4.2183 (1.0) 0.0000 (1.0) 4.2183 (1.0) 0.0000 (1.0) 0;0 0.2371 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 3 tests ----------------------------------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0001_2.6.3) 523.7239 (1.18) 523.7239 (1.18) 523.7239 (1.18) 0.0000 (1.0) 523.7239 (1.18) 0.0000 (1.0) 0;0 1.9094 (0.85) 1 1
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0002_2.6.0) 555.1034 (1.25) 555.1034 (1.25) 555.1034 (1.25) 0.0000 (1.0) 555.1034 (1.25) 0.0000 (1.0) 0;0 1.8015 (0.80) 1 1
test_sharing-push-noop (benchmarks-tests-benchmarks-cli-stories-use-cases-test_sharing.py-test_sharing/Linux-CPython-3.7-64bit/0003_2.5.4) 445.1812 (1.0) 445.1812 (1.0) 445.1812 (1.0) 0.0000 (1.0) 445.1812 (1.0) 0.0000 (1.0) 0;0 2.2463 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------ benchmark 'test_status-status': 3 tests -------------------------------------------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0001_2.6.3) 4.7250 (1.91) 4.7250 (1.91) 4.7250 (1.91) 0.0000 (1.0) 4.7250 (1.91) 0.0000 (1.0) 0;0 0.2116 (0.52) 1 1
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0002_2.6.0) 4.6857 (1.89) 4.6857 (1.89) 4.6857 (1.89) 0.0000 (1.0) 4.6857 (1.89) 0.0000 (1.0) 0;0 0.2134 (0.53) 1 1
test_status-status (benchmarks-tests-benchmarks-cli-commands-test_status.py-test_status/Linux-CPython-3.7-64bit/0003_2.5.4) 2.4769 (1.0) 2.4769 (1.0) 2.4769 (1.0) 0.0000 (1.0) 2.4769 (1.0) 0.0000 (1.0) 0;0 0.4037 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```

@efiop efiop changed the title [WIP] dvc-bench: rework our framework dvc-bench: rework our framework Aug 18, 2021
@efiop
Copy link
Copy Markdown
Contributor Author

efiop commented Aug 18, 2021

Ok, let's use it in master for now. There are a few todo's left, but the framework is working and showing the results for main commands, so we can already use for tracking regressions. Remotes and the rest of todos are coming in follow-ups.

@efiop efiop merged commit 60431f9 into master Aug 18, 2021
@pared pared mentioned this pull request Sep 20, 2021
Closed
This was referenced Dec 9, 2021
Closed
Closed
Closed
Closed
@efiop efiop deleted the pytest branch July 1, 2022 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@efiop