Compare all operations with cold/warm cache

[radoering]

At the moment the comparison cold cache vs. warm cache is only made for Install but it might be interesting for all commands. (During development you can often assume a partially filled cache.)

[ipmb]

My assumption is that when you are doing an update, add, or lock, you've already installed the project and its dependencies and are working from a warm cache. There may be scenarios where that isn't the case, but it seems like an edge case to me.

[radoering]

Then it seems to me that your assumption does not match the implementation:

python-package-manager-shootout/.github/workflows/benchmark.yml

Lines 37 to 43 in 738f868

	- name: lock
	run: |
	make poetry-clean-cache
	make poetry-clean-venv
	make poetry-clean-lock
	/usr/bin/time --output=timings/lock.txt --format="%e,%S,%U,%P,%M,%I,%O" \
	make poetry-lock

Or am I misunderstanding this?

[ipmb]

🤔 hmm, good point. I'll consider using a warm cache for this.

[dimbleby]

The poetry-update case also shows poetry in the case where its cache is not populated. I'm not sure whether this is what's intended or not.

Although the 'update' follows both a cold and warm install, for poetry at least that will only have populated the caches relating to installation (ie it will have downloaded the wheels and source distributions for packages). This will not have allowed poetry to re-populate the separate cache that it uses in locking (which holds information about dependencies between packages).

If you were to do a 'warm update' after the existing 'update', it would be much faster - comparable, I expect, to what you are currently showing at 'add package'.

(Indeed that's why 'add package' is apparently so much faster than 'update' - it is taking advantage of the cache as populated during the 'update' test).

Benchmarking is hard, I expect if any of the other package managers looked at this they could make similar comments...

[matteius]

Something I've been wondering about ...

Indeed that's why 'add package' is apparently so much faster than 'update' - it is taking advantage of the cache as populated during the 'update' test

I am confused on this point, because the current graphs show that poetry's add package is significantly faster than poetry's re-lock but the add package says "This test measures the time to add a new package and regeneration the lock file." -- I am confused on how could it be that its faster to add a package and regenerate the lock than it is just to regenerate the lock?

[dimbleby]

I am confused on how could it be that its faster to add a package and regenerate the lock than it is just to regenerate the lock?

because - per my last - the poetry cache relating to package dependencies is not available at "update", but it is available at "add package".

[ipmb]

Fixed in #15