Explanation of performance & upatreaming

[adriangb]

From the benchmarks it looks like PDM performs much better than Poetry. Would it be possible to get a blog post or something summarizing where the performance improvement are coming from?

It would also be really cool if some of it could be ported over to Poetry (which is not going away anytime soon).

[frostming]

There is a post: https://frostming.com/2021/03-26/pm-review-2021/#performance
But I am not sure whether it covers your concern.

BTW may I know what benchmark result are you referring to?

[frostming]

Here are some efforts that PDM takes to improve the performance, I am not clear of Poetry though.

1. Cache, HTTP requests, package dependencies, and wheel builds are all cached
2. Package hashes are extracted from the URL fragment without actually downloading them
3. Information in the lock file is used in dependency resolution to avoid unnecessary HTTP requests
4. The solver is different, which is the same as used by pip
5. Be aware that the performance testing in the post also includes installation. PDM uses an extremely simple installer and runs in parallel, which also contributes to the result.

[adriangb]

Very interesting. The use of info from URLs sounds like a smart move.

I now wonder how those numbers would look now that poetry has a parallelized resolver as well. So you by any chance have the source for your benchmarks? It might be interesting to put it in a /benchmarks directory for transparency and knowledge sharing.

[frostming]

From rough testing, it seems the latest Poetry(1.1.12) doesn't improve. I will paste the result and source code later #794

[frostming]

Here it is! https://pdm.fming.dev/dev/benchmark/

[adriangb]

Awesome, thank you for getting through that so quickly! It's interesting that there isn't much of a change for Poetry, I thought their new installer might make a difference.