uv LBS

[ggalloni]

This PR is a first attempt at using uv instead of poetry to manage litebird_sim.

Copilot gave a huge hand in setting this up, but probably this needs some refinement.

I think this could be a good place to test the performance of tests using uv and eventually leverage it to add more workflows. I am thinking of an automatic Build&Publish workflow, for example.

[github-actions]

Coverage report

This PR does not seem to contain any modification to coverable code.

[ziotom78]

Great work, @ggalloni! I tested the code on my Linux machine and everything works fine. I made a few minor changes, but I think the PR is ready for review by others. It would be good if a couple more people could test it, since the workflow change might not be smooth for everyone.

[ggalloni]

Yeah, that would be very useful!

Also, I would like to have a look at the current workflows to check whether they can be optimized with uv.

[ggalloni]

@ziotom78, while working on this, I found some issues with the caching system of PySM, so I worked a bit on that. Essentially, there was a typo in the cache hit variable, which is now solved (cache_hit -> cache-hit), which saves O(10 sec) per run if PySM data are already present.

Triggered by that, I added a cache of the venv created by uv. This sped up the installation steps.
Compared with another commit of this PR, before reworking the caching, I get these improvements:

Build	Basic deps	litebird_sim deps
ubuntu-none-3.10	~1 min	~2 sec
ubuntu-none-3.13	~2 min	~3 min
ubuntu-openmpi-3.10	~20 sec	~1 min
ubuntu-openmpi-3.13	~7 sec	~3 min
macOS-none-3.10	~2 sec	~10 sec
macOS-none-3.13	~0 sec	~3 min
macOS-openmpi-3.10	~6 sec	~10 sec
macOS-openmpi-3.13	~10 sec	~4 min

Instead, comparing with the latest master builds, so summing the effect of the new cache and uv I get:

Build	Basic deps	litebird_sim deps
ubuntu-none-3.10	~1 min	~30 sec
ubuntu-none-3.13	~ -1 min	~3.5 min
ubuntu-openmpi-3.10	~2 min	~1 min
ubuntu-openmpi-3.13	~7 sec	~3.5 min
macOS-none-3.10	~2 sec	~50 sec
macOS-none-3.13	~10 sec	~4 min
macOS-openmpi-3.10	~40 sec	~3 sec
macOS-openmpi-3.13	~10 sec	~4 min

Of course, these numbers oscillate a lot from run to run, so they should be taken with a grain of salt. Still, they show a clear improvement in optimizing our CI tests as you suggested. Also, there is a single negative number, which was probably just a bad oscillation.

So I am adding two new entries in the CHANGELOG to keep track of both (uv and the improved caching system).

[ggalloni]

@ziotom78, I merged the new actions in here.
Should we wait for some more feedback on the new workflow?

If not, and if you're OK with this, I'll merge everything into master.

[ziotom78]

@ziotom78, I merged the new actions in here. Should we wait for some more feedback on the new workflow?

If not, and if you're OK with this, I'll merge everything into master.

I do not expect much feedback: it has been almost two weeks since we mentioned this PR at the last telecon. I think it’s ok to merge it.

Many thanks!