chore(python): ensure that make requirements fully refreshes unpinned packages/deps

[alexander-beedie]

Until running the equivalent commands I was getting some mysterious new typing errors from pre-commit checks that a vanilla make requirements did not fix. Have confirmed locally that this properly refreshes unpinned packages/deps, ensuring that your local env will match what runs in CI (which does a fresh install).

If preferred, could make this a separate option, say make requirements-clean? (Though given that, post initial install, you're likely to want it to be thorough I'm not sure there's any real value in two options - I'd say just make this one robust by default unless there's a compelling reason otherwise 🤔).

[stinodego]

I like the idea of make requirements upgrading packages to the latest unpinned versions. But isn't just adding --upgrade enough to achieve that? In other words, why would we need --force-reinstall and --no-deps here?

[alexander-beedie]

I like the idea of make requirements upgrading packages to the latest unpinned versions. But isn't just adding --upgrade enough to achieve that? In other words, why would we need --force-reinstall and --no-deps here?

Observationally --upgrade by itself didn't seem to do the trick; as for --no-deps that was something I was trying locally and didn't mean to include - removing! 😅 (It seemed like we could have a slightly smaller set of required packages in CI by omitting the deps; it works, but could bite us later for only marginal gain).

[stinodego]

Observationally --upgrade by itself didn't seem to do the trick

This really puzzles me, how could that be?

I'd rather not have --force-reinstall in there - seems to me it's just wasting compute and the developers time. If the dependencies are up to date, leave them be.

[alexander-beedie]

Well, we can try it without; maybe you'll have more luck than me. Perhaps it does a better job resolving from scratch after having uninstalled everything, or it had somehow got in a bad state? 🤷 I'll take it out for now and see how it goes - and/or we could add the extra flag as make requirements-clean in case the nuclear option is required?

[stinodego]

Well, we can try it without; maybe you'll have more luck than me. Perhaps it does a better job resolving from scratch after having uninstalled everything, or it had somehow got in a bad state? I'll take it out for now and see how it goes

Indeed let's do that. If this for some reason does not do the trick, get a pip list of the problematic environment and we can see what's really going on. We can always add --force-reinstall later if it turns out to be needed in some edge case.

we could add the extra flag as make requirements-clean in case the nuclear option is required?

We already have a nuclear option in make clean && make build, so I don't think that will be necessary.