New check: Report /usr/bin/python invocations during rpmbuild

[hroncok]

This will be a tricky one.

[hroncok]

One idea we discussed with @encukou IRL:

Change Python 2 to log it's invocation to some dedicated file if RPM_BUILD_ROOT is set and Python was invoked as /usr/bin/python. Change some RPM macro to dump this file into the logs at the end of rpmbuild (if it exists). Provide a way to opt out of this in case it's breaking some scenarios (possibly with another env var).

Then, parse the logs from taskotron task.

This can be announced and later changed in a way that invoking /usr/bin/python during rpmbuild will fail immediately (although I realize there will be voices against that saying we are breaking their workflows).

[hroncok]

I've been thinking about this for a while and I consider this:

Logging the invocation to stderr immediatelly

• 👍 should be basically a one liner
• 👍 would be very easy to understand, if it says something like: DEPRECATION WARNING: python2 invoked with /usr/bin/python, use /usr/bin/python3 or /usr/bin/python2, /usr/bin/python will be removed or switched to Python 3 in the future. Set the WHATNOT environment variable to supress this warning if it breaks your build. Using Python 2 for now. Full invocation: python -m spam -o eggs
• 👎 there might be situations where the stderr is not displayed in the logs (e.g. python is invoked withing a subprocess and the stderr is consumed by the app, or plain simple redirect to /dev/null in a build script)

Logging the invocation to a file and dumping the file to the logs at the end

• 👍 covers the cases where stderr would be normally hidden
• 👎 brings much higher complexity
• 👎 is more error prone
• 👎 may be confusing for the packager

That said, I prefer the easier option. Opinions?

[encukou]

+1; this gives us a first-order approximation of how much of a problem this is. Let's fix everything the easy option finds, and then disable /usr/bin/python in RPMBUILD to find & fix the rest.

If we provide a workaround, we should also track who uses it. Instead of telling users to set an env variable, let's link to a Wiki page that says to set the variable and file a bug we can track (and help with)?

I don't think this needs a Change, does it? We just announce/discuss on fedora-devel & python-devel?

---

Note: there is an entirely separate problem that this issue doesn't (and shouldn't) address: figuring out if python2 is needed to build python3 subackages. The solution there might be similar, and if it is, we'd preferably do both in the same update to prevent churn.
But that's not necessary, just something to think about.

[hroncok]

I don't think this needs a Change, does it? We just announce/discuss on fedora-devel & python-devel?

It can be part of phase 1 https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3 (that phase itself might end up either with change or without it). But other than that announce/discuss on fedora-devel & python-devel should be enough IMHO.

If we provide a workaround, we should also track who uses it. Instead of telling users to set an env variable, let's link to a Wiki page that says to set the variable and file a bug we can track (and help with)?

Having a wikipage we might as well have a wiki page with change :)

Nevertheless, linking to docs about this that say more is a good idea. Tracking this is a good idea as well; my original tought was that we can easily grep for the envvar in dist-git. (Hoping that there will not by a crazy case where somebody workarounds this by pushing the envvar to the upstream build script.)

The solution there might be similar, and if it is, we'd preferably do both in the same update to prevent churn.

I don't think that the solution will be similar (but it might be). At least it will be much more complicated to figure out what to do and how. I rather do what this issue is about sooner to gather the feedback and learn from any potential mistakes, later propose a solution for the second problem.

[encukou]

OK, looks like it's not a Change, but a Change-like page (with contact details, nice description, etc.) would be nice. I should get around to making one this week, if you don't beat me to it.

Simple +1 for the rest :)

[irushchyshyn]

It can be part of phase 1 https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3 (that phase itself might end up either with change or without it).

It does sound as a part pf Phase 1. I have started to draft a change for phase 1 here: https://fedoraproject.org/wiki/User:Ishcherb/Phase1, we can add info on the /usr/bin/python invocations there as well.

[hroncok]

Note to self:

similar deprecation warnings might be needed for %{python_sitelib} etc

Edit: Doh, they all use %{__python} and that one is /usr/bin/python. So as long as we make it clear in the warning / linked page, the warning should propagate there as well.

[encukou]

The Change-like document is here: https://fedoraproject.org/wiki/User:Pviktori/Avoid_usr_bin_python_in_RPM_Build

Next time I'll get to spend significant time on this will be late next week. If anyone gets to it before that, please:

• proofread
• move to an official namespace
• post to python-devel
• post to fedora-devel (posted as a change)

[hroncok]

I've done some tweaks. Also discussing it with @ignatenkobrain and @keszybz (zbyszek) on IRC to gather early feedback.

[hroncok]

If anyone gets the same idea, JFYI: I'll prepare a skeleton for a task that parses the build.log.