Python 2: Transition programs to Python 3

[jefferyto]

This is a meta-issue to track progress for transitioning programs that use Python 2 to Python 3.

These should be packages that use Python in some way, as part of their core functionality. Packages that provide a Python 2-only library or plugin will be tracked in a separate issue (TBD).

Section	Package	Status	Maintainers	Related issues/PRs
devel	asu	⭐ Done	@aparcar	#10976
lang	node	⭐ Done	@nxhack	#10179
lang	node-mozilla-iot-gateway	✅ Resolved	@ratkaj	comment, #9670
mail	mailman		@Shulyaka
net	aircrack-ng	⭐ Done	@jbenden	comment, #11145
net	danish	🚫 Won't Fix	@smutt	comment
net	iotivity	✅ Resolved	@hauke	#11587
net	nmap	⭐ Done	@nunojpg	#10162, #11605
net	obfsproxy	🚫 Won't Fix	@jefferyto	comment
net	openvswitch	⭐ Done	@yousong	#8797
net	radicale	⭐ Done		comment, #9082
net	samba4	⭐ Done	@Andy2244	#10707, #10781
net	scapy	⭐ Done	@MikePetullo	#8824
net	seafile-ccnet	⭐ Done	@commodo	#11565
net	seafile-seahub	⭐ Done	@commodo	#11565
net	seafile-server	⭐ Done	@commodo	#11565
net	sysrepo	⭐ Done	@mislavn	#9491, #10554
net	uwsgi-cgi	⭐ Done	@Ansuel	#9843
sound	upmpdcli	⭐ Done	@ignisf	comment, #9687
utils	bluez	⭐ Done		#9080, #9094
utils	gammu	⭐ Done	@aTanW	#10186, #10171
utils	hplip	⭐ Done	@luizluca	#9323
utils	yunbridge		@blogic	comment

Maintainers: If your program provides a Python 2 library package, please be reminded to keep it around (if possible) until the mass removal in 2020.

Please comment on / reference this issue as things develop. I will keep this post updated "periodically" 😂

[neheb]

Nice! Want this pinned?

[neheb]

Maintainer of aircrack-ng is @jbenden

[jefferyto]

@neheb

Nice! Want this pinned?

Sure 😆

Maintainer of aircrack-ng is @jbenden

Updated - thanks!

[jefferyto]

Added devel/asu (ping @aparcar) because while the server itself uses Python 3, it also wants to setup an imagebuilder environment on target and so is pulling in Python 2. (This is mostly speculation though, I haven't looked too deeply into the details.)

If so, sounds like asu won't be resolved until OpenWrt core has moved to Python 3.

[smutt]

I'm currently rewriting Danish in Rust so there will be no Python 3 version.
https://github.com/smutt/danish-rust

I hope to be done with it by December and I'll call it Danish 2.0. The Python version will be abandoned and only the Rust version will be supported going forward.

The current Python 2 version worked fine for its purposes, and I've been running it for over a year on my home OpenWRT box. But it was an experiment in doing middle-box DANE for HTTPS that I now want to get more serious about and also bring to a more general Linux audience, not just OpenWRT. And for that I needed to re-implement in something like Rust or C/C++.

[jefferyto]

@smutt No problem - thanks for the update 😄

[Andy2244]

Samba4.10.x will only fully support python3 from here on out, so if i get over this issue #8944 we will be fine.

[jefferyto]

@Andy2244 Thanks for the update!

[aparcar]

asu is python3 only, @dangowrt did the porting for various dependencies, however only Python3 if I recall correctly...

[jefferyto]

@aparcar So you're saying the +python-light in these lines (labelled IMAGEBUILDER_DEPENDS) should be Python 3 and not Python 2?

[aparcar]

Good point, I'm not sure if the ImageBuilder needs python2 or can handle also 3, the docs require python...

@dangowrt do you know if the ImageBuilder itself is already Python3 compatible? If not, that's kind of a new issue...

[danielfdickinson]

radicale 1.x is never going to Python3 -- radicale2 replaces radicale package and requires Python 3 -- the reason for renaming and having both in tree was to give current radicale users a chance to transition to radicale2 (requires a database conversion, which I think has to be done from radicale 1.x later version).

[jefferyto]

@cshoredaniel Thanks for the update - I guess I have two questions:

1. Is there any reason why a radicale 1.x user would want to stick with 1.x and not upgrade to 2.x?

2. Is there a way we can prompt 1.x users to upgrade to 2.x, before the 1.x package is gone (and they potentially lose the ability to convert their database)? Maybe log a warning to syslog on every startup? 😆

[jefferyto]

@cshoredaniel Wait, sorry I just checked the radicale package and there are Py2 and Py3 versions of the package?

If this is true, we can just remove the Py2 version and keep the Py3 version... But I guess the question still is, should we keep 1.x around?

[ignisf]

upmpdcli's songcast web interface should be compatible with python3. Will take a look at it and update.

[danielfdickinson]

@jefferyto Oh, you're right, I forgot there is a Py3 radicale, just not a Py2 radicale2...I think we should keep radicale 1.x for 19.0x but have a deprecation alert (actually there might already be such -- it's been a while since I last looked and I might be confusing with another distro).

[jefferyto]

@ignisf Thanks for the update!

[luizluca]

For hplip, it should be compatible with python3 since 3.15.2. I'll test, commit and report.

[luizluca]

@jefferyto , hplip is clean. It didn't actually need python for OpenWrt usage.

[jefferyto]

@luizluca thanks for the update!

[jefferyto]

Yes, still happening 😂 I'm not sure if it will be 31 Mar or 1 Apr, but basically the plan is:

• Open several giant PRs (one at a time) to remove packages, each one removing packages with no Python 2 dependants (including the programs listed in this issue that are marked Won't Fix or appear to be abandoned by their maintainers, i.e. mailman, iovitity, yunbridge)
• At the same time, opening PRs in openwrt/packages-abandoned to add those packages (but under a python2 namespace/prefix, to avoid conflicting with python which we may decide to use again here)

AFAIK nmap is the only outstanding package that needs work (#10162), I'm going to try to get to it soon.

[jefferyto]

A brief update: I'm still working on the PR (I think one giant PR will do), but it may take a day or two more than expected (due to a combination of large number of packages that need editing and compile testing, and laziness 😂 )

[hnyman]

Might be better to do it in 2-3 phases, so that there would be less testing for circleci at the same time.

If you have packages ready for the action, just submit the first batch.

Giant PRs sound always dangerous ;-)

[jefferyto]

@hnyman You're right - funny how something can seem like a good idea until you say it out loud 😂

[jefferyto]

The transition for programs is complete so I'll close this issue. Thanks to everyone for your kind assistance 🙏 😂

[jefferyto]

Hmm I guess closed issues can stay pinned. @neheb @hnyman can you help unpin this issue?

[hnyman]

We could leave it pinned for a while.
I am quite sure that there will be some questions, as not all casual users have been aware of the removal plan.
Keeping it pinned for a few weeks might reduce unnecessary issues being opened.

[jefferyto]

Sure - sounds good 😂

[hnyman]

Apparently no major breakage from the python2 removal yesterday.

I looked at the faillogs of the first package architectures that buildbot has built after the removal (e.g. mips_24kc): https://downloads.openwrt.org/snapshots/faillogs/mips_24kc/

There seems to be only some breakage in the telephony feed:

• kamailio-5.x, maintained by @jslachta and @micmac1 , apparently still tries to use python2
• freeswitch-stable, maintained by @micmac1 , also tries to use python2

Both fail with almost identical error:
Makefile:235: /builder/shared-workdir/build/sdk/feeds/packages/lang/python/python-host.mk: No such file or directory

Otherwise it seems to be the usual suspects, not related to python2 removal.

[jslachta]

@hnyman, I will have an update to kamailio-5.x within a few days with a complete removal the python2 usage. Thank you for notifying me.

[commodo]

onet thing i noticed real quick
we'd probably need a new big-sweep to remove PACKAGE_python3-xxxx cond-deps;

i didn't like them much and if we can remove it, i would

[hnyman]

@jslachta

I will have an update to kamailio-5.x within a few days with a complete removal the python2 usage. Thank you for notifying me.

Great, as there is already a new bug filed about that, as users run into that warning regularly.
https://bugs.openwrt.org/index.php?do=details&task_id=2982

[jslachta]

@hnyman I have updated the kamailio-5.x package in master branch. Since the python support in kamailio was not really used, I completely removed the python support. I am going to reintroduce it later.