Update python to 2.7.18 #3648

hgy59 · 2019-03-04T06:23:07Z

Motivation: Update python to 2.7.18 (the last 2.x version)

Checklist

Build rule all-supported completed successfully
Package upgrade completed successfully
New installation of package completed successfully

upgrade and new installation verified with x64-6.1

Updates an fixes of dependent packages

Fix and update pyload (fixes Pyload and DSM 6 #2259), includes update of tesseract and fix of js (except js on intel archs)
flexget in progress by FlexGet: update to v3.1.80, use python3 and generic service support #3747 (from version 3.x, flexget requires python >= 3.6)
ffsync (fails with python 2.7.15 too) nothing done, see separate PR ffsync: update to 1.8.0 #3849
deluge (fixes for geoip-api-c and libtorrent) (includes Wrong argument passed to start-stop-deamon #3933), see remarks below, update to version 2.0.3 closes [Deluge] Deluge 1.3.15-9 can't start on DS918+ #3650, closes Deluge randomly stopping and unable to run again without resetting the NAS (pid error in start-stop) #3381, closes [Package Request] Deluge 2.0.3 #3762. Implement watcher (autoadd plugin) for version 2.0.3. Fix service start (closes Wrong argument passed to start-stop-deamon #3933).
domoticz (updated to version 4.9700, migrated to python3, cross/boost enhanced for python3 without python(2) binding)
duplicity update to 0.7.19 (closes Update duplicity (0.7.13.1 released) #2882, closes Duplicity seems not installed correct #2908)
sabnzbd-testing package removed
oscam package removed (closes Retire oscam package? #2636)
plexivity
itools (updated to versions released until 2019.11, fix service privilege, use full device name as mount point)
rdiff-backup update to v2.0.3 and python3 (did not see Update rdiff-backup and librsync #3916 before) - only x64 and evansport working (related to Fix Python 3 wheels cross-compilation #3944)

Validated dependent packages

Remarks

This PR includes a redesign on how the boost library's BOOST_LIBRARIES are built. The dependency on cross/python is removed from cross/boost/Makefile to avoid conflicts with cross/python3. To build cross/boost with python(2) bindings, the dependency on cross/python must now precede cross/boost. With this dynamic dependency, it is possible for packages (like updated domoticz) to depend on python3 and boost (boost python3 bindings not yet required/implemented).
Update deluge to python3 may need cross/libtorrent with python3 bindings in the boost libraries. This is beyond the scope of this python2 package update.
Cross compilation of ossp-js (cross/js) is fixed but still does not work for pyload on intel archs - js interactive shell works on intel archs, but execution of javascript code like js -e "print(23+19)" fails with out of memory.
May be related to sickbeard cannot find sabnzdb download folder - same NAS #3308

ymartin59 · 2019-03-04T20:54:59Z

@hgy59 Thanks. In case of Python, I am not only interested in package own-upgrade but also in effect on dependent applications. I will do some testing

hgy59 · 2019-03-04T21:32:47Z

It would be very helpful if we had some makefile targets like "all-python" or "all-python3" to build python2.7 or python3 with all packages depending on it.

ymartin59 · 2019-03-04T21:39:32Z

@hgy59 I agree... A basic "grep" may do the trick... By the way, compilation success is far not enough, according to current troubles with stuck 3.6 upgrade.

m4tt075 · 2019-03-04T21:54:01Z

...and mono for qoriq platforms... ;-)

chickenandpork · 2019-03-05T00:29:40Z

I already have logic to find the things with DEPENDS -tracking (it showed me the build load for an OpenSSL change recently that I backed away from) I only track build dependencies, not install-dependencies, so it would be logic at a different level but I can mimic the same for a test-matrix of some kind.

…

On Mar 4, 2019, at 13:32, hgy59 ***@***.***> wrote: It would be very helpful if we had some makefile targets like "all-python" or "all-python3" to build python2.7 or python3 with all packages depending on it. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

hgy59 · 2019-03-07T07:53:57Z

Safihre · 2019-03-07T07:59:03Z

Feel free to remove sabnzbd-testing.

@ymartin59 I think it can be completely removed from SynoCommunity, also from the website etc.

Safihre · 2019-05-24T10:27:48Z

Any updates here? :)

Safihre · 2020-03-17T06:23:08Z

Maybe we should get this out now :)

hgy59 · 2020-03-17T17:43:56Z

@Safihre I am working on it. As it has so much dependencies, it takes some time to build and verify.

For python2, all packages that depend on python2 and all packaes depending on python modules, it takes more than 36 hours for me to build (results in ~ 220 spk files).
As there are some issues reported by dependabot (bot), there is still some work to do to update used python modules with security issues.
And python3 incl. dependent packages must be built and validated too.

And the (really) final python2 version is in sight (2.7.18 release candidate early April, 2020, 2.7.18 mid-April, 2020).
I hope that there will be none or only small adjustments to be done for the final update from version 2.7.17 to 2.7.18.
I started this PR as update to 2.7.16 and my target is to go out with version 2.7.18 and than freeze Python2 here.

Safihre · 2020-03-21T16:44:20Z

Any help to use Travis to do the building? But I guess we have some time if we wait untill 2.7.18.
Makes sense.

hgy59 · 2020-03-21T21:55:50Z

@Safihre thanks for the offer, but now it is more a matter of testing than building the packages.

ymartin59 · 2020-03-22T08:30:20Z

@hgy59 @Safihre At the moment, I focus on Python 3 and moving applications to Python 3. I really wonder if it is worth the effort to build and publish Python 2.7 and all other dependent applications - that may probably break after Python 2 update...
Except if you are convincend upgrading/publishing Python 2.7.17 will not break currently published application packages...

ymartin59 · 2020-04-14T10:03:05Z

@hgy59 Do not put effort on maintaining applications that now have Python 3 support. (Firefox Sync Server does not have yet).

hgy59 · 2020-04-14T18:51:04Z

@hgy59 Do not put effort on maintaining applications that now have Python 3 support. (Firefox Sync Server does not have yet).

not for ffsync (as you are working on it, and future update changed to rust).
So for the final commit I will have to revert changes on packages like octoprint, )
Changed domiticz to python3 with redesign of boost build. (But gave up to update to version 2020.1, as dependencies failed to update).
The final 2.7.18 will be released on 04/18/2020.

ymartin59 · 2020-04-18T07:28:34Z

@hgy59 I am (almost) over with ffsync, I will have a look at Subliminal.

…tire fix saltpad homepage use latest version of wheel that works with python 2 fix linking to rt lib update itools - update dependencies - fix ppc-archs - fix configure for linking to rt library in cross/usbmuxd fix usage of DSM_APP_NAME in spk makefiles retire oscam package (close SynoCommunity#2636) Fix pyload/tesseract-ocr for ppc853x-5.2

- fix DSM 6 compatibility - create certificate by installer (paths used by app for certificate creation does not work on dsm 6)

- update domoticz and migrate to python3 - archs not supported anymore: $(ARM5_ARCHES) powerpc ppc824x ppc853x ppc854x - update app icon

- use compiled bcrypt dependency - enable arch qoriq (unsupported are: powerpc ppc824x ppc853x ppc854x)

- fix duplicity wheels (missing modules requirements.txt) - update librsync - update duply

- add compression to cross/librsync - older versions of rdiff-backup fail to build with updated cross/librsync

- remove definition of LOG_FILE to use generic one - mylar: use subfolder for data to avoid conflicts with log files - fix couchpotatoserver - fix couchpotatoserver-custom - fix headphones - fix headphones-custom - fix lazylibrarian - fix mylar - fix nzbmegasearch - fix plexpy-custom - fix sabnzbd

- fix salt-api dependencies and add port - update salt-master to version 2015.5.11 (may be the last supported version by saltpad) - update salt-master dependencies (salt-api needs CherryPy < 12) - use branch saltpad_v1 as this is the latest python app - update dependencies to use current Flask - reduce icons to 512x512 pixel

- fix deluge start-stop-status - adjust deluge dependencies and add hi3535 to unsupported archs - archs not supported anymore: 88f6281 powerpc ppc824x ppc853x ppc854x hi3535 - use src/requirements.txt for wheels and use explicit versions - fix boost PLIST - fix watch folder configuration - configure autoadd plugin and activate on demand - update python module cryptography to 2.9.2 - add python module rencode 1.0.6 - deluge depends on rencode and latest python - fix python dependency - configure geoip data (still not working) - add wheels of rencode and PyHamcrest

- update pyload to 0.4.20 - fix DSM 6 service installation (use generic service approach where possible) - fix pyload.conf - fix host ip (use 0.0.0.0 instead of ::) and remove IPv6 patch since config does not work with "::" - remove obsolete patches - install config to var instead of etc folder for generic update handling - set user name and group for permissions in pyload.conf - fix cross compile js - js compiled for intel archs (x86_64, i386), does still not work for pyload - pyload JS engine test fails for intel archs as calling `js -e "print(23+19)"` returns "out of memory" instead of "42" - add diyspk/js (helpful for testing cross compiled js engine) - remove SPK_COMMANDS (not working with SERVICE_SETUP)

- use full device name for shares of mounted devices - fix service user privilege - need root for mount/umount and add/remove udev rules - update itools dependencies - update libplist and libusbmuxd to official releases - suppress call of configure in autogen.sh (libplist, libusbmuxd, usbmuxd, libimobiledevice) - use var folder for access-log and lock files - use package log (not install log) for service_prestart and service_poststop - log unexpected errors of mount/umount in error.log

- fix dsmappname configuration - fix quotes in spk description - make gateone icon square - popt: fix download url - sabnzbd/Makefile: use spksrc.install-resoruces.mk for package download - minio/src/wizard/install_uifile: fix html of link to permission management - haproxy/Makefile: use requirements.txt for wheels

- itools: remove obsolete install_target - flexget: revert update - rdiff-backup: generic installer

hgy59 · 2020-06-23T21:36:56Z

@hgy59 Is there anything left to add here? I hope not, this single PR contains already lots of varia fixes and updates.
If you expect to use "rebase-and-merge" (sounds more relevant than "squash-merge"), would it be possible to refactor some commit messages to prefix with package name - for instance "fix haproxy" would become "haproxy: fix for DSM 6" (for homogeneity)?

@ymartin59 I updated the commit messages (except the Fix service start/installation of Python2 dependents as there are too many packages updated)
Is it ok to rebase and mergenow?

ymartin59 · 2020-08-04T16:38:21Z

Safihre · 2020-08-04T20:22:24Z

@ymartin59 I would leave out sabnzbd as the Python 3 package is already finished and ready to merge: #4035

ymartin59 · 2020-08-09T14:19:09Z

spk/deluge/src/core.conf

@@ -16,7 +16,7 @@
  "daemon_port": 58846, 
  "torrentfiles_location": "@download_dir@", 
  "max_active_limit": 8, 
-  "geoip_db_location": "/usr/local/deluge/share/GeoIP/GeoIP.dat", 
+  "geoip_db_location": "/usr/share/geoip/GeoIP.dat", 


@hgy59 Notice I do not find that file on DSM 6.1 system, it is only available on DSM 6.2

hgy59 changed the title ~~Update python to 2.7.16~~ [WIP] Update python to 2.7.16 Mar 7, 2019

hgy59 mentioned this pull request Mar 28, 2019

Fix downloads and builds (without package updates) #3672

Merged

9 tasks

ymartin59 mentioned this pull request Jun 2, 2019

gevent compilation fails #3704

Closed

ymartin59 added the status/work-in-progress label Sep 22, 2019

hgy59 changed the title ~~[WIP] Update python to 2.7.16~~ [WIP] Update python to 2.7.17 Nov 9, 2019

hgy59 force-pushed the python_2.7.16 branch from c734e3e to f75172b Compare December 1, 2019 22:35

ymartin59 mentioned this pull request Jan 5, 2020

Python Issues & SABnzdb #3845

Closed

chickenandpork mentioned this pull request Jan 13, 2020

Determine and build affected spks #3854

Closed

hgy59 mentioned this pull request Mar 10, 2020

Python 2 builds broken #3918

Closed

hgy59 force-pushed the python_2.7.16 branch from 1c6c581 to bfc2395 Compare March 18, 2020 22:23

ymartin59 added this to In progress in Python 2.7.18 via automation Apr 5, 2020

hgy59 changed the title ~~[WIP] Update python to 2.7.17~~ Update python to 2.7.18 Apr 24, 2020

publicarray force-pushed the master branch from 1fc474a to 749ff73 Compare April 30, 2020 23:32

hgy59 mentioned this pull request May 3, 2020

Add Github Actions #3982

Merged

hgy59 added 15 commits June 23, 2020 23:28

haproxy: fix for DSM 6

f8d0982

- fix DSM 6 compatibility - create certificate by installer (paths used by app for certificate creation does not work on dsm 6)

plexivity: update dependencies

76575d8

domoticz: update to version 4.9700

09f70a1

- update domoticz and migrate to python3 - archs not supported anymore: $(ARM5_ARCHES) powerpc ppc824x ppc853x ppc854x - update app icon

homeassistant: update dependencies

a026691

- use compiled bcrypt dependency - enable arch qoriq (unsupported are: powerpc ppc824x ppc853x ppc854x)

duplicity: update duply and other dependencies

d7917d8

- fix duplicity wheels (missing modules requirements.txt) - update librsync - update duply

rdiff-backup: update to v2.0.0 with python3

e7f96e8

- add compression to cross/librsync - older versions of rdiff-backup fail to build with updated cross/librsync

incorporate code review

e216e3e

- itools: remove obsolete install_target - flexget: revert update - rdiff-backup: generic installer

incorporate fixes for mistakes with rebase

a5abc3b

hgy59 force-pushed the python_2.7.16 branch from cd6b733 to a5abc3b Compare June 23, 2020 21:34

hgy59 merged commit a8cbf70 into SynoCommunity:master Jun 25, 2020

Python 2.7.18 automation moved this from Reviewer approved to Done Jun 25, 2020

hgy59 added status/to-publish update request to update existing package and removed status/ready-to-merge labels Jun 25, 2020

hgy59 mentioned this pull request Jun 25, 2020

Update rdiff-backup and librsync #3916

Closed

5 tasks

hgy59 deleted the python_2.7.16 branch June 27, 2020 07:40

hgy59 mentioned this pull request Jun 28, 2020

[github-action] limit changed files for pull requests #4049

Merged

meolite mentioned this pull request Jul 1, 2020

[Package Request] Deluge 2.0.3 #3762

Closed

ymartin59 removed the status/to-publish label Aug 6, 2020

ymartin59 reviewed Aug 9, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update python to 2.7.18 #3648

Update python to 2.7.18 #3648

hgy59 commented Mar 4, 2019 •

edited

ymartin59 commented Mar 4, 2019

hgy59 commented Mar 4, 2019

ymartin59 commented Mar 4, 2019

m4tt075 commented Mar 4, 2019

chickenandpork commented Mar 5, 2019 via email

hgy59 commented Mar 7, 2019 •

edited

Safihre commented Mar 7, 2019

Safihre commented May 24, 2019

Safihre commented Mar 17, 2020

hgy59 commented Mar 17, 2020

Safihre commented Mar 21, 2020

hgy59 commented Mar 21, 2020

ymartin59 commented Mar 22, 2020

ymartin59 commented Apr 14, 2020

hgy59 commented Apr 14, 2020

ymartin59 commented Apr 18, 2020

hgy59 commented Jun 23, 2020

ymartin59 commented Aug 4, 2020 •

edited

Safihre commented Aug 4, 2020

ymartin59 Aug 9, 2020

Update python to 2.7.18 #3648

Update python to 2.7.18 #3648

Conversation

hgy59 commented Mar 4, 2019 • edited

Checklist

Updates an fixes of dependent packages

Validated dependent packages

Remarks

ymartin59 commented Mar 4, 2019

hgy59 commented Mar 4, 2019

ymartin59 commented Mar 4, 2019

m4tt075 commented Mar 4, 2019

chickenandpork commented Mar 5, 2019 via email

hgy59 commented Mar 7, 2019 • edited

Safihre commented Mar 7, 2019

Safihre commented May 24, 2019

Safihre commented Mar 17, 2020

hgy59 commented Mar 17, 2020

Safihre commented Mar 21, 2020

hgy59 commented Mar 21, 2020

ymartin59 commented Mar 22, 2020

ymartin59 commented Apr 14, 2020

hgy59 commented Apr 14, 2020

ymartin59 commented Apr 18, 2020

hgy59 commented Jun 23, 2020

ymartin59 commented Aug 4, 2020 • edited

Safihre commented Aug 4, 2020

ymartin59 Aug 9, 2020

Choose a reason for hiding this comment

hgy59 commented Mar 4, 2019 •

edited

hgy59 commented Mar 7, 2019 •

edited

ymartin59 commented Aug 4, 2020 •

edited