[tools/depends] Python 3.8.5

[fuzzard]

Description

Python patches updated for target platforms
native python updated to 3.8.5 to accomodate change in pgen
references to python3.7 updated to python3.8
updated pythonmodules builds to build against 3.8
Use official APIs for accessing interpreter list (PyInterpreterState_ThreadHead, PyThreadState_Next)

Motivation and Context

Update python interpreter to latest release version for Apple/Android (linux ppa?), Windows (Thanks Paxxi)

How Has This Been Tested?

runtime tested on osx.

Screenshots (if appropriate):

Types of change

• Bug fix (non-breaking change which fixes an issue)
• Clean up (non-breaking change which removes non-working, unmaintained functionality)
• Improvement (non-breaking change which improves existing functionality)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that will cause existing functionality to change)
• Cosmetic change (non-breaking change that doesn't touch code)
• None of the above (please explain below)

Checklist:

• My code follows the Code Guidelines of this project
• My change requires a change to the documentation, either Doxygen or wiki
• I have updated the documentation accordingly
• I have read the Contributing document
• I have added tests to cover my change
• All new and existing tests passed

[Paxxi]

I'll try and get it done sometime in the coming week. There's some new modules and header files reorganization so build system needs an update but shouldn't be too bad

[fuzzard]

Thanks, greatly appreciated

[Paxxi]

@fuzzard pushed the packages now.

[fuzzard]

Thanks Paxxi. I'll try get Jenkins all green (having repeated issues around package (automake for iOS in particular) downloads in other PRs unfortunately). Then I'll kick off some test builds and make a call out for python Devs to test

Jenkins build this please

[fuzzard]

Testbuilds have been triggered and they will go into http://mirrors.kodi.tv/test-builds/ for winx64, win-uwp64, android arm7/8, osx. They should start appearing in the next hour or so and will have PR18240-merge in the name of the build. All binary addons are being built (ie inputstream will be available for android)

Any testing/feedback would be greatly appreciated.

[Paxxi]

Would be great if @razzeee or @KarellenX could try these and see if they fix the datetime issue. I doubt it but one can always hope 😀

If anyone is curious, here's the windows patches https://github.com/Paxxi/cpython/tree/kodi/3.8

[razzeee]

I'm actually not getting the issue so @KarellenX or @sarbes could test

[CastagnaIT]

I have tested NF add-on with relative dependencies with an installation from scratch
under Windows 10 x64
and seem works well as usual

[KarellenX]

@Paxxi

I haven't had the issue for a while now, after Razzeee updated the scraper.

It was definitely TVDB Python scraper related.

[puenktchen]

I've got an error with the test build and the video addon (EmbyCon) i'm using. I know the addon isn't in the official repo anymore.

INFO <general>: initializing python engine.
ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
               Error Type: <class 'ModuleNotFoundError'>
               Error Contents: No module named '_lsprof'
               Traceback (most recent call last):
                 File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/plugin.video.embycon/default.py", line 6, in <module>
                   from resources.lib.functions import main_entry_point
                 File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/plugin.video.embycon/resources/lib/functions.py", line 9, in <module>
                   import cProfile
                 File "/data/user/0/org.xbmc.kodi/cache/apk/assets/python3.8/lib/python3.8/cProfile.py", line 9, in <module>
                   import _lsprof
               ModuleNotFoundError: No module named '_lsprof'
               -->End of Python script error report<--
INFO <general>: Python interpreter stopped

I can provide a debug log later if it's really needed.
Beside this little error, this pull request is a great addition and really needed for Android devices.
With Python 3.7 all is very slow on my Nvidia Shield. Addons and also Skins that rely on Python based script addons.
Until this pull request #18092 Matrix was completely unusable, but even after this it makes not really fun.
For example if i browse a library with the EmbyCon video addon, before that said pull request it was up to 4 times slower than Leia. With the current nightly it's still at least 2 times or more slower than Leia. But with this Python 3.8 build it's back to Leia speed.
So really great work @fuzzard!
Hope to see this get merged soon.

[fuzzard]

Thanks for the feedback.
I'll look into the lsprof module. At a glance, it should be being built for android, but i'll have a closer look.

[fuzzard]

Have added a commit for the _lsprof module. The default distibution setup file has changed a bit, and was no longer in it for 3.8. Not an issue to add, let me know if you come across any other modules.

Jenkins should have some android artifacts once built for the next 3 days if @puenktchen would like to do any further testing.

[puenktchen]

@fuzzard i think i've found the PR on Jenkins, but there's nothing to download. Was it already deleted by Jenkins?

[fuzzard]

Yeah, a PR build only gets stored for 3 days. I've got to rebase anyway, I'll get that done today/tomorrow when I get this other pillow PR fixup merged

[puenktchen]

I will hurry up next time but someone should check Jenkins' clock ;-) It was already gone 2 and a half day after the build process finished.
Thanks again for your afforts.

[fuzzard]

jenkins build this please

[fuzzard]

@puenktchen OSX build is available here currently http://mirrors.kodi.tv/test-builds/osx/x86_64/kodi-20200819-c78dcd35-PR18240-merge-x86_64.dmg

Android 32/64 and win64 can also be found on mirror testbuilds. Look for the one dated 20200819 PR18240-merge

[CiNcH83]

This PR seems to cause an exception in the Backup add-on on Android:

2020-08-19 17:07:30.563 T:31293   ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                                    - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                                   Error Type: <class 'ImportError'>
                                                   Error Contents: cannot import name 'splitattr' from 'urllib.request' (/data/user/0/org.xbmc.kodi/cache/apk/assets/python3.8/lib/python3.8/urllib/request.py)
                                                   Traceback (most recent call last):
                                                     File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.xbmcbackup/service.py", line 1, in <module>
                                                       from resources.lib.scheduler import BackupScheduler
                                                     File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.xbmcbackup/resources/lib/scheduler.py", line 8, in <module>
                                                       from resources.lib.backup import XbmcBackup
                                                     File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.xbmcbackup/resources/lib/backup.py", line 10, in <module>
                                                       from . vfs import XBMCFileSystem, DropboxFileSystem, ZipFileSystem
                                                     File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.xbmcbackup/resources/lib/vfs.py", line 11, in <module>
                                                       from . authorizers import DropboxAuthorizer
                                                     File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.xbmcbackup/resources/lib/authorizers.py", line 4, in <module>
                                                       import resources.lib.tinyurl as tinyurl
                                                     File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.xbmcbackup/resources/lib/tinyurl.py", line 3, in <module>
                                                       from future.moves.urllib.request import urlopen
                                                     File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.module.future/libs/future/moves/urllib/request.py", line 9, in <module>
                                                       from urllib.request import (getproxies,
                                                   ImportError: cannot import name 'splitattr' from 'urllib.request' (/data/user/0/org.xbmc.kodi/cache/apk/assets/python3.8/lib/python3.8/urllib/request.py)
                                                   -->End of Python script error report<--

[CastagnaIT]

Has been moved under "parse" module and is deprecated
see: https://github.com/python/cpython/blob/3.8/Lib/urllib/parse.py#L1141

[phunkyfish]

This runs my setup ok on OSX.

[lrusak]

Do we need more test builds or is this good to go?

[Maven85]

I've had this PR on my android private builds for over 2 weeks. I didn't see any problems.

Hope to see this get merged soon.

[Paxxi]

Based on the comments I'd say this is good enough to merge to get the most possible time in alpha builds

[thexai]

Is this already working for Windows too?

Shouldn't "python-3.8.5" be up to date at https://github.com/xbmc/xbmc/blob/master/project/BuildDependencies/scripts/0_package.target-x64.list ???

instead of python-3.7.5-x64-v141-20200105.7z

[fuzzard]

Looks like it lost paxxis commit during a rebase.
Feel free to PR the changes, I believe Paxxi uploaded all the relevant binaries for windows platforms already