Drop dependency on python2 #109

rojkov · 2017-04-11T09:40:51Z

No description provided.

rojkov · 2017-04-11T13:25:54Z

Ok, the test build failed at running the tests that check for python presence which is actually what we want. I'll update the PR not to use bbappends for tweaking PACKAGECONFIG.

rojkov · 2017-04-12T08:20:37Z

For some reason bitbake creates empty package libxml2-python which the QA check reports about if PACKAGECONFIG is tweaked in a distro config. If bbappend is used this doesn't happen.

rojkov · 2017-04-12T13:37:04Z

Added INSANE_SKIP_libxml2-python =+ "build-deps" to refkit.conf for now. Also python3 needs some update-alternatives magic to pass the failing tests.

ipuustin · 2017-04-18T11:27:37Z

meta-refkit/conf/distro/refkit.conf

+# drop python2 support
+PACKAGECONFIG_remove_pn-opkg-utils = "python"
+PACKAGECONFIG_remove_pn-libxml2 = "python"
+# there's a bug in bitbake that causes the build-deps QA check to check an empty package


Have you filed a bug about this?

Yocto is such a nice thing I'm still struggling with how to formulate the bug properly. So far I've come up with this:

Given localdata is a local copy of d after calling localdata.setVar('OVERRIDES', 'libxml2-python') its method getVar() starts to return a whitespace (" ") for localdata.getVar('RDEPENDS') if libxml2-python is made an empty package with PACKAGECONFIG_remove = "python" in a bbappend

But if libxml2-python is made an empty package with PACKAGECONFIG_remove_pn-libxml2 = "python" in a distro config then localdata.getVar('RDEPENDS') returns " python-core"
and that breaks the 'build-deps' QA check because it thinks that libxml2-python actually exists even if it's empty.

Can you get it?

rojkov · 2017-04-18T12:40:27Z

Submitted https://bugzilla.yoctoproject.org/show_bug.cgi?id=11374 Hopefully it's clear enough.

…

On Tue, Apr 18, 2017 at 2:27 PM, Ismo Puustinen ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In meta-refkit/conf/distro/refkit.conf <#109 (comment)> : > @@ -74,6 +74,13 @@ PACKAGECONFIG_remove_pn-pulseaudio = "avahi" # leave out readline, gdbm, and db PACKAGECONFIG_pn-python3 = "" +# drop python2 support +PACKAGECONFIG_remove_pn-opkg-utils = "python" +PACKAGECONFIG_remove_pn-libxml2 = "python" +# there's a bug in bitbake that causes the build-deps QA check to check an empty package Have you filed a bug about this? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#109 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACCYeik7oR8oDee44IX0o7EL5g-DLh8ks5rxJ4pgaJpZM4M52px> .

rojkov · 2017-04-18T13:30:08Z

rebased and force pushed.

mythi · 2017-04-19T07:13:18Z

@rojkov I have oe-core and bitbake updated in master

rojkov · 2017-04-19T11:20:56Z

Rebased and force pushed.

@mythi It seems the oe-core devs oppose the idea that python3 become python at some point in spite of what the python devs themselves believe in [1]. Should we stay with the current workaround or start adding locally maintained patches to ROS scripts to replace shebang lines?

[1] https://patchwork.openembedded.org/patch/139143/

mythi · 2017-04-20T06:17:02Z

meta-refkit/recipes-devtools/nodejs/nodejs_4.5.0.bbappend

+# NodeJS needs Python for node-gyp which is a NPM module bundled with
+# GYP together. GYP is a build system generator written in Python that
+# stands for "Generate Your Projects" and is used for building Node
+# modules written in C. Since Refkit doesn't support on-target


This isn't exactly true so let's be careful with the wording. Furthermore, even if refkit did not support this today, a refkit user might.

AFAICS this cripples node-gyp by keeping it installed (via node-npm) but simply drops dependencies to python. How about doing packaging such that npm and node-gyp are in their own packages.

Submitted ostroproject/meta-iot-web#37 introducing nodejs-gyp, but then the supported recipes check fails even though nodejs-gyp is not installed. It needs to be amended to ignore RDEPENDS for not installed packages.

Actually nodejs-npm is not installed on any image. So, it's just the check's greediness issue.

mythi · 2017-04-20T06:19:36Z

meta-refkit/conf/distro/refkit.conf

+# to check an empty package:
+# https://bugzilla.yoctoproject.org/show_bug.cgi?id=11374
+INSANE_SKIP_libxml2-python =+ "build-deps"
+PACKAGECONFIG_remove_pn-libxml2-native = "python"


Is this -native part related to that bug?

mythi · 2017-04-20T06:20:08Z

meta-refkit/conf/distro/refkit.conf

 PACKAGECONFIG_pn-python3 = ""

+# drop python2 support
+PACKAGECONFIG_remove_pn-opkg-utils = "python"


Does opkg-utils get installed?

There's no such package in the image manifests. But the supported recipes check fails if it's missing from refkit-supported-recipes.txt
@pohly could you please help with this? Perhaps it gets pulled because of update-alternatives in coreutils.

mythi · 2017-04-20T06:28:39Z

@rojkov What's the current workaround and why local ROS patches would be needed? My thinking is #120 would be the way to start.

rojkov · 2017-04-20T06:57:35Z

@rojkov <https://github.com/rojkov> What's the current workaround and why local ROS patches would be needed? My thinking is #120 <#120> would be the way to start.

As a workaround I've added a bbappend to python3 that makes it inherit update-alternatives and create a sym link (/usr/bin/python -> /usr/bin/python3). Without the link all python scripts that have /usr/bin/python (/usr/bin/env python) in their shebang line need to be patched to reference python3 explicitly.

mythi · 2017-04-20T07:22:32Z

On Thu, Apr 20, 2017 at 9:57 AM, Dmitry Rozhkov ***@***.***> wrote: > @rojkov <https://github.com/rojkov> What's the current workaround and why > local ROS patches would be needed? My thinking is #120 > <#120> would be the way to > start. > > As a workaround I've added a bbappend to python3 that makes it inherit update-alternatives and create a sym link (/usr/bin/python -> /usr/bin/python3). Without the link all python scripts that have /usr/bin/python (/usr/bin/env python) in their shebang line need to be patched to reference python3 explicitly. OK, I was on the same page. With #120 neither of these would be needed.

Did ROS work with the update-alternatives approach btw?

rojkov · 2017-04-20T07:42:03Z

OK, I was on the same page. With #120 neither of these would be needed. Did ROS work with the update-alternatives approach btw?

At least a master ROS node got launched successfully (written in python). Unofficially ROS supports both versions in its core modules starting from Indigo [1]. Unfortunately OE-core has dropped python2 bindings for Boost and it forces switching to python3 if ROS computer vision modules get installed on an image. [1] http://wiki.ros.org/indigo/Migration

mythi · 2017-04-21T12:06:20Z

@rojkov hmm the lack of python2 boost bindings in oe-core sounds like an argument to accept 0bb82bd

ipuustin · 2017-04-21T12:10:02Z

@mythi: Boost is difficult to compile so that it works for both Pythons at the same time. See https://bugzilla.yoctoproject.org/show_bug.cgi?id=11104

pohly · 2017-04-24T07:19:38Z

On Thu, 2017-04-20 at 00:23 -0700, Dmitry Rozhkov wrote: There's no such package in the image manifests. But the supported recipes check fails if it's missing from refkit-supported-recipes.txt @pohly could you please help with this? Perhaps it gets pulled because of update-alternatives in coreutils.

If "python" needs to be built, then it must be listed as supported, even if it does not get installed on the images. There's no distinction between "installed vs. not installed" target code and that can't be added because it's simply unknown how the resulting binaries will be used.

rojkov · 2017-04-24T08:16:20Z

Ok, I see now. opkg's dep resolver can have different opinion on what to install indeed.
I'll put python@core back to the list of supported recipes.

rojkov · 2017-04-24T09:49:33Z

rebased;
put python@core back to refkit-supported-recipes.txt;
dropped unneeded overrides;
force pushed.

rojkov · 2017-05-08T11:23:35Z

rebased and force pushed.

rojkov · 2017-05-18T05:29:34Z

rebased and force pushed.

refkit-ci · 2017-05-26T09:48:26Z

Can one of the admins verify this patch?

rojkov · 2017-05-31T08:04:37Z

rebased and force pushed.

Arthil · 2017-06-15T13:22:15Z

There was a problem in CI, where one of the boards froze at test phase. Should be fixed now.
Test this please.

mythi · 2017-06-16T08:34:02Z

@rojkov please rebase

rojkov · 2017-06-16T08:57:19Z

rebased and force pushed.

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

lowpan-tools provides lowpan-tools-python package which is not compatible with python3. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

This is a temporary bbappend until the patch https://patchwork.openembedded.org/patch/139143/ gets accepted upstream. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

rojkov · 2017-06-16T13:15:59Z

rebased and force pushed.

This combined merge takes PRs intel#109, intel#113, and intel#188.

rojkov force-pushed the python2-less branch from a5afe71 to 3ba9559 Compare April 11, 2017 11:35

rojkov force-pushed the python2-less branch 2 times, most recently from 3c641bf to b044c5d Compare April 11, 2017 14:15

rojkov force-pushed the python2-less branch from b044c5d to ed4c326 Compare April 12, 2017 12:57

rojkov force-pushed the python2-less branch from d238331 to ca1edca Compare April 13, 2017 10:34

ipuustin reviewed Apr 18, 2017

View reviewed changes

rojkov force-pushed the python2-less branch from ca1edca to 4f1fbc2 Compare April 18, 2017 12:39

rojkov force-pushed the python2-less branch from 4f1fbc2 to 9750d36 Compare April 19, 2017 08:50

ipuustin mentioned this pull request Apr 19, 2017

Configure Python2 to drop GPLv3 dependencies. #120

Merged

rojkov mentioned this pull request Apr 19, 2017

beignet: add ncurses dependency. #121

Merged

mythi reviewed Apr 20, 2017

View reviewed changes

rojkov force-pushed the python2-less branch from 9750d36 to 0bb82bd Compare April 21, 2017 10:41

rojkov force-pushed the python2-less branch from 0bb82bd to 1e954dd Compare April 24, 2017 08:22

rojkov force-pushed the python2-less branch from 1e954dd to c23637c Compare May 8, 2017 11:22

rojkov mentioned this pull request May 8, 2017

Add industrial profile #49

Merged

rojkov force-pushed the python2-less branch from c23637c to 7a8972d Compare May 18, 2017 05:02

rojkov force-pushed the python2-less branch from 7a8972d to dc00dd3 Compare May 31, 2017 08:04

rojkov force-pushed the python2-less branch 2 times, most recently from 6554438 to 556c8b8 Compare June 2, 2017 13:41

rojkov force-pushed the python2-less branch from 556c8b8 to cb5c8e0 Compare June 16, 2017 08:42

Dmitry Rozhkov added 6 commits June 16, 2017 16:01

mraa: switch to python3

318a0d5

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

upm: switch to python3

2cfedd6

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

refkit-supported-recipes.txt: drop python2 libraries

0af1330

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

refkit-supported-recipes.txt: drop lowpan-tools

20240ae

lowpan-tools provides lowpan-tools-python package which is not compatible with python3. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

meta-iotqa: make python tests compatible with python3

51a8abc

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

python3: make python3 an alternative for the python command

9e23630

This is a temporary bbappend until the patch https://patchwork.openembedded.org/patch/139143/ gets accepted upstream. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>

rojkov force-pushed the python2-less branch from cb5c8e0 to 9e23630 Compare June 16, 2017 13:12

mythi added a commit to mythi/intel-iot-refkit that referenced this pull request Jun 16, 2017

Combine multiple PRs in one

46f2dd3

This combined merge takes PRs intel#109, intel#113, and intel#188.

mythi added a commit to mythi/intel-iot-refkit that referenced this pull request Jun 17, 2017

Combine multiple PRs in one

7b781b3

This combined merge takes PRs intel#109, intel#113, and intel#188.

mythi merged commit 9e23630 into intel:master Jun 17, 2017

Drop dependency on python2 #109

Drop dependency on python2 #109

Uh oh!

Conversation

rojkov commented Apr 11, 2017

Uh oh!

rojkov commented Apr 11, 2017

Uh oh!

rojkov commented Apr 12, 2017

Uh oh!

rojkov commented Apr 12, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rojkov commented Apr 18, 2017 via email

Uh oh!

rojkov commented Apr 18, 2017

Uh oh!

mythi commented Apr 19, 2017

Uh oh!

rojkov commented Apr 19, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mythi commented Apr 20, 2017

Uh oh!

rojkov commented Apr 20, 2017 via email

Uh oh!

mythi commented Apr 20, 2017 via email

Uh oh!

rojkov commented Apr 20, 2017 via email

Uh oh!

mythi commented Apr 21, 2017

Uh oh!

ipuustin commented Apr 21, 2017

Uh oh!

pohly commented Apr 24, 2017 via email

Uh oh!

rojkov commented Apr 24, 2017

Uh oh!

rojkov commented Apr 24, 2017

Uh oh!

rojkov commented May 8, 2017

Uh oh!

rojkov commented May 18, 2017

Uh oh!

refkit-ci commented May 26, 2017

Uh oh!

rojkov commented May 31, 2017

Uh oh!

Arthil commented Jun 15, 2017

Uh oh!

mythi commented Jun 16, 2017

Uh oh!

rojkov commented Jun 16, 2017

Uh oh!

rojkov commented Jun 16, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants