Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

factory openSUSE Urllib3Using.py on Python 2 #373

Closed
jayvdb opened this issue Apr 27, 2019 · 16 comments

Comments

@jayvdb
Copy link

commented Apr 27, 2019

When using branch factory Nuitka-0.6.3.1556202417, on openSUSE Python 2.7, .spec

[  774s] Consider output of recursively compiled program: Urllib3Using.py
[  774s] Comparing output of 'Urllib3Using.py' using '/usr/bin/python2' with flags silent, expect_success, standalone, remove_output ...
[  839s] --- /usr/bin/python2 (stderr)
[  839s] +++ nuitka (stderr)
[  839s] 
[  839s] @@ -1,5 +1,4 @@
[  839s] 
[  839s]  Traceback (most recent call last):
[  839s] -  File "/home/abuild/rpmbuild/BUILD/Nuitka-0.6.3.1556202417.e23ccdbb/tests/standalone/Urllib3Using.py", line 29, in <module>
[  839s] -    from http.server import BaseHTTPRequestHandler, HTTPServer
[  839s] +  File "/home/abuild/rpmbuild/BUILD/Nuitka-0.6.3.1556202417.e23ccdbb/tests/standalone/Urllib3Using.dist/Urllib3Using.py", line 29, in <module>
[  839s]  ImportError: No module named http.server
[  839s]  
[  839s] Error, outputs differed.
[  839s] Error exit! 1
[  839s] error: Bad exit status from /var/tmp/rpm-tmp.ysUnJG (%check)
@tommyli3318

This comment has been minimized.

Copy link
Member

commented Apr 27, 2019

Looks like the issue comes from Urllib3Using.py trying to use the http.server import, which seems to be only in the standard library of python 3 and not python 2.

@tommyli3318

This comment has been minimized.

Copy link
Member

commented Apr 27, 2019

@kayhayen would you like me to work on this issue?

@jayvdb

This comment has been minimized.

Copy link
Author

commented Apr 27, 2019

Confirming this only occurs on Python 2. It passes on Python 3

@kayhayen

This comment has been minimized.

Copy link
Member

commented Apr 27, 2019

@tommyli3318 Yes, please do so. I think the issue there is that http.server is a Python3 rename, and the Python2 module should be used as a fallback. I have on factory or develop, I forget, added a requirement on http.server as a stop gap measure the other day. But of course we would love to see this run on Python2 too.

@kayhayen kayhayen self-assigned this Apr 27, 2019

@kayhayen kayhayen added the bug label Apr 27, 2019

@tommyli3318

This comment has been minimized.

Copy link
Member

commented Apr 29, 2019

@jayvdb which version of Python2.7 are you using? I just ran this using Windows with python2.7.16 and I didn't get this error. Importing http.server seems to raise no issue.

@kayhayen

This comment has been minimized.

Copy link
Member

commented Apr 29, 2019

@tommyli3318 you probably have a backport installed? This is clearly one of the one Python3 namespace cleanups

@jayvdb

This comment has been minimized.

Copy link
Author

commented Apr 29, 2019

All issues with openSUSE in the title are using latest openSUSE rpms. In this case, for Python 2, the python and python-base rpms build according to recipes at https://build.opensuse.org/package/show/devel:languages:python:Factory/python . On the RHS are logs, which show the steps of the build.

As @kayhayen mentioned, you must have a backport, as http isnt valid in vanilla Python 2.

https://docs.python.org/3/library/http.html

vs

https://docs.python.org/2.7/search.html?q=http&check_keywords=yes&area=default

p.s. I'd love to know of a good backport of http for Python 2, as I could add that backport to my collection at https://build.opensuse.org/project/show/home:jayvdb:py-backports , which usually make it into the openSUSE main collection. At the end of the year, no more Python 2 stuff will be allowed into the main collection, and they will probably evict them then, but they should still be available in the released version of the OS, giving Python 2 a bit more life for those who need it.

@tommyli3318

This comment has been minimized.

Copy link
Member

commented Apr 29, 2019

I see, I wasn't even aware that I have backports installed. It might've just came with my Python3 installation. Thanks for clarifications

@tommyli3318

This comment has been minimized.

Copy link
Member

commented Apr 29, 2019

@jayvdb
I think https://docs.python.org/2.7/glossary.html#term-2to3 is the reason why my Python2 http import worked

@jayvdb

This comment has been minimized.

Copy link
Author

commented Apr 29, 2019

2to3 is an installation tool. It doesnt help unless you install something which provides http and explicitly asked 2to3 to help (usually getting python2 stuff working under python3, not the other way around).

tommyli3318 added a commit to tommyli3318/Nuitka that referenced this issue Apr 29, 2019
@jayvdb

This comment has been minimized.

Copy link
Author

commented Apr 29, 2019

I think https://python-future.org/standard_library_imports.html may be where your http.server is coming from.

@jayvdb

This comment has been minimized.

Copy link
Author

commented Apr 29, 2019

Yup. I installed python2-future, and the urllib3 test is now passing on Python 2.

It might be useful to move future to near the top of your list of PyPI packages to test, and then add it as a runtime dependency of Nuitka and derivatives on Python 2. It may solve a lot of heartache, as your tests will often have to copy the same voodoo that future has already built.

Soon this will become a loosing battle, as the moment any of the libraries you are testing need future, to properly test the others you need to isolate the test rig for the others, otherwise futures magic (and voodoo by similar libraries) will polute the testing environment. And libraries are also having the same problem -- increasingly future is (indirectly) being used by some part of their test environment, so they dont notice they even depend on its magic.

@tommyli3318

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

When you say

add it [future] as a runtime dependency of Nuitka

do you mean make it so that Nuitka doesn't run unless future is present? Please elaborate if I'm wrong.
Sorry I'm new to big open-source projects like Nuitka so I'm not familiar with technical terms.

@jayvdb

This comment has been minimized.

Copy link
Author

commented Apr 30, 2019

do you mean make it so that Nuitka doesn't run unless future is present?

Yes, for Python 2, and only for Python 2. For the things it silently does to make Python 2 much more similar to Python 3. But that is a big decision. Beyond the scope of this problem.

@kayhayen

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

Nah, "Urllib3Using.py" is first and foremost a program tasked to test that "urllib3" is working correctly, with all the Pythons. Therefore It is supposed to be written portable, and not have too unusual dependencies.

All we there need is a small HTTP server, and I believe that https://docs.python.org/2/library/simplehttpserver.html fits the bill here simple enough, so that is what we are going to use.

Then, what the test shows, is that the working program continue to work when compiled with Nuitka in standalone mode. No more.

For nice and more complete test coverage, we are going to run the test suite of urllib3 against the compiled package, but that is not what this test is supposed to do. There we are going to have their test dependencies installed. For now, standalone tests are not really allowed to require lots of things that are not normally there.

@kayhayen

This comment has been minimized.

Copy link
Member

commented Jun 7, 2019

This is fixed in the current factory and soon 0.6.4, I am closing it therefore.

@kayhayen kayhayen closed this Jun 7, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.