-
-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
python: wxPython: init at 4.0.6 #64047
Conversation
also of interest to @deliciouslytyped, sorry for delay! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should build for python2 according to pypi, please try to build that as well
gtk3 libjpeg libtiff SDL gst-plugins-base libnotify freeglut xorg.libSM | ||
which | ||
]; | ||
nativeBuildInputs = [ pkgconfig which doxygen wxGTK libX11 ]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Like I said before, wxGTK
and libX11
are libraries, so let's move them to buildInputs
if they are needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@veprbl if I do this, the build fails shrug
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The build succeeds if I remove libX11
. Moving wxGTK
, indeed, fails because it can't find "wx-config", which I don't understand ATM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this still matter? I'm going back to take another shot at enabling tests and I might look into this or add it as an open problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is probably okay now
nixpkgs/pkgs/development/python-modules/wxPython/4.0.nix
Lines 40 to 48 in b1bffdf
nativeBuildInputs = [ pkgconfig which doxygen wxGTK ]; | |
buildInputs = [ libjpeg libtiff SDL | |
gst-plugins-base libnotify freeglut xorg.libSM ncurses | |
requests libpng gstreamer libX11 | |
pathlib2 | |
(wxGTK.gtk) | |
] | |
++ lib.optional openglSupport pyopengl; |
85dfa10
to
deaf7e3
Compare
@veprbl @jonringer thx for great comments! I took suggestions as possible, except where it would break the build |
''; | ||
|
||
buildPhase = '' | ||
python build.py -v --use_syswx dox etg --nodoc sip build_py |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please build a wheel using bdist_wheel
. The installPhase
then won't need to be overridden.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @FRidh! When I add python build.py bdist_wheel
to the end of buildPhase, I get:
File "setup.py", line 15, in <module>
from setuptools import setup, find_packages
ModuleNotFoundError: No module named 'setuptools'
Per the manual, I tried adding setuptools_scm
to nativeBuildInputs
and setuptools
to propogatedBuildInputs
, but the error is unchanged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The one time I had an error like that I think something was wrong with how PYTHONPATH was set up (determined by comparing the failing version to something else maybe?), but I don't think I spent the time to figure it out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are using a custom build system based on waf so I don't think our generic builders will be able to handle this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I worked on my own version of this PR a while ago, and was able to get bdist_wheel
working using this: lopsided98@47786de#diff-5dbc383e8d8890c6c54121d966199558R31
ping? |
nativeBuildInputs = [ | ||
pkgconfig | ||
]; | ||
doCheck = false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why? I haven't tried it yet, just asking.
''; | ||
|
||
installPhase = '' | ||
${python.interpreter} setup.py install --skip-build --prefix=$out |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This uses python.interpreter
but buildPhase just calls python? Why the difference?
I don't know what this does offhand but maybe this has something to do with the setuptools
error? (or maybe not)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In our generic builder we use a following:
${python.pythonForBuild.interpreter} nix_run_setup ${setupPyGlobalFlagsString} ${setupPyBuildExtString} bdist_wheel |
wrapPythonPrograms | ||
''; | ||
|
||
passthru = { inherit wxGTK openglSupport; }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@veprbl I'm trying to enable tests but working on it is really frustrating when a failure in the test phase fails the whole derivation and I have to wait an hour for the next cycle. Is there any way to separate tests and the build into separate derivations? Edit: meanwhile I worked around this by using nix-shell. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I worked on my own version of this PR a while ago but hadn't gotten around to submitting it. Overall, this PR seems better (it doesn't rely on wx-config and builds the documentation), but there might a few parts of my version that you could use.
@@ -4660,6 +4660,11 @@ in { | |||
inherit (pkgs) pkgconfig; | |||
}; | |||
|
|||
wxPython_4_0 = callPackage ../development/python-modules/wxPython/4.0.nix { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should either follow the naming convention of wxPython30
, or wxPython30
should be renamed (with an alias for compatibility).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, since wxPython 3.0 does not support Python 3, we might want to make the wxPython
package point to 4.0 on Python 3. Even if some packages aren't compatible, it seems better than having every package that uses wxPython fail to evaluate on Python 3.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is WxPython 4 API compatible with 3?
Edit: I suppose python 3 apps wont have been coded to use wxpython 3 anyway though because there was no support for wxpython in python 3 before... I think what I was trying to say is: is incompatibility even a thing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are not fully API compatible. Any application that wants to target both Python 2 and 3 is probably going to use wxPython 4.0, but it might be possible to be compatible with both 3.0 and 4.0.
Making 4.0 the default for Python 3 will only make packages that currently fail to evaluate, fail to build.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lopsided98 see #64047 (comment)
I agree re pointing wxPython to 4.0 for python3..do you know how to do this? I’m familiar with overrides but not with how to do this in nixpkgs
''; | ||
|
||
buildPhase = '' | ||
python build.py -v --use_syswx dox etg --nodoc sip build_py |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I worked on my own version of this PR a while ago, and was able to get bdist_wheel
working using this: lopsided98@47786de#diff-5dbc383e8d8890c6c54121d966199558R31
@lopsided I can't reply to your post for some reason but the original buildPhase has similar code to that
|
Was just making sure :) Another option would be to make 2 derivations, one for the building the initial package, then another that imports that package and runs the tests |
Yes, I adapted mine from that. |
It annoys me that you had to duplicate the code like that but I don't know what to do about it. :( |
It appears to me that It might be good to get some wxPython people to review our stuff at some point? Edit: |
@jonringer Thanks, I totally missed your comment. I was thinking about making two derivations like you said but I thought of how to do it with Currently stuck on |
However, I don't understand why that would be the case given that both |
@deliciouslytyped your installCheckPhases are being overriden most likely because you're using a
This will happen if you override the buildPhase, but use the default checkPhase. If you're using installCheckPhase, i would move it to checkPhase. Or even better yet, try to use the preCheck, and postCheck hooks if it fits your problem domain. |
Doh, the order on the Seems a bit unorthodox that they go and override that... |
@deliciouslytyped for the average python package, this behavior makes sense. The typical "build and install"for python is just dumping .py's into a directory. If you do need all the phases and "more" control, you could do |
Edit: see also: https://github.com/wxWidgets/Phoenix/pull/1135/files for test thingy usage example WIP, has tests at least running: { openglSupport ? true
, lib
, stdenv
, buildPythonPackage
, fetchPypi
, python
, pyopengl
, wxGTK
, cairo
, pango
#nativeBuildInputs
, which
, doxygen
, pkgconfig
#buildInputs
, libjpeg
, libtiff
, SDL
, gst-plugins-base
, libnotify
, freeglut
, xorg
, ncurses
, requests
, libpng
, gstreamer
, libX11
, pathlib2
#checkInputs #TODO clean
, numpy
, pytest, pytest_xdist, pytest-timeout
, sphinx
, xvfb_run
}@args:
let
pname = "wxPython";
version = "4.0.6";
in
buildPythonPackage rec {
inherit pname version;
src = fetchPypi {
inherit pname version;
sha256 = "35cc8ae9dd5246e2c9861bb796026bbcb9fb083e4d49650f776622171ecdab37";
};
checkInputs = [ xvfb_run sphinx numpy pytest pytest_xdist pytest-timeout ];
disabledTests = [
"access_Tests" #TODO because not implemented yet and automatically fails
"test_lib_agw_piectrlMethods" #TODO because hangs the build and timeout skips the rest of tests
];
checkPhase = ''
runHook preCheck
xvfb-run -s '-screen 0 800x600x24' \
${python.pythonForBuild.interpreter} build.py test --verbose --pytest_timeout=60 \
--extra_pytest="-k 'not ( ${lib.concatStringsSep " or " disabledTests} )'"
runHook postCheck
'';
nativeBuildInputs = [ pkgconfig which doxygen wxGTK ];
buildInputs = [
libjpeg libtiff SDL
gst-plugins-base libnotify freeglut xorg.libSM ncurses
requests libpng gstreamer libX11
pathlib2
(wxGTK.gtk)
] ++ (lib.optional openglSupport pyopengl);
hardeningDisable = [ "format" ];
DOXYGEN = "${doxygen}/bin/doxygen";
preConfigure = lib.optionalString (!stdenv.isDarwin) ''
substituteInPlace wx/lib/wxcairo/wx_pycairo.py \
--replace 'cairoLib = None' 'cairoLib = ctypes.CDLL("${cairo}/lib/libcairo.so")'
substituteInPlace wx/lib/wxcairo/wx_pycairo.py \
--replace '_dlls = dict()' '_dlls = {k: ctypes.CDLL(v) for k, v in [
("gdk", "${wxGTK.gtk}/lib/libgtk-x11-2.0.so"),
("pangocairo", "${pango.out}/lib/libpangocairo-1.0.so"), #TODO does this need to refer to .out
("appsvc", None)
]}'
'';
prePatch = ''
substituteInPlace build.py \
--replace "os.environ['PYTHONPATH'] = phoenixDir()" "os.environ['PYTHONPATH'] = os.environ['PYTHONPATH'] + ':' + phoenixDir()"
'';
buildPhase = ''
python build.py -v --use_syswx dox etg --nodoc sip build_py
'';
installPhase = ''
${python.interpreter} setup.py install --skip-build --prefix=$out #TODO this uses python.interpreter but the above does not?
wrapPythonPrograms
'';
passthru = { inherit args; };
meta = {
description = "Cross platform GUI toolkit for Python, Phoenix version";
homepage = http://wxpython.org/;
license = lib.licenses.wxWindows;
};
} |
Something looks sketchy here:
Does it just abort the rest of the test process on a timeout? Note the last PASSED is at 44%. Edit: The help string does say |
@deliciouslytyped thanks for picking up, been offline in the wilderness. Will try to look at tests in a couple days, they are a real PITA which is why I disabled doCheck. I did try some applications though, like Helloworld, that worked fine. |
This pull request has been mentioned on Nix community. There might be relevant details there: |
Test termination behavior of timeout is explained at https://pypi.org/project/pytest-timeout/ |
@FRidh stuff seems to generally work, how should failing tests be handled? It would be nice to get this merged finally... I'm also not sure how to process test ouput. |
@@ -4660,6 +4660,11 @@ in { | |||
inherit (pkgs) pkgconfig; | |||
}; | |||
|
|||
wxPython_4_0 = callPackage ../development/python-modules/wxPython/4.0.nix { | |||
inherit (pkgs) pkgconfig; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why? (Well ok this isn't specific to this PR)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason why we need to explicitly pass things from pkgs
is because the callPackage
defined in python-packages.nix
callPackage = pkgs.newScope self; |
will only substitute arguments from
self
(aka pythonPackages
) and not from pkgs
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pythonPackages seems to have its own pkgconfig attribute too somehow? Meanwhile pkgs has pkg-config... (or I got mixed up somewhere)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok so pkgs has a pkgconfig and a pkg-config, and pyhtonPackages has a pkgconfig.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, so my explanation was wrong. The callPackage
will also substitute arguments from pkgs
, but self
(pythonPackages
) takes priority. And, indeed we already have pkgconfig
attribute in pythonPackages
, but we don't want to get the "pkgconfig" python package, we want just the regular pkgconfig. And this is why we need to explicitly state that it must come from pkgs
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct, you should be able to import something like "tree", even though it doesnt exist in top-level/python-modules.nix, because it inherits from pkgs. However, if you do something like "docker" you'll get pythonPackages.docker
instead of ./nixpkgs -A docker
due to how callPackage prioritizes overlapping attrs.
@deliciouslytyped you can use the pytest testrunner, it has more options to disabling certain tests or files.
example can be seen here 93ebc61 |
@FRidh can you please set some minumum criteria for getting this merged so I have something to work towards? |
Can/should we do anything about using wrapgappshook here? or does that need to be used in the libraries/binaries that consume this package? |
I made a change and pushed to master. Thanks for the effort. In the future, though, we should improve this to build a wheel instead. |
Yay, since this PR is closed, I should probably create a TODO list in a new issue "soon". |
What did you change? |
Using ${python.interpreter} instead of python |
I'm not volunteering right now but does anyone want to work on getting tests enabled and following up with upstream about the tests? |
Motivation for this change
add wxPython 4, which is unlisted in python-modules and broken.
Things done
Split off from #61253 and addressed comments by @veprbl
sandbox
innix.conf
on non-NixOS)nix-shell -p nix-review --run "nix-review wip"
./result/bin/
)nix path-info -S
before and after)Resolves: #55426