Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cpython: add support for windows #5

Open
6 of 18 tasks
myselfhimself opened this issue Dec 16, 2019 · 23 comments · May be fixed by #74
Open
6 of 18 tasks

cpython: add support for windows #5

myselfhimself opened this issue Dec 16, 2019 · 23 comments · May be fixed by #74
Assignees
Labels
enhancement New feature or request release Release management

Comments

@myselfhimself
Copy link
Owner

myselfhimself commented Dec 16, 2019

  • add msys2 to Github Workflow recipe
  • install all libraries
  • install all python dependencies (pkg-config, wurlitzer)
  • complement setup.py with msys2/mingw libomp libs and include paths
  • build .dll sucessfully (debug)
  • build .dll sucessfully (optimized)
  • have test suite running green
  • have someone test GDI display
  • build .whl successfully (without missing .dll repair in place)
  • build .whl successfully (.dll repair in place)
  • build .whl successfully (optimized, .dll repair in place)
  • make set up for python 3.6-3.9 or so (cibuildwheel?)
  • release for python 3.6
  • release for python 3.7
  • release for python 3.8
  • release for python 3.9 (?)
  • announce release
  • work back in master branch so that tag-based releases trigger both windows and linux etc.. releases
@myselfhimself
Copy link
Owner Author

This is deprioritized in favor of making first a fully fledged gmic-py v1 for Linux (and Mac OS)

@myselfhimself
Copy link
Owner Author

Since we want to bring G'MIC into Blender thanks to this gmic-py binding, looking at Blender's Open Data shows that 2/3 of Blender users are coming from Windows... So Windows support will be needed pretty soon :)

image
Per: https://opendata.blender.org/

@myselfhimself
Copy link
Owner Author

Consider using cibuildwheel with github action support. See related issue having good progress: pypa/cibuildwheel#159

@paulhart2
Copy link

Attempting to build for Win64 platform. Major errors, not sure how to get going. Thanks for the efforts to get GMIC Python to work in Blender. I will build for GraphicAll.org when it works.

gmic-install_errors.txt

@myselfhimself
Copy link
Owner Author

Hello @paulhart2
Thank you very much for the errors reporting. I see how to correct them but will come back to you once I have tried compilation on the Github Actions CI platform which probably has MSVC like you, I will ping you back if I struggle or succeed.
Note that gmic-py is indeed intended for any Python 3.5+ environment, but if imported as is within Blender3d, developers using it will need higher skills in order to grab Blender buffers and pipe them into G'MIC or will have to work files only. The latest binary builds of gmic-py are on pypi.org here and I will care that this pypi repository always remains up to the latest version of the underlying G'MIC library as long as I am contracted on this project.

What you may want for Blender3d is maybe the G'MIC Blender plugin, which is to expose more than 500 compositing or texture nodes, but for now that plugin has only Linux support and does not even create 1 node.

I wanted to finish a work on https://github.com/douglaskastle/blender-addon-tester first (very simple MacOSX support). This is a matter of 1/2 day I hope.

Could you tell me what is GraphicAll.org for? Do you build just Blender 3d or a variety of open-source graphics software? How would you see g'mic as a python binding or Blender plugin in it?
Thanks

@myselfhimself
Copy link
Owner Author

WIP Windows build job on Github Action: https://github.com/myselfhimself/gmic-py/runs/526438299?check_suite_focus=true

  • setup.py needs removal of pkgconfig checks for Windows
  • wget is not an available command, maybe use it non-aliased WindowsShell form
  • ensure GDI is used in place of X11

@myselfhimself
Copy link
Owner Author

@paulhart2 is MSVC compilation a must for you? gmic-py would compile best on Windows using MSYS2 instead from what I have heard. Though I do not want to prevent so many Windows users from self-compiling gmic-py (I intend to release just source and as to Windows, gmic-py Windows binary pre-compiled packages)

@myselfhimself
Copy link
Owner Author

Based on this 2017 post on compiling G'MIC on Windows using either MSYS2+Mingw or MSVC, given that the latter MSVC option would probably force to disable OpenMP, I prefer not to even try using MSVC compilation... So Windows Python developers wanting to compile their own binding will have to follow compiling instructions or get inspiration from my upcoming compilation scripts.

I will let you know once I manage to compile gmic-py for Windows, hopefully this week.

@jfolz
Copy link

jfolz commented Mar 23, 2020

Python for Windows is compiled with a specific MSVC version (see Python wiki). The same MSVC version must be used to compile extensions. I'm not certain whether this applies to dynamically loaded libraries that are used by the extension.

@myselfhimself
Copy link
Owner Author

Thank you @jfolz , MingW is also possible up to Python 3.4 according to that same Python wiki.... We could hope that it actually does work for Python versions not just equal to but greater than 3.4.

Possible MSVC versions that I could use (if OpenMP eventually would want to work on it) on Github Actions (I do not to do the windows/packaging locally at all) are listed here for Windows Server 2016, 2019.

This will really be my first compiling a complex project for Windows in the "blind" of a remote CI toolchain. Keep suggesting ideas or try PRs (Github Actions do run on PRs as well) etc..

@jfolz
Copy link

jfolz commented Mar 23, 2020

Some alternative compilers exist like MinGW, but incompatibilities may occur with a CPython official distribution that is built with Microsoft Visual C++.

Essentially: there be dragons. Segfaults that suddenly occur in 3rd-party libraries, and thus almost impossible to debug.

@myselfhimself
Copy link
Owner Author

It has been many years already that gmic compiles and runs well with MSYS2/MINGW, Gimp for Windows also follows that path... Let us experiment, I have a pytest suite that runs for all OSes Windows users such as paulhart2 will be able to test for sure.

@myselfhimself
Copy link
Owner Author

Out of curiosity here is a github action to set up msys2
Most important for me is that final final library or wheel needs no extra depencencies apart from what's bundled.

@myselfhimself
Copy link
Owner Author

Windows tool similar to macos delocate and linux auditwheel repair, for including DLLs: https://discuss.python.org/t/need-auditwheel-like-utility-for-other-platforms/2028/6

@myselfhimself
Copy link
Owner Author

Related dependencies checker for windows: https://lucasg.github.io/2018/04/29/Dependencies-command-line/

@myselfhimself
Copy link
Owner Author

Focusing on this for today on until its finished.
I will ensure that the virtual machine for building and that for testing are different, namely so that the testing environment does not have any .dll to cheat.

@myselfhimself
Copy link
Owner Author

No, focusing on numpy for linux first. Sorry :)

@myselfhimself
Copy link
Owner Author

https://github.com/d0m3nik/pycimg/blob/master/.travis.yml shows a windows build using CI Build wheel elegantly, for a similar project (CImg is the backbone of GMic)

@myselfhimself
Copy link
Owner Author

@paulhart2 welcome aboard for attempting to compile gmic-py on Windows / MSYS2 by hand firsthand
I can then help transpose our efforts into config-as-a-code Github Workflow files containing build and release jobs for every new gmic-py version

@myselfhimself
Copy link
Owner Author

Good news, MSYS2 / pacman is already on the Windows images for Github Actions: https://github.com/actions/virtual-environments/blob/main/images/win/Windows2019-Readme.md

myselfhimself added a commit that referenced this issue Nov 13, 2020
myselfhimself added a commit that referenced this issue Nov 13, 2020
myselfhimself added a commit that referenced this issue Nov 13, 2020
myselfhimself added a commit that referenced this issue Nov 13, 2020
myselfhimself added a commit that referenced this issue Nov 13, 2020
myselfhimself added a commit that referenced this issue Nov 13, 2020
myselfhimself added a commit that referenced this issue Nov 13, 2020
myselfhimself added a commit that referenced this issue Nov 13, 2020
myselfhimself added a commit that referenced this issue Nov 13, 2020
@myselfhimself myselfhimself unpinned this issue Dec 22, 2020
@myselfhimself myselfhimself added enhancement New feature or request release Release management labels Apr 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request release Release management
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants