Support Windows for client software and RQD

[bcipriano]

The database and Cuebot will continue to run on Linux but all other pieces - PyCue, PyOutline, CueGUI, and RQD - should run on Windows.

RQD should in theory run on all platforms but there is some OS-specific code in there especially around system metrics collection.

[JohnWelborn]

#175 might indicate that google.protobuf.pyext._message is not available on windows

[bcipriano]

I've repurposed this Issue to focus on Windows and cover both client-side components as well as RQD.

[bcipriano]

For completeness, reporting some info from the forum here.

The latest update here is - unfortunately at the moment there's not great support for PySide when using Windows+Python 2.

https://wiki.qt.io/Qt_for_Python

After trying a few different things we came to the conclusion that without PySide planning any official build for Python 2, it made the most sense to move towards Python 3 compatibility instead, as the best way to get to full Windows support. Environments that still use Python 2 (e.g. Maya) tend to ship with a prebuilt Python+PySide anyway, so we should be pretty well covered at that point.

We've made some progress on this - see the linked PR above for example.

[salvaom]

Hello!

I've managed to get a semi-functional version of cuegui on Windows after making some minor python 3 compatibility changes to outline, I've got cuebot and the database on a docker instance but I'm having a hard time figuring out how to use the Jenkins/Docker containers provided to test the changes. Is it written anywhere how are they supposed to be used?

Thank you.

[bcipriano]

Hi Salvador! That's great to hear.

To be clear you're looking to test changes you made to the outline library, aka pyoutline, is that right? As far as I'm aware that library isn't used in CueGUI itself, but it is used in the CueSubmit application - am I correct in taking that to mean that you also have CueSubmit running on Windows?

[salvaom]

After a couple more modifications I am now able to open CueSubmit as well, yes! About CueGUI, I did have to make one small change that is to set in the setup.py zip_safe=False, without that it was not able to load the plug-ins embedded since they were inside an .egg file.

Right now I've got both working and connected to the database, but I'm having some issues with the RPC that will fail when submitting jobs.

To be clear, I'm interested in using the test suite in general, to be able to make changes in the code and feel confident that I'm not breaking anything.

Thank you.

[bcipriano]

This is a timely request, because we're in the process of cleaning up and reenabling the existing unit tests right now.

See for example https://github.com/imageworks/OpenCue/pull/240 which enables this for PyOutline - there are some unrelated testing issues we need to clear up there, but we hope to have that submitted tomorrow.

Our plan was to do something like: clean up and enable unit tests -> make the code Python3 compatible -> test on Windows and fix as needed. Sounds like we're on the same page with that - having functional unit tests makes the second and third steps WAY easier.

[salvaom]

Oh, very good! I'll keep a look out for the unit test issues then, thank you very much!

[j4qfrost]

I ran into an issue where the protobuf imports weren't being found on Windows and resolved the problem by adding from . to the front of each line. Is there a path variable I needed to set?

[bcipriano]

Hi @j4qfrost, are you running the code from a release or from a Git checkout? Python 2 or 3?

It sounds like you are hitting the problem described by this fix:

OpenCue/pycue/Dockerfile

Line 45 in 44a7804

RUN sed -i 's/^\(import.*_pb2\)/from . \1/' pycue/opencue/compiled_proto/*.py

This should be fixed automatically in the official releases which are built via these Docker files but if you're compiling the protos yourself the problem could still pop up.

[j4qfrost]

I had checked out v0.2.31 for stability since I was having issues with the master branch initially. Linux compiled fine. It took me a while to figure out how to fix such a trivial problem. I was looking for a bit less of a hacky solution, but beggars can't be choosers.

[bcipriano]

Yeah, it's a horrible hack. We're looking to see if there's some other way we can resolve this.

[ndeebook]

Before getting RQD working on windows it would be nice to be able to send jobs with CueSubmit first.
(Maybe I should open another issue for this ?)

I installed CueSubmit on Windows with Anaconda (installed PySide2 with conda, the rest with pip).
The UI works fine but when submitting a job: it fails when hitting outline/util.py get_uid().
I fixed it by returning some hardcoded int (if Windows). I don't know what would be the proper way of doing this.

[smith1511]

One item still remaining is the default editor used by CueGui to view logs.
Check RQD logging.

[bcipriano]

Windows is supported now. Closing this, any followup issues should be filed as new issues.