Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: Update for latest VTK #1050

Merged
merged 4 commits into from Oct 18, 2021
Merged

Conversation

larsoner
Copy link
Contributor

Closes #1049

@opoplawski can you see if this fixes your build hang on Fedora 34? It at least "fixes" the hang for me on Windows. Plotting still doesn't work there (mlab.test_plot3d()) despite pyvistaqt.BackgroundPlotter working okay, so there is more to fix. But this might already be good enough progress in that it will prevent total pip install hangs for people using the VTK pre wheels. (They'll have a problem later when they try to actually plot, but that's at least easier to diagnose than an empty build log that took an hour of CI time).

@larsoner
Copy link
Contributor Author

It would also be good to add a CI job that tests with pip install --upgrade --pre vtk, but that should be another PR since I'm not really familiar with the CI setup here

@opoplawski
Copy link

@larsoner It seems to fix the hang I was seeing, thanks.

@mjg
Copy link

mjg commented Jun 29, 2021

On Fedora 34 with Mayavi-4.7.3 including this fix here, vtk-9.0.1 and python 3.9.5, Mayavi2 (the app) now starts and plots but throws some error (see below). from mayavi import mlab still core dumps (even with ETS_TOOLKIT=wx).

2021-06-29 12:39:44.250 (  40,375s) [        FD47C740]     vtkOpenGLState.cxx:505   WARN| Error glBindFramebuffer1 OpenGL errors detected
  0 : (1282) Invalid operation

 with stack trace of
0x7f8aee7ae67e : ??? [(???) ???:-1]
0x7f8aee7af3a2 : vtksys::SystemInformation::GetProgramStack[abi:cxx11](int, int) [(libvtksys.so.1) ???:-1]
0x7f8aec605089 : ??? [(???) ???:-1]
0x7f8aec606b0e : vtkOpenGLState::vtkBindFramebuffer(unsigned int, vtkOpenGLFramebufferObject*) [(libvtkRenderingOpenGL2.so.1) ???:-1]
0x7f8aee2e97fd : vtkRenderWindow::Render() [(libvtkRenderingCore.so.1) ???:-1]
0x7f8aec5e7582 : vtkOpenGLRenderWindow::Render() [(libvtkRenderingOpenGL2.so.1) ???:-1]
0x7f8aec6a6e47 : vtkXOpenGLRenderWindow::Render() [(libvtkRenderingOpenGL2.so.1) ???:-1]
0x7f8aea7667ad : ??? [(???) ???:-1]
0x7f8afd8e0f68 : ??? [(???) ???:-1]
0x7f8afd8d2de7 : _PyObject_MakeTpCall [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8cfcee : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8cf73f : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8dfa92 : ??? [(???) ???:-1]
0x7f8ab8b742fa : ??? [(???) ???:-1]
0x7f8ab8b743e3 : ??? [(???) ???:-1]
0x7f8ab3d6f649 : ??? [(???) ???:-1]
0x7f8ae865885e : QWidget::event(QEvent*) [(libQt5Widgets.so.5) ???:-1]
0x7f8ab3d7095b : ??? [(???) ???:-1]
0x7f8ae8617e73 : QApplicationPrivate::notify_helper(QObject*, QEvent*) [(libQt5Widgets.so.5) ???:-1]
0x7f8ab3b385b6 : ??? [(???) ???:-1]
0x7f8ae7b89f48 : QCoreApplication::notifyInternal2(QObject*, QEvent*) [(libQt5Core.so.5) ???:-1]
0x7f8ae865093a : QWidgetPrivate::sendPaintEvent(QRegion const&) [(libQt5Widgets.so.5) ???:-1]
0x7f8ae86514df : QWidgetPrivate::drawWidget(QPaintDevice*, QRegion const&, QPoint const&, QFlags<QWidgetPrivate::DrawWidgetFlag>, QPainter*, QWidgetRepaintManager*) [(libQt5W
idgets.so.5) ???:-1]
0x7f8ae8651d52 : QWidgetPrivate::paintOnScreen(QRegion const&) [(libQt5Widgets.so.5) ???:-1]
0x7f8ae8651e88 : QWidgetPrivate::syncBackingStore() [(libQt5Widgets.so.5) ???:-1]
0x7f8ae8658f8d : QWidget::event(QEvent*) [(libQt5Widgets.so.5) ???:-1]
0x7f8ab3d7095b : ??? [(???) ???:-1]
0x7f8ae8617e73 : QApplicationPrivate::notify_helper(QObject*, QEvent*) [(libQt5Widgets.so.5) ???:-1]
0x7f8ab3b385b6 : ??? [(???) ???:-1]
0x7f8ae7b89f48 : QCoreApplication::notifyInternal2(QObject*, QEvent*) [(libQt5Core.so.5) ???:-1]
0x7f8ae7b8cc76 : QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) [(libQt5Core.so.5) ???:-1]
0x7f8ae7bd6c57 : ??? [(???) ???:-1]
0x7f8aec0374cf : g_main_context_dispatch [(libglib-2.0.so.0) ???:-1]
0x7f8aec08b4e8 : ??? [(???) ???:-1]
0x7f8aec034c03 : g_main_context_iteration [(libglib-2.0.so.0) ???:-1]
0x7f8ae7bd66f8 : QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) [(libQt5Core.so.5) ???:-1]
0x7f8ae7b889b2 : QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) [(libQt5Core.so.5) ???:-1]
0x7f8ae7b90544 : QCoreApplication::exec() [(libQt5Core.so.5) ???:-1]
0x7f8ab3b39b2e : ??? [(???) ???:-1]
0x7f8afd8e0f68 : ??? [(???) ???:-1]
0x7f8afd8d2de7 : _PyObject_MakeTpCall [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8cfcee : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8caa5a : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8df9f1 : ??? [(???) ???:-1]
0x7f8afd8cf73f : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8df9f1 : ??? [(???) ???:-1]
0x7f8afd8cf73f : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8c96dd : ??? [(???) ???:-1]
0x7f8afd8d745e : _PyFunction_Vectorcall [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8df9f1 : ??? [(???) ???:-1]
0x7f8afd8cf73f : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8caa5a : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8c96dd : ??? [(???) ???:-1]
0x7f8afd9460e5 : _PyEval_EvalCodeWithName [(libpython3.9.so.1.0) ???:-1]
0x7f8afd94607d : PyEval_EvalCodeEx [(libpython3.9.so.1.0) ???:-1]
0x7f8afd94602f : PyEval_EvalCode [(libpython3.9.so.1.0) ???:-1]
0x7f8afd9722e4 : ??? [(???) ???:-1]
0x7f8afd96e566 : ??? [(???) ???:-1]
0x7f8afd84621a : ??? [(???) ???:-1]
0x7f8afd968ac3 : PyRun_SimpleFileExFlags [(libpython3.9.so.1.0) ???:-1]
0x7f8afd965ec8 : Py_RunMain [(libpython3.9.so.1.0) ???:-1]
0x7f8afd938add : Py_BytesMain [(libpython3.9.so.1.0) ???:-1]
0x7f8afd619b75 : __libc_start_main [(libc.so.6) ???:-1]
0x55f2fbd2509e : _start [(python3) ???:-1]

@larsoner
Copy link
Contributor Author

VTK just released 9.0.2, and all my pip install commands are stalled on Windows. I'm going to work around it by using my branch, but I expect this to be a (somewhat hard to diagnose) timeout problem for a lot of folks!

@larsoner
Copy link
Contributor Author

@mjg for your error I would open a bug with the VTK folks (of course after some searching), it's possible they will recognize something from the traceback:

https://gitlab.kitware.com/vtk/vtk/-/issues

Alternatively you could engage in a time consuming git bisect + wheel build + test to figure out the VTK commit that caused the problem, e.g. what pyvista folks did here:

pyvista/pyvista#1394 (comment)

@bcalvr
Copy link

bcalvr commented Oct 12, 2021

What's the latest on this please? I can't install mayavi, it just seems to hit a wall and get stuck!

Copy link
Member

@prabhuramachandran prabhuramachandran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank @larsoner! This looks good to me, I will refactor the print debug statements in a separate PR.

@prabhuramachandran
Copy link
Member

@larsoner -- I apologize for the delay in getting this done. I am going to try to get a bit more regular on this going forward.

@larsoner
Copy link
Contributor Author

Even on this branch on 9.1.0-rc2 wheels on macOS at least I get:

Failed on Logger
(#3 of 11 nodes, #18 of 32 subnodes)

Traceback (most recent call last):
...
  File "/Users/larsoner/python/mayavi/tvtk/wrapper_gen.py", line 749, in _gen_get_set_methods
    if not meths[vtk_attr_name] and get_sig[0][1]:
IndexError: list index out of range

and then after fixing that:

Failed on InteractorObserver
(#4 of 11 nodes, #187 of 367 subnodes)
...
  File "/Users/larsoner/python/mayavi/tvtk/indenter.py", line 238, in get_method_doc
    name = camel2enthought(orig_name)
  File "/Users/larsoner/python/mayavi/tvtk/common.py", line 175, in __call__
    if ret[0] == '_':
IndexError: string index out of range

I can push a commit to work around (fix?) those here or in a separate PR @prabhuramachandran , let me know which is better

@larsoner
Copy link
Contributor Author

... and then after fixing those:

../mayavi/tvtk/pyface/tvtk_scene.py:28: in <module>
    VTK_VER = tvtk.Version().vtk_version
E   AttributeError: 'Version' object has no attribute 'vtk_version'

@larsoner
Copy link
Contributor Author

Clearly my workarounds are not good, or something else is wrong :(

>>> tvtk.Version()._vtk_obj.GetVTKVersion()
'9.1.0'

@larsoner
Copy link
Contributor Author

And this exists:

(Pdb) p tvtk.Version().get_vtk_version()
'9.1.0'

So I'm not sure where/why the traits-based wrapper is failing :(

@prabhuramachandran
Copy link
Member

@larsoner -- I will merge this for now to make life easier for everyone, I will look at the additional failures for 9.1.0 tomorrow and will not be able to look at this today. Looking at the errors, it looks like 9.1.0 changed the docstring signature which is going to break many things for sure. Thanks for bringing this up. My first goal is to get 9.0.3 all set and the tests going there and then I will look at 9.1.0-rc2. Am hoping that by next weekend things will be in better shape.

@prabhuramachandran
Copy link
Member

@rahulporuri -- I am not able to merge this since the tests are required and don't have the time to sort this out today, so if you are able to merge this, please do so so I can build on top of this to fix additional issues. Thanks.

@codecov-commenter
Copy link

Codecov Report

Merging #1050 (e00c6de) into master (e360db9) will decrease coverage by 0.00%.
The diff coverage is 0.00%.

❗ Current head e00c6de differs from pull request most recent head 4e7f3b3. Consider uploading reports for the commit 4e7f3b3 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1050      +/-   ##
==========================================
- Coverage   49.83%   49.83%   -0.01%     
==========================================
  Files         263      263              
  Lines       23885    23887       +2     
  Branches     3248     3249       +1     
==========================================
  Hits        11903    11903              
- Misses      11210    11211       +1     
- Partials      772      773       +1     
Impacted Files Coverage Δ
tvtk/vtk_parser.py 90.00% <0.00%> (-0.54%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7b007d1...4e7f3b3. Read the comment docs.

@mdickinson
Copy link
Member

I've merged the master branch into this one; that should sort out the test requirements.

@mdickinson mdickinson merged commit e5b612c into enthought:master Oct 18, 2021
@@ -640,7 +643,10 @@ def _find_get_set_methods(self, klass, methods):

# Find the default and range of the values.
if gsm:
# Useful for debugging on failures:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For future evolution of this code, how about using logging in place of the commented-out print statements?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is primarily used at build time, so would logging be appropriate? I was thinking of using an environment variable like "MAYAVI_DEBUG_BUILD" or so to turn on these print statements. What do you recommend?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, but just from a code cleanliness point of view I'd really like to get away from having commented-out code in the codebase.

A "verbose" or "debug" flag to turn on the print statements doesn't seem unreasonable; that's probably better than using an environment variable at this point - we probably want the environment variable handling in the topmost layer.

@prabhuramachandran
Copy link
Member

@mdickinson -- who am I to talk to about the admin privileges? Until this point I was merging some of the PRs but I am not sure if it is something on my end or in the setup of the repo.

@mdickinson
Copy link
Member

@prabhuramachandran The inability to merge was purely related to the PR, not to you. I wasn't being allowed to merge either. :-) Everyone in the "enthought" team in the "enthought" org currently has write privileges on this repository, which should allow merging PRs.

The merge was blocked because the expected GitHub Actions test runs hadn't passed, and the branch protections that we recently put in place require those test runs to pass. And the test runs didn't pass because they weren't running, which in turn was because the connected GitHub Actions workflow was present in the master branch but not in this branch. That's why the merge from master into this branch fixed things. We're likely to see the same effect with other open PRs that haven't had a merge from master recently. The fix should be the same for those PRs: merge master into the branch.

@mdickinson
Copy link
Member

@prabhuramachandran Re: admin privileges, @rahulporuri should be able to push that through on our side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build hangs on Fedora 34
8 participants