New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VTK Support #27

Open
wants to merge 8 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@banesullivan

banesullivan commented Jul 16, 2018

Synopsis: Add new VTK conversion methods to project elements.

Please consider this branch experimental in its current state. I want to get a pull request going to discuss whether these features are appropriate to have in the omf package.

The Visualization Toolkit (VTK) is a cross-platform, open-source library for data processing and visualization built on C++ and wrapped for use in Python. VTK provides a framework for scalable visualizations and powers many open source visualization software platforms like ParaView. I believe that adding VTK support to omf will enable geoscientists to directly pair the Open Mining Format with their VTK powered processing or visualization routines/software and provide a set of functionality that will work out of the box.

I initially began writing these conversion methods for the PVGeo repository to build an OMF project file reader for ParaView. After working a bit with the OMF package and file format, I began to realize I needed a means to directly convert the project elements to VTK data objects to use in some of my other processing routines outside of ParaView. Rather than importing an external package to call various conversions methods for each element’s type, I thought it’d be best to allow each element to convert itself within omf directly. Thus providing stable conversion methods for each ProjectElement and a place to start for using omf in VTK processing routines without the need for static file conversions.

New Features:

For every subclass of the ProjectElement (or ProjectElementGeometry when appropriate), I have added a method called toVTK() which will convert the element to a VTK data object that corresponds to its features/topology. This allows for users to iterate over a Project’s elements list and directly convert that element to a VTK data object like the example below:

Example:

>>> import omf

>>> # Read the test file in the repo's assets
>>> reader = omf.OMFReader('test_file.omf')
>>> project = reader.get_project()

>>> # Iterate over the elements and add converted VTK object to dictionary:
>>> data = dict()
>>> for e in project.elements:
>>>     data[e.name] = e.toVTK()

Now we have a dictionary of VTK data objects to use in VTK processing/visualization routines:

>>> data
{u'Basement': (vtkCommonDataModelPython.vtkUnstructuredGrid)0x10726e598,
 u'Block Model': (vtkCommonDataModelPython.vtkRectilinearGrid)0x10726e738,
 u'Cover': (vtkCommonDataModelPython.vtkUnstructuredGrid)0x10726e3f8,
 u'Dacite': (vtkCommonDataModelPython.vtkUnstructuredGrid)0x10726e530,
 u'Early Diorite': (vtkCommonDataModelPython.vtkUnstructuredGrid)0x10726e600,
 u'Intermineral diorite': (vtkCommonDataModelPython.vtkUnstructuredGrid)0x10726e668,
 u'Topography': (vtkCommonDataModelPython.vtkUnstructuredGrid)0x10726e4c8,
 u'collar': (vtkCommonDataModelPython.vtkPolyData)0x10726e258,
 u'wolfpass_WP_assay': (vtkCommonDataModelPython.vtkPolyData)0x10726e6d0}

Directly convert/check a single element:

>>> el = project.elements[-1] # the `Block Model` element
>>> el.name
Block Model
>>> grid = el.toVTK()
>>> grid
(vtkCommonDataModelPython.vtkRectilinearGrid)0x10726e870
>>> assert(grid.GetNumberOfCells() == el.geometry.num_cells)
>>> assert(grid.GetNumberOfPoints() == el.geometry.num_nodes)

Concerns

I realize that a goal of omf is likely to remain a lightweight tool and provide a simple API so that users can integrate the Open Mining Format with their python powered processing software. In my experience, VTK can be cumbersome to install on non-Unix operating systems and thus it might make omf less lightweight out of the box. For this reason, I have kept all vtk imports internally within the toVTK() methods to ensure that the package can be used entirely without VTK in the active environment.

I imagine that omf was built with the idea that functionality to use the ProjectElements would be implemented in external libraries, but I am making a proposition that VTK is widely used enough that a standard means of generating VTK data objects should be included with the OMF package.

Progress:

  • Line Set

    • Generate vtkPloyData with CELL data attributes
    • If subtype is a borehole: make cylinders instead of lines.
  • Point Set

    • Generate vtkPloyData with POINT data attributes
    • Handle textures
  • Surface

    • Generate vtkUnstructuredGrid for SurfaceGeometry
    • SurfaceGridGeometry:
      • vtkStructuredGrid if axis are not default XYZ cartesian
      • vtkRectilinearGrid if axis is aligned with Cartesian system
    • Add POINT data attributes
    • Handle textures
  • Volume

    • VolumeGridGeometry
      • vtkStructuredGrid if axis are not default XYZ cartesian
      • vtkRectilinearGrid if axis is aligned with Cartesian system
    • Add CELL data attributes

PVGeo

I also want to include here what this enables me to build in the PVGeo package. By providing these methods to generate VTK data objects for each element type, I have created a reader for ParaView that will directly read an OMF project file, allow a user to select the elements they want to load, then have these elements directly rendered by ParaView:

EXAMPLE SCENE

What this looks like within ParaView:

screen shot 2018-07-16 at 12 26 23 pm

screen shot 2018-07-16 at 12 19 57 pm

@fwkoch

This comment has been minimized.

Show comment
Hide comment
@fwkoch

fwkoch Jul 26, 2018

Member

Hey @banesullivan - Thank you for the contribution! Really exciting to see some interest. I haven't reviewed the code yet since it's still a work in progress, but OMF/VTK interoperability seems quite useful.

My only concerns are somewhat parallel to yours. Keeping OMF lightweight is a goal, so I appreciate that you are including VTK as non-required functionality that only "turns on" if VTK is available. However, more than that, this sets a precedent for other external data types. Where do we draw the line for what's widely used enough to warrant inclusion? My preference would be to create a standard way to build libraries to interface with OMF. For example make a cookiecutter repo that simplifies filling in toOMF/fromOMF for a certain data type. Then you can pip install omf-vtk (for example) and have all the OMF interoperability you need for that external data type. I don't think it would be a problem to host these OMF plugin libraries on the GMSGDataExchange github and give them the same first-class treatment as the base OMF library.

Anyway - I'd be curious what opinions other people may have about including VTK here.

And regardless, I'm so happy to see a contribution! I'll do what I can to help get VTK/OMF interop out into the world.

Member

fwkoch commented Jul 26, 2018

Hey @banesullivan - Thank you for the contribution! Really exciting to see some interest. I haven't reviewed the code yet since it's still a work in progress, but OMF/VTK interoperability seems quite useful.

My only concerns are somewhat parallel to yours. Keeping OMF lightweight is a goal, so I appreciate that you are including VTK as non-required functionality that only "turns on" if VTK is available. However, more than that, this sets a precedent for other external data types. Where do we draw the line for what's widely used enough to warrant inclusion? My preference would be to create a standard way to build libraries to interface with OMF. For example make a cookiecutter repo that simplifies filling in toOMF/fromOMF for a certain data type. Then you can pip install omf-vtk (for example) and have all the OMF interoperability you need for that external data type. I don't think it would be a problem to host these OMF plugin libraries on the GMSGDataExchange github and give them the same first-class treatment as the base OMF library.

Anyway - I'd be curious what opinions other people may have about including VTK here.

And regardless, I'm so happy to see a contribution! I'll do what I can to help get VTK/OMF interop out into the world.

@banesullivan

This comment has been minimized.

Show comment
Hide comment
@banesullivan

banesullivan Jul 26, 2018

Hi @fwkoch, thanks for checking this out!

I think you make an excellent point and I agree it would be a difficult line to draw for what external libraries should be in the base OMF package. So, I now think these VTK features will be best suited in their own interface package for which I would be happy to lead development and host here on GMSGDataExchange.

I do not have any experience with making cookiecutters but they seem like an excellent approach for building interfaces. Perhaps we could lay the groundwork for a cookiecutter in near future?

Once there's an OMF-VTK interface, building the PVGeo / ParaView project readers/writers will be a breeze so I'm excited to move this forward!

banesullivan commented Jul 26, 2018

Hi @fwkoch, thanks for checking this out!

I think you make an excellent point and I agree it would be a difficult line to draw for what external libraries should be in the base OMF package. So, I now think these VTK features will be best suited in their own interface package for which I would be happy to lead development and host here on GMSGDataExchange.

I do not have any experience with making cookiecutters but they seem like an excellent approach for building interfaces. Perhaps we could lay the groundwork for a cookiecutter in near future?

Once there's an OMF-VTK interface, building the PVGeo / ParaView project readers/writers will be a breeze so I'm excited to move this forward!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment