New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

D4.12: JUPYTER extension for 3D visualisation, demonstrated with computational fluid dynamics #86

Closed
minrk opened this Issue Sep 8, 2015 · 112 comments

Comments

@minrk
Contributor

minrk commented Sep 8, 2015

Tasks:

  • evaluate existing 3d vis tools and produce landscape report
  • identify specific use cases we want to make sure are met
  • demonstrate displaying VTK data in the notebook (rendered in the browser)
  • demonstrate displaying large VTK data in the notebook (rendered server-side)

@minrk minrk added this to the D4.12 milestone Sep 8, 2015

@minrk minrk self-assigned this Nov 3, 2015

@nthiery nthiery modified the milestones: Month 24: 2017-08-31, D4.12 Mar 22, 2016

@fangohr

This comment has been minimized.

Contributor

fangohr commented Jul 4, 2016

Hi visualisation people. I understand that Simula (Min @minrk ), Logilab (Florent @fcayre), and Silesia (Marcin @marcinofulus) are key players? Southampton acts as providing user requirement (Marijan @mb1a15 and Hans @fangohr).

I wanted to connect the various discussions that are going on. With Marcin, I have had some emails regarding their work based on K3D. A demo can be seen at http://mybinder.org/repo/K3D-tools/K3D-jupyter (need to use Chrome to make it work, at least Safari doesn't work at the moment).

With Min and Florent we had discussions at Bremen about requirements and technologies. Marijan @mb1a15 is preparing a repository with test data. VTK is the standard we need to support in some way(s) to cater for a wide range of science and engineering 3d visualisation.

During the meeting in Bremen, we agreed that we need a write up of possible technologies to render these images in the notebook (@fcayre - I think you said you were going to write this?).

Broadly speaking, we could see two options for interactive (VTK-style) visualisation in the notebook:

(i) to convert the VTK scene into (open-gl) HTML. This gives instant responses of the scene when we try to rotate it. We have seen examples for this in the K3D work and in the mesh visualisation of fenics that I saw a couple of weeks ago at the NGCM Summer Academy in Southampton. This is great (and will cater for a very large part of our requirements). This is unlikely to work for large data files with many elements as the html/memory requirements may become too big.

(ii) carry out some rendering on the server, create a bitmap and display this. Sense mouse interaction (which requests to rotated the scene, say), then re-render the frame on the server and send the bitmap to the notebook. This will be less responsive, but works for larger data sets. [Somebody say they had a prototype].

I hope this is a useful starting point - can I hand over to Florent to write the overview of technologies and Min to coordinate activities?

@fangohr

This comment has been minimized.

Contributor

fangohr commented Jul 4, 2016

Hi all, I'd like to introduce you to Prabhu Rachmandran (@prabhuramachandran) who is the original author of the Mayavi visualisation package. I promised him to keep him up to date on our work. I think he said he had some interest in running Mayavi (which is based on VTK) in the notebook. His contributions (even if only thoughts and advice) would be very useful of course.

@marijanbeg

This comment has been minimized.

Contributor

marijanbeg commented Jul 4, 2016

Some examples of data we are interested in visualising in Southampton are now in the new repository visualisation-requirements.

@minrk

This comment has been minimized.

Contributor

minrk commented Jul 5, 2016

@fangohr thanks! That's a good summary. For the interactive stuff, we have K3D, X3D, and three.js, and perhaps more tools to look through. Prabhu's idea sounds similar to what matplotlib has done with their recently added nbagg backend.

@prabhuramachandran

This comment has been minimized.

prabhuramachandran commented Jul 6, 2016

FWIW, Mayavi can export X3D, mlab.savefig('file.x3d') will work out of the box, this can be easily modified to not generate a file. The major downside is that if there are many polygons it tends to be slow. I just experimented a bit with x3dom.js so am not sure this is the best way to do it but it isn't fast enough on my demos.

@prabhuramachandran

This comment has been minimized.

prabhuramachandran commented Jul 6, 2016

@minrk -- will you be around at SciPy 2016?

@minrk

This comment has been minimized.

Contributor

minrk commented Jul 7, 2016

I will, we can talk about ideas there.

@marcinofulus

This comment has been minimized.

Contributor

marcinofulus commented Jul 7, 2016

K3D-tools aimed for an interactive widget with "high" level functionalities (matplotlib style). The point is to prvide the iteractivity - like live update of data available (as in the demo).

There are some browsers limitations (memory) which will prohibit the display of really big data, but for that purpose there is an option of server side preprocessing. We will build on the K3D CFD visualisation, which will use server side scross sections and pathlines which will be transfered to the 3d widget, rather than postprocessed in javascript from raw data.

@prabhuramachandran

This comment has been minimized.

prabhuramachandran commented Jul 21, 2016

FWIW, I've added support for x3d based rendering to Mayavi and it works well on jupyter notebooks. See enthought/mayavi#415. This will be merged soon. I'll look at other options later but this is a good start as it is pretty functional.

@fcayre

This comment has been minimized.

fcayre commented Nov 10, 2016

Hi there, @martinRenou has been working on this at Logilab, results can be seen here with a live demo here. Martin is currently working on a Jupyter widget.

Any feedback very welcome. Thanks to @mb1a15 and @fangohr for the data and requirements.

@defeo

This comment has been minimized.

Contributor

defeo commented Nov 10, 2016

My 2 cents: Sage devs have been working for a looong time on three.js integration, and this seems to be almost ready now: https://trac.sagemath.org/ticket/12402

@fcayre

This comment has been minimized.

fcayre commented Nov 10, 2016

@defeo good to know, thanks.

Current ticket is so general that a lot of very different use cases fit in.

Southampton's requirements are pretty clear and similar to ours: 3D visualization of mesh-based computational data, that's why we are focusing on this use case, and there is a lot of work to be done for browser application to get close to what mayavi and paraview have been doing for years.

@marcinofulus

This comment has been minimized.

Contributor

marcinofulus commented Dec 3, 2016

@fcayre Do you think that K3D-jupyter and SciviJS could possibly benefit from each other?

@fcayre

This comment has been minimized.

fcayre commented Dec 5, 2016

@marcinofulus the fact that both K3D and scivijs are based on Threejs makes it at least possible to mix their objects in the same Threejs scene, which could be pretty interesting. I can think of a few use cases, but yours are very welcome.

@martinRenou

This comment has been minimized.

martinRenou commented Dec 8, 2016

We improved our SciviJS demo: https://demo.logilab.fr/SciviJS/
You can see that it's really interactive. We will (very) soon be able to include this GUI on Jupyter as an ipywidget with a simple python function call in our coming-up ipyscivijs extension. And also support vtk as a data input format.

@gouarin

This comment has been minimized.

Contributor

gouarin commented Dec 8, 2016

It's a really impressive work. Do you plan to open the code soon ? I would like to know what you have done and probably give my help if it's possible.
For example, I can work on the HDF5 support if it is not already done.

@martinRenou

This comment has been minimized.

martinRenou commented Dec 8, 2016

Probably this month, that would be great !

@marcinofulus

This comment has been minimized.

Contributor

marcinofulus commented Jan 23, 2017

Hi

@fangohr - can you give me a couble of representative vtk files from MM simulation which you might want to be displayed in notebook?

mk

@minrk

This comment has been minimized.

Contributor

minrk commented Mar 16, 2017

We're starting to focus on this deliverable, and I think we need to set some more specific goals than we have in the grant.

I think the first thing we need to do is a survey of the existing tools, and how well they serve specific use cases. For that, I think we need to collect a few use cases to explore. Can @fangohr and some others provide a few examples of 3D vis cases that they would like to run in notebooks? Then @vidartf and others can explore the current landscape of 3D vis in Jupyter in a short report, which I think will be useful to the community at large.

The visualisation-examples repo is a great start. If any other use cases can be added there, especially cases other than "display this vtk scene", that would be great.

Known tools for 3d vis in Jupyter notebooks today (it's a lot more than existed when we wrote the grant!):

Given this, I doubt that creating a new package is something we want to do for this deliverable. Instead, it probably makes the most sense to identify and pursue existing packages that best serve the current use cases, and contribute to them where they can be improved.

threejs seems to be a common baseline for most of these on the browser side. I think it's providing to be the d3 of web 3d.

@marcinofulus

This comment has been minimized.

Contributor

marcinofulus commented Mar 16, 2017

@vidartf

This comment has been minimized.

Member

vidartf commented Mar 27, 2017

Sorry for the delay, but I have completed a review of the existing packages.

Please consider the report as a short summary of the different current tools, to help make an informed decision about how to best proceed. I wanted to keep it brief to make it something that can quickly be skimmed through, but please let me know if there is any other information you feel is important to make a good decision.

@bpilorget

This comment has been minimized.

Contributor

bpilorget commented Jan 11, 2018

#88 was merged in #86 after the 3rd amendment to the contract

@marcinofulus

This comment has been minimized.

Contributor

marcinofulus commented Jul 2, 2018

Hi

We organize on 16-18.07 in Katowice (Poland) a small meeting devoted to the K3D-jupyter development and its integration and future.

Anybody who would like to get involved into K3D-development or create/test examples and applications is welcome.

There will be no strict agenda but we will start from architecture overview on the first day and then we will do sprints.

Marcin

@minrk

This comment has been minimized.

Contributor

minrk commented Aug 2, 2018

@marcinofulus @vidartf I have pushed the first parts of the report on this deliverable to the repo. If you could fill in some info on contributions to K3D and pythreejs/unray/ipydatawidgets/widgets-in-general, that would be very helpful. Some illustrative screenshots of the various tools in action would also be appropriate for this report.

@nthiery

This comment has been minimized.

Contributor

nthiery commented Aug 24, 2018

Hi @minrk, @vidartf, @marcinofulus,
I just went briefly through the report, and read in more details the beginning. I made some edits and added some TODO's. This is taking a good shape; I am looking forward the full report!
Early next week would be great.
Thanks,

@minrk

This comment has been minimized.

Contributor

minrk commented Aug 27, 2018

@nthiery thanks for the review! I'm filling them in. Most should be addressed now.

@marcinofulus and @fcayre could I get an estimate of PMs that you have spent on K3D and scivijs? I'm going to assume it was as-planned in the proposal (4 for Silesia, 10 for Logilab), unless I hear otherwise.

@nthiery

This comment has been minimized.

Contributor

nthiery commented Aug 27, 2018

@SylvainCorlay

This comment has been minimized.

SylvainCorlay commented Aug 27, 2018

Quick notes on widgets and 3-D in particular:

  • plotly migrated to Jupyter Widgets for their Jupyter integration in the latest release (3.0). This is a pretty important vote of confidence for the framework, and probably worth mentioning in a report.
  • kitware also adopted ipywidgets for the itk widget project, which includes 3-D visualization.

Concerning people: @martinRenou is currently with @QuantStack also doing widgets work. (He iterated on ipyleaflet, xwidgets & xleaflet), and we currently also collaborate jupyter widgets (including ipyvolume) with @maartenbreddels.

@marcinofulus

This comment has been minimized.

Contributor

marcinofulus commented Aug 28, 2018

@minrk

This comment has been minimized.

Contributor

minrk commented Aug 28, 2018

@marcinofulus thanks!

@nthiery a couple of figures need to be added, but the text should be reviewable

@nthiery

This comment has been minimized.

Contributor

nthiery commented Aug 29, 2018

Done reviewing. The report is indeed looking good and close to completion. Thanks @minrk, @vidartf and @marcinofulus!

Since we merged D4.14 (#88) Computational Fluid dynamics visualisation in web notebook into this deliverable, I did some edits to bring more focus on the CFD work. Please proofread.

All the cool features that are described or alluded to for CFD really deserve more inspiring screenshots. And possibly some diagrams explaining things like live visualization of remote computations occurring on GPU, or how to integrate K3D in a VTK pipeline. E.g. as we wrote above: demonstrate displaying VTK data in the notebook (rendered in the browser); demonstrate displaying large VTK data in the notebook (rendered server-side).

@minrk

This comment has been minimized.

Contributor

minrk commented Aug 29, 2018

@nthiery excellent, thanks for the review! I think we are just about set, then. @marcinofulus do you perhaps have another screenshot or diagram to add for the CFD? If so, please add them and then this report should be complete.

@marcinofulus

This comment has been minimized.

Contributor

marcinofulus commented Aug 30, 2018

@nthiery - I have added some better figs for CFD with captions.

@Juanlu001

This comment has been minimized.

Contributor

Juanlu001 commented Aug 30, 2018

A comment from the peanut gallery - is the draft report publicly viewable somewhere? I can as well wait for the final version, you know :)

@embray

This comment has been minimized.

Collaborator

embray commented Aug 30, 2018

@Juanlu001 You should be ale to find it in the repository under: https://github.com/OpenDreamKit/OpenDreamKit/blob/master/WP4/D4.12/report.tex ; last updated an hour ago.

@Juanlu001

This comment has been minimized.

Contributor

Juanlu001 commented Aug 30, 2018

Thanks!

@nthiery

This comment has been minimized.

Contributor

nthiery commented Aug 30, 2018

You can also access a recent pdf from the "(Final) report" link on top of this page.

@nthiery

This comment has been minimized.

Contributor

nthiery commented Aug 31, 2018

Submitted!

@nthiery nthiery added the Submitted label Aug 31, 2018

@nthiery

This comment has been minimized.

Contributor

nthiery commented Aug 31, 2018

Thanks everyone the soft behind, the software and the really cool pictures :-)

I am looking forward this report as a blog post on opendreamkit.org; it will attract attention!

@marcinofulus

This comment has been minimized.

Contributor

marcinofulus commented Sep 7, 2018

We are experimenting with programatical screenshots, and there is a nice side effect - a movie of an animation renderend with high resolution and fps:

https://www.youtube.com/watch?v=ZBYP1hxwUcc

@nthiery

This comment has been minimized.

Contributor

nthiery commented Sep 7, 2018

@IzabelaFaguet

This comment has been minimized.

Contributor

IzabelaFaguet commented Dec 7, 2018

Accepted by the EU on December 5th.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment