Add streamlines. #138

danlipsa · 2017-03-03T22:43:19Z

No description provided.

danlipsa · 2017-03-03T22:44:09Z

Associated PRs:
CDAT/cdat#2156
CDAT/uvcdat-testdata#162
Needs VTK 73fb094d17ccdd594a6abf9ff8732149c88dc2e6 or later. This has the mac highres fix as well.

danlipsa · 2017-03-03T22:44:49Z

@doutriaux1 @aashish24 Please review.

doutriaux1 · 2017-03-06T16:19:21Z

@danlipsa shouldn't streamlines have arrows?

aashish24 · 2017-03-06T16:23:16Z

This is one of the most recent beautiful picture I have seen from vcs @williams13 @doutriaux1

@danlipsa this looks very cool.... I have few questions but let me first look at the code and then I will ping .

jypeter · 2017-03-06T17:09:00Z

Has someone looked at the nice vcs based (not sure) streamlines in the vacumm package gallery? Somebody from our group used it last week. Lots of great stuff in Stephane's package :-) I'm in the train, so can't give you the exact link

…

On March 6, 2017 5:19:22 PM GMT+01:00, Charles Doutriaux ***@***.***> wrote: @danlipsa shouldn't streamlines have arrows? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #138 (comment)

-- JYP @ SGS 5

durack1 · 2017-03-06T17:12:46Z

@jypeter it's http://www.ifremer.fr/vacumm/ - we had that as one of the supported packages when things were being built from source.. Since moving to conda this should also be available to users - the conda install -c vacumm vacumm command should work

danlipsa · 2017-03-06T22:04:13Z

@doutriaux1 By default they don't, but I can add them. I can push something today or tomorrow.

doutriaux1 · 2017-03-06T22:34:55Z

@danlipsa that would be nice, and I think we probably want the arrows on by defaults. Also as @chaosphere2112 mentioned it would be nice to have the color of the streamline determined by the speed/norm of the vector they follow. What do you think?

aashish24 · 2017-03-07T00:33:24Z

Has someone looked at the nice vcs based (not sure) streamlines in the vacumm package gallery? Somebody from our group used it last week.

@jypeter cool thanks for sharing the info. vaccum looks similar to matplotlib (mpl) but mpl has scaling issues meaning that it can only rendering so many streamlines. Also, not sure about the API (feature comparison wise). On this note, how did he implement streamlines in vcs? Is it using UV-CDAT 2.8?

aashish24 · 2017-03-07T00:37:02Z

@chaosphere2112 mentioned it would be nice to have the color of the streamline determined by the speed/norm of the vector they follow. What do you think?

@danlipsa and I talked about coloring in general (as @danlipsa also mentioned it for vector plots as well). As for the color, I think we want to have color by a scalar, if we have arrows, then not sure we want to color by vector (direction of streamlines) although a vector could be applied for coloring (though will require a lookup table that works with vector). For now we could add color by scalar (vtk supports it).

glpotter · 2017-03-07T17:48:18Z

I have attached a figure that shows what we want. The streamlines need arrows or you don't know which direction the wind is blowing, we also must have a way of telling the magnitude. The grads example I have attached uses colors. These are the minimum requirements.

aashish24 · 2017-03-07T22:02:22Z

@glpotter this is great, thanks for sharing it. I would opt for one arrow (at least for now) since I think too many arrows make the plot crowded but having a option in the future for many arrows would be nice.

As far the color goes, we already captured that in our discussions I think.

jypeter · 2017-03-08T09:19:19Z

@durack1 you can see the example streamline code and plot (plot attached below) from vacumm in the gallery: http://www.ifremer.fr/vacumm/tests/test_plot_streamplot.html

Looks like it uses cdms2 for reading the data, and basemap for plotting

danlipsa · 2017-03-08T17:02:17Z

@aashish24 @doutriaux1 Take a look at the new streamline plots with arrows added at seeding points. One improvement I can see is that the triangles for the arrows are positioned with the tip at the anchor point. To look better they should be positioned with the triangle center at the anchor point. I'll add colors next.

doutriaux1 · 2017-03-08T18:29:43Z

vcs/streamline.py

+                except:
+                    pass  # oh well we stick to default value
+    # Tl
+    i = code.find("Tl")


@danlipsa you get yank out all the code for reading in streamlines from .scr files since they didn't exist before and we wont generate .scr files for them.

@doutriaux1 That is 'process_src' isn't it? I'll do that - thanks!

glpotter · 2017-03-09T21:26:04Z

@aashish24 One arrow head per streamline is not enough. The plots that @durack1 and I have attached are what we want to have.

aashish24 · 2017-03-09T21:37:35Z

@glpotter I was suggesting that it would be good to have an option. I personally think too many arrows gets crowded and make a streamline plot looks closer to a vector plot. I would think having some flexibility there would be helpful.

stickler-ci · 2017-03-14T15:51:09Z

vcs/Canvas.py

@@ -247,7 +247,7 @@ def _determine_arg_list(g_name, actual_args):
                 arglist[igraphics_method].g_nslabs))
    else:
        if arglist[igraphics_method].lower() in (
-                'scatter', 'vector', 'xvsy', 'stream', 'glyph', '3d_vector', '3d_dual_scalar'):
+                'scatter', 'vector', 'streamline', 'xvsy', 'stream', 'glyph', '3d_vector', '3d_dual_scalar'):


E501 line too long (109 > 79 characters)

stickler-ci · 2017-03-14T15:51:10Z

vcs/Canvas.py

@@ -1792,6 +1792,49 @@ def vector(self, *args, **parms):
        arglist = _determine_arg_list('vector', args)
        return self.__plot(arglist, parms)

+
+    def createstreamline(self, name=None, source='default'):


E303 too many blank lines (2)

stickler-ci · 2017-03-14T15:51:10Z

vcs/Canvas.py

+
+    def streamline(self, *args, **parms):
+        """
+        Generate a streamline plot given the data, streamline graphics method, and


E501 line too long (82 > 79 characters)

stickler-ci · 2017-03-14T15:51:11Z

vcs/Canvas.py

+    def streamline(self, *args, **parms):
+        """
+        Generate a streamline plot given the data, streamline graphics method, and
+        template. If no streamline class object is given, then the 'default' streamline


E501 line too long (87 > 79 characters)

stickler-ci · 2017-03-14T15:51:11Z

vcs/Canvas.py

+        """
+        Generate a streamline plot given the data, streamline graphics method, and
+        template. If no streamline class object is given, then the 'default' streamline
+        graphics method is used. Similarly, if no template class object is given,


E501 line too long (81 > 79 characters)

stickler-ci · 2017-03-14T15:51:12Z

vcs/Canvas.py

+            .. doctest:: canvas_streamline
+
+                >>> a=vcs.init()
+                >>> a.show('streamline') # Show all the existing streamline graphics methods


E501 line too long (92 > 79 characters)

stickler-ci · 2017-03-14T15:51:12Z

vcs/Canvas.py

+                >>> a.show('streamline') # Show all the existing streamline graphics methods
+                *******************Streamline Names List**********************
+                ...
+                *******************End Streamline Names List**********************


E501 line too long (82 > 79 characters)

doutriaux1 · 2017-03-27T21:35:41Z

@danlipsa do you mind preparing a jupyter notebook on how to use streamlines (with various options explained)? Thanks.

durack1 · 2017-03-27T21:37:53Z

@doutriaux1 the opacity problem visualized above was created with vtk-cdat-7.1.0.2017.3.23.73fb094d17ccdd594a6abf9ff8732149c88dc2e6-py27_0.json if that helps

doutriaux1 · 2017-03-27T21:43:15Z

@aashish24 @danlipsa among other verified issue with latest vtk most vtk_ui tests fail to draw lines

doutriaux1 · 2017-03-27T21:44:33Z

@danlipsa your baseline and the ones I run on my nox are widely different. Which machine did you generate the baselines on? Mac?

doutriaux1 · 2017-03-27T21:45:39Z

@danlipsa also I notice there's no legend associated with streamlines. @glpotter @williams13 is that an issue? Is there legends for streamlines?

doutriaux1 · 2017-03-27T22:42:34Z

@danlipsa ok this is weird, if i run your test as follow:

python /git/uvcdat/testing/vcs/test_vcs_basic_gms.py --source=./uvcdat-testdata/baselines/vcs/test_vcs_basic_streamline_transparent.png --gm_type=streamline --transparent

it passes, gm list gives:

 --------Streamline (Gs) member (attribute) listings --------
graphics method = Gs
name = __streamline_833884096041739
projection = linear
xticlabels1 = *
xticlabels2 = *
xmtics1 = 
xmtics2 = 
yticlabels1 = *
yticlabels2 = *
ymtics1 =  
ymtics2 =  
datawc_x1 = 1e+20
datawc_y1 =  1e+20
datawc_x2 =  1e+20
datawc_y2 =  1e+20
datawc_timeunits =  days since 2000
datawc_calendar =  135441
xaxisconvert =  linear
yaxisconvert =  linear
levels =  [[1.0000000200408773e+20, 1.0000000200408773e+20]]
ext_1 =  False
ext_2 =  False
fillareacolors = [1]
linecolor =  [100, 0, 0, 50.0]
linewidth =  None
reference =  1e+20
numberofseeds =  500
integratortype =  2
integrationdirection =  2
integrationstepunit =  2
initialsteplength =  0.2
minimumsteplength =  0.1
maximumsteplength =  0.5
maximumsteps =  200
maximumstreamlinelength =  0.25
terminalspeed =  0.1
maximumerror =  0.1
glyphscalefactor =  0.01
glyphbasefactor =  0.75
filledglyph =  True
coloredbyvector =  False
numberofglyphs =  1

if I run it inside nose, it fails but the gm list is identical!

python run_tests.py -v2  tests/test_vcs_basic_gms_2.py

gives:

 --------Streamline (Gs) member (attribute) listings --------
graphics method = Gs
name = __streamline_455988310493801
projection = linear
xticlabels1 = *
xticlabels2 = *
xmtics1 =
xmtics2 =
yticlabels1 = *
yticlabels2 = *
ymtics1 =
ymtics2 =
datawc_x1 = 1e+20
datawc_y1 =  1e+20
datawc_x2 =  1e+20
datawc_y2 =  1e+20
datawc_timeunits =  days since 2000
datawc_calendar =  135441
xaxisconvert =  linear
yaxisconvert =  linear
levels =  [[1.0000000200408773e+20, 1.0000000200408773e+20]]
ext_1 =  False
ext_2 =  False
fillareacolors = [1]
linecolor =  [100, 0, 0, 50.0]
linewidth =  None
reference =  1e+20
numberofseeds =  500
integratortype =  2
integrationdirection =  2
integrationstepunit =  2
initialsteplength =  0.2
minimumsteplength =  0.1
maximumsteplength =  0.5
maximumsteps =  200
maximumstreamlinelength =  0.25
terminalspeed =  0.1
maximumerror =  0.1
glyphscalefactor =  0.01
glyphbasefactor =  0.75
filledglyph =  True
coloredbyvector =  False
numberofglyphs =  1

files are:
Difference:

Source (baseline)

Nose test

glpotter · 2017-03-28T00:15:16Z

I’m not sure how you could tell the magnitude if there aren’t. Gerald L. (Jerry) Potter Ph.D. Data and Analysis Consultant Phone: 707 315-8314 email: jerry.potter@gmail.com

…

On Mar 27, 2017, at 2:45 PM, Charles Doutriaux ***@***.***> wrote: @danlipsa <https://github.com/danlipsa> also I notice there's no legend associated with streamlines. @glpotter <https://github.com/glpotter> @williams13 <https://github.com/williams13> is that an issue? Is there legends for streamlines? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#138 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ANjM_DDWoEHolZlllZm3_bkrxfCh0ZxPks5rqC4EgaJpZM4MS1O4>.

aashish24 · 2017-03-28T04:34:51Z

@danlipsa do you mind preparing a jupyter notebook on how to use streamlines (with various options explained)? Thanks.

+1 but Let's do it after we fix the circleci and any other issues that are pending for the release.

aashish24 · 2017-03-28T04:38:52Z

@danlipsa also I notice there's no legend associated with streamlines. @glpotter @williams13 is that an issue? Is there legends for streamlines?

this is a good point. I would add a legend only if we have colors that represents a scalar. wha you think @danlipsa

@doutriaux1 would it be okay if @danlipsa push that a separate PR? Or should be do in the same PR. I think the change might be small enough so I am fine either way.

aashish24 · 2017-03-28T04:41:55Z

@aashish24 @danlipsa among other verified issue with latest vtk most vtk_ui tests fail to draw lines

So are you saying that at point the vtk_ui tests started failing. But how that is related to this branch? If the new VTK is available in VCS / UV-CDAT master and if the tests are failing then I think we should file a bug and fix it separately since this branch or the VTK used by this BRANCH didn't break it (it was broken before). If the VTK this branch is bringing is causing the VTK_UI tests to fail, then we have some issue.

Although I think @chaosphere2112 disable few things from the vtk_ui but I do not remember all the details.

Fixes #135

danlipsa · 2017-03-28T13:27:29Z

@doutriaux1 @glpotter @williams13 @aashish24 There is a legend associated with streamlines colored by velocity magnitude. If there the streamlines are just one color there is no legend.

danlipsa · 2017-03-28T13:42:38Z

@danlipsa do you mind preparing a jupyter notebook on how to use streamlines (with various options explained)? Thanks.
+1 but Let's do it after we fix the circleci and any other issues that are pending for the release.

Do you have a link with other jupyter notebook examples? Are they linked from the main uvcdat page? I was thinking to add an example to the Galery.

danlipsa · 2017-03-28T14:12:04Z

@doutriaux1 With the new commits I have only these tests failing on my machine (Ubuntu 16.04).
flake8_vcs
test_vcs_read_old_scr
test_vcs_read_old_scr_2
vcs_test_dump_json
vcs_addons_test_histogram_inherit
/home/danlipsa/projects/uvcdat/build/cdms/Test/openCreateData.py
They are all valid failures which I think just fell through the cracks.

danlipsa · 2017-03-28T14:16:09Z

@doutriaux1 Note I have 2 extra fixes (from Sam and Sankhesh) in my branch which you have in your branch as well.

danlipsa · 2017-03-28T14:18:11Z

@doutriaux1 @aashish24 I would merge this into master and go onto fixing the remaining issues there. This way, you can build conda nightly and people can test streamlines. What do you think?

doutriaux1 · 2017-03-28T15:16:17Z

@danlipsa for jupyter notebook examples see:
https://uvcdat.llnl.gov/tutorials.html
@danlipsa except for your added test I have everything (except ui) passing in streamlines_plus_tests branch
@danlipsa ok, but let's merge streamlines_plus_tests then. So that we get the nightly and we can start new branch with the correct tests.

doutriaux1 · 2017-03-28T15:17:56Z

@aashish24 i agree that @danlipsa code did not break the ui, but this branch requires a VTK that breaks a few things. I'll open a separate issue.

danlipsa · 2017-03-28T15:44:40Z

@doutriaux1 @aashish24 All vtk_ui pass on my machine with the new VTK (Ubuntu 16.04).

durack1 · 2017-03-28T16:29:50Z

@danlipsa does the current test suite include an opacity test?

danlipsa · 2017-03-28T16:47:41Z

@durack1 There are several transparency/opacity tests that pass, but they might test something different than what you've seen:
[~/projects/uvcdat/build]$ ctest -R 'transparency|opacity'
Test project /home/danlipsa/projects/uvcdat/build
Start 68: test_vcs_fillarea_transparency
1/4 Test #68: test_vcs_fillarea_transparency ...... Passed 3.45 sec
Start 179: test_vcs_missing_opacity_isofill
2/4 Test #179: test_vcs_missing_opacity_isofill .... Passed 4.29 sec
Start 180: test_vcs_missing_opacity_boxfill
3/4 Test #180: test_vcs_missing_opacity_boxfill .... Passed 4.40 sec
Start 181: test_vcs_missing_opacity_meshfill
4/4 Test #181: test_vcs_missing_opacity_meshfill ... Passed 4.33 sec

100% tests passed, 0 tests failed out of 4

How can I reproduce your test? Is it the same script/data as in
#121

durack1 · 2017-03-28T17:29:38Z

@danlipsa yep the #121 data and example is what you'd need.. The example above also pulls in the EzTemplate to plot the dual panels.. I've just had some system problems, but could forward you the script if it was helpful

danlipsa · 2017-03-28T17:35:34Z

@durack1 Yes, please attach the script here so that I can compare with what you get.

durack1 · 2017-03-28T19:47:48Z

@doutriaux1 should that script (demo requested by @danlipsa above) go in another open issue?

danlipsa · 2017-03-28T20:11:20Z

@durack1 Its fine to add it here.

doutriaux1 · 2017-03-28T20:20:11Z

@durack1 I think it would go fine there: #155

danlipsa added 2 commits March 8, 2017 11:52

ENH: Add streamlines.

8d57202

ENH: Add arrows to streamlines

6fae9b0

danlipsa force-pushed the streamline branch from b1f5786 to 6fae9b0 Compare March 8, 2017 16:54

doutriaux1 reviewed Mar 8, 2017

View reviewed changes

danlipsa force-pushed the streamline branch from 85f01fe to de1d0c5 Compare March 10, 2017 21:46

ENH: Color streamlines by velocity magnitude.

f73c21a

danlipsa force-pushed the streamline branch from de1d0c5 to f73c21a Compare March 13, 2017 16:57

stickler-ci reviewed Mar 14, 2017

View reviewed changes

Proper tracking of color array index in meshfill pipeline

a6523bf

Fixes #135

Patches the issue, fix #136

1e19632

doutriaux1 closed this Mar 28, 2017

doutriaux1 deleted the streamline branch March 28, 2017 19:43

durack1 mentioned this pull request Mar 28, 2017

Newest VTK breaks some tests #155

Closed

Add streamlines. #138

Add streamlines. #138

Conversation

danlipsa commented Mar 3, 2017

danlipsa commented Mar 3, 2017 • edited

danlipsa commented Mar 3, 2017

doutriaux1 commented Mar 6, 2017

aashish24 commented Mar 6, 2017

jypeter commented Mar 6, 2017 via email

durack1 commented Mar 6, 2017 • edited

danlipsa commented Mar 6, 2017

doutriaux1 commented Mar 6, 2017

aashish24 commented Mar 7, 2017

aashish24 commented Mar 7, 2017

glpotter commented Mar 7, 2017

aashish24 commented Mar 7, 2017

jypeter commented Mar 8, 2017 • edited

danlipsa commented Mar 8, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

glpotter commented Mar 9, 2017

aashish24 commented Mar 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

doutriaux1 commented Mar 27, 2017

durack1 commented Mar 27, 2017

doutriaux1 commented Mar 27, 2017

doutriaux1 commented Mar 27, 2017

doutriaux1 commented Mar 27, 2017

doutriaux1 commented Mar 27, 2017

glpotter commented Mar 28, 2017 via email

aashish24 commented Mar 28, 2017

aashish24 commented Mar 28, 2017

aashish24 commented Mar 28, 2017

danlipsa commented Mar 28, 2017

danlipsa commented Mar 28, 2017 • edited

danlipsa commented Mar 28, 2017 • edited

danlipsa commented Mar 28, 2017

danlipsa commented Mar 28, 2017

doutriaux1 commented Mar 28, 2017

doutriaux1 commented Mar 28, 2017

danlipsa commented Mar 28, 2017

durack1 commented Mar 28, 2017

danlipsa commented Mar 28, 2017

durack1 commented Mar 28, 2017

danlipsa commented Mar 28, 2017

durack1 commented Mar 28, 2017

danlipsa commented Mar 28, 2017

doutriaux1 commented Mar 28, 2017

danlipsa commented Mar 3, 2017 •

edited

durack1 commented Mar 6, 2017 •

edited

jypeter commented Mar 8, 2017 •

edited

danlipsa commented Mar 8, 2017 •

edited

danlipsa commented Mar 28, 2017 •

edited

danlipsa commented Mar 28, 2017 •

edited