Update VTK #2146

danlipsa · 2016-12-01T22:39:46Z

No description provided.

danlipsa · 2016-12-01T22:41:08Z

Data in
CDAT/uvcdat-testdata#160

VCS in:
CDAT/vcs#112

VTK conda recipes:

git_rev: b101d863d524412b53f972e408ceaa8d5bed1122
git_url: https://gitlab.kitware.com/vtk/vtk.git

danlipsa · 2016-12-01T22:46:56Z

Fixes:
CDAT/vcs#47
CDAT/vcs#76
CDAT/vcs#33

danlipsa · 2016-12-01T22:57:54Z

@doutriaux1 @aashish24 Please review. Especially pay attention to the 400+ baseline changes. Not sure how do we update VTK. I included the md5 for the VTK I tested with (from yesterday).

danlipsa · 2016-12-06T15:29:17Z

@doutriaux1 @aashish24 Ping!
These are the tests failing on my linux:
4 - testAccuracy (Failed)
39 - flake8_vcs (Failed)
646 - flake8_Thermo (Failed)
669 - flake8_xmgrace (Failed)
I think the flake tests fail because of newer flake on conda. I don't know about testAccuracy.

doutriaux1 · 2016-12-06T15:59:50Z

@danlipsa we should look into the newer flake8, probably just a matter of adding a few rules to the excepted ones.

danlipsa · 2016-12-06T16:38:50Z

@doutriaux1 OK, I'll fix those.

danlipsa · 2016-12-06T16:54:45Z

@doutriaux1 They all pass. flake8 was not installed in my conda env.

danlipsa · 2016-12-13T18:10:27Z

@doutriaux1 @aashish24 Lets merge this! :-)
I rerun the tests: These are the failures:
4 - testAccuracy (Failed)
654 - /home/danlipsa/projects/uvcdat/build/cdms/Test/test_all_formats.py (Failed)

The last test reason is:
ncopen: filename "http://test.opendap.org/opendap/hyrax/data/nc/coads_climatology.nc": NetCDF: file not found

danlipsa · 2016-12-13T18:20:11Z

@doutriaux1 @aashish24 I can do it myself, I just need the go ahead!

aashish24 · 2016-12-13T19:45:01Z

+2 :LGTM

doutriaux1 · 2016-12-14T05:09:49Z

@chaosphere2112 @aashish24 I'm offline for another day at least, please take a careful look and let's have @chaosphere2112 merge in.

aashish24 · 2016-12-14T07:02:06Z

@doutriaux1 I would prefer the developers merge their branch. I am fine with @chaosphere2112 review the branch. This is master branch and not a release. if someone finds a new bug which is not captured by our testing then we can always fix them later. Let's merge it on Thursday as I need this branch for some of my changes.

aashish24 · 2016-12-14T07:35:37Z

@danlipsa I believe you will push another branch with GL2PS fixes after this one? Also, is travis broken ever since we moved our vcs to separate repo?

danlipsa · 2016-12-16T19:12:18Z

@doutriaux1 @chaosphere2112 Guys, any progress on reviewing this? My finger itches to press "Merge pull request".

aashish24 · 2016-12-19T21:52:15Z

@danlipsa I will review the baselines tomorrow and will run the tests on my linux. If both looks good to me, I will merge it.

aashish24 · 2016-12-19T21:52:37Z

@danlipsa I will review the baselines tomorrow and will run the tests on my linux. If both looks good to me, I will merge it.

Unless someone else beats me to it.

aashish24 · 2016-12-22T19:30:29Z

@danlipsa what's your process of running your tests right now?

danlipsa · 2016-12-22T19:38:14Z

This is what I use. Not all steps are necessary once are done the first time.

cd projects

Create uvcdat repo and build the conda env

mkdir uvcdat
cd uvcdat
git clone https://github.com/UV-CDAT/uvcdat.git src
mkdir build
cd build
cmake ../src
make -j10

See the name of the created conda env

conda env list
Currently the conda env is: v2.8-1-gbf17786

Delete the created conda env and create another one with the same name using conda-forge

conda env remove -n v2.8-1-gbf17786
conda create -n v2.8-1-gbf17786 -c conda-forge -c uvcdat uvcdat

Clone vcs repo and overwrite vcs in the conda env.

cd ..
mkdir vcs
cd vcs
git clone https://github.com/UV-CDAT/vcs.git src
cd src
source activate v2.8-1-gbf17786
python setup.py install --old-and-unmanageable

Rebuild a new VTK and instal it

Update the location for VTK
cd ~/projects/conda-recipes/src
Modify git_url and git_env in vtk-cdat/meta.yaml.in
Add -DCMAKE_BUILD_TYPE=Debug in vtk-cdat/build.sh to build a debug version of VTK
python prep_for_build.py
conda build vtk-cdat -c conda-forge
source activate v2.8-1-gbf17786
conda install --use-local -f vtk-cdat

Run the tests

cd ../../uvcdat/build
ctest -j6

aashish24 · 2016-12-22T19:41:57Z

thanks @danlipsa

danlipsa · 2016-12-22T19:43:22Z

@aashish24 Note I added a step for rebuilding VTK

aashish24 · 2017-01-03T17:03:08Z

@danlipsa I am at the last step of building everything, hoping to publish my results soon.

aashish24 · 2017-01-04T20:50:18Z

@danlipsa the new test_vcs_basic_boxfill_aeqd_proj_SH.png looks little off. Mind having a look at it?
also

test_vcs_basic_boxfill_aeqd_proj_SH.png
test_vcs_basic_boxfill_aeqd_proj_SH_via_gm.png
test_vcs_basic_isofill_0_proj_SH.png

aashish24 · 2017-01-04T20:50:47Z

aashish24 · 2017-01-04T21:45:11Z

@danlipsa I would suggest comparing the baselines in some viewer: I wrote this script (below) to do that

for i in ls *.png; do echo "<html><head></head><body><table><tr><td><img src=$i></img></td><td><img src=../../../uvcdat-testdata.new/baselines/vcs/$i></img></td></tr></table></body></html>" >> $i.html; done;

danlipsa · 2017-01-09T15:38:15Z

@aashish24 Yes, I am doing something similar. Do you see this difference on Linux, or a Mac?
Are you sure you are on the update-vtk branch for uvcdat-testdata as well as for uvcdat?

danlipsa · 2017-01-09T19:23:47Z

@aashish24 All these tests pass in 2.8 because of a _1 version with the blue-yellow color map. Now the _1 version was promoted to the main version as it should.

danlipsa · 2017-01-09T19:24:54Z

@aashish24 @doutriaux1 Can I merge this?

doutriaux1 · 2017-01-09T19:28:46Z

reviewing this afternoon.

aashish24 · 2017-01-09T20:37:26Z

@aashish24 All these tests pass in 2.8 because of a _1 version with the blue-yellow color map. Now the _1 version was promoted to the main version as it should.

Sounds good.

aashish24 · 2017-01-09T20:38:40Z

@aashish24 @doutriaux1 Can I merge this?

Let's give @doutriaux1 this afternoon to review 🤓 but I would suggest we merge it tomorrow morning. We need to move forward with some branches so unless we find any critical issue with this branch we merge it tomorrow.

doutriaux1 · 2017-01-10T01:06:13Z

Since github.com/CDAT/vcs/pull/112 is already merged i am merging this as well

doutriaux1 · 2017-01-10T01:08:52Z

@danlipsa shouldn't there be a vtk pr to merge as well? Also travis still fails....

danlipsa · 2017-01-10T14:44:27Z

@doutriaux1 the first comment has the SHA and url for the VTK I used in the tests. Do we need to merge anything in or we can use that in conda-receipes? I'll have to merge in the data as well.

danlipsa · 2017-01-10T15:02:55Z

@doutriaux1 @aashish24 conda-recipes/vtk-cdat/meta.yaml.in has
git_rev: master
git_url: git://github.com/Kitware/VTK.git

This includes the changes we want, but it is a moving target. I think we should use a specific SHA we tested with.

aashish24 · 2017-01-10T15:47:58Z

This includes the changes we want, but it is a moving target. I think we should use a specific SHA we tested with

+1 I was wondering about it as well.

doutriaux1 · 2017-01-10T15:58:01Z

@danlipsa we need to update our vtk repo and build a nightly.

doutriaux1 · 2017-01-10T16:03:41Z

@danlipsa I will update .travis.yaml to use nightly builds

danlipsa · 2017-01-10T16:54:32Z

@doutriaux1 Not sure I follow. What is our VTK repo? As I said conda-recipes/vtk-cdat/meta.yaml.in points to git://github.com/Kitware/VTK.git, master now.

aashish24 · 2017-01-10T16:55:03Z

@doutriaux1 we need to use a SHA vs the master. Using master of VTK repo is risky in some ways.

doutriaux1 · 2017-01-10T17:02:10Z

@aashish24 @danlipsa we have https://github.com/uv-cdat/VTK just for that purpose. @aashish24 created this. uvcdat-master is the branch that we should use for nightly I retag uvcdat-master wth each release.

danlipsa · 2017-01-10T18:11:55Z

@doutriaux1 @aashish24 OK, so I'll point that branch to the VTK SHA I tested with. We should probably change conda-recipes/vtk-cdat/meta.yaml.in to point to that as well isn't it?

danlipsa · 2017-01-10T19:06:57Z

@doutriaux1 @aashish24 I updated uvcdat-master on https://github.com/uv-cdat/VTK to point to the SHA I tested (VTK from Dec 22). @doutriaux1 Are you going to update conda-recipes or you want me to do it?

doutriaux1 · 2017-01-10T21:59:02Z

@danlipsa conda-recipe should point to uvcdat-master of https://github.com/uv-cdat/VTK which at this point I think should really be just master I dont see the point of having a uvcdat-master branch.

doutriaux1 · 2017-01-10T21:59:38Z

@danlipsa so I don't think conda-recipe needs an update.

danlipsa · 2017-01-11T19:21:40Z

@doutriaux1 We got used with uvcdat-master, so maybe we should leave it as is. Also master could be interpreted as VTK master which is not. So I vote to keep using uvcdat-master.

aashish24 · 2017-01-11T20:02:47Z

@danlipsa @doutriaux1 I think if we use uvcdat/vtk, using master is fine since we will only update the master when we know vtk original master is in good condition for UV-CDAT>

doutriaux1 · 2017-01-11T22:15:00Z

@danlipsa @aashish24 I'm fine either way. Let's revisit this next time we merge something from VTK into our VTK repo.

danlipsa changed the title ~~Generate test files with same names as baselines.~~ Update VTK Dec 1, 2016

danlipsa force-pushed the update-vtk branch 2 times, most recently from 9f87557 to 9e050b9 Compare December 5, 2016 20:26

Generate test files with same names as baselines and fix test.

bf17786

danlipsa force-pushed the update-vtk branch from 9e050b9 to bf17786 Compare December 6, 2016 15:01

doutriaux1 mentioned this pull request Jan 10, 2017

ENH: Save pdf files compressed. CDAT/vcs#112

Merged

doutriaux1 merged commit b01856c into master Jan 10, 2017

doutriaux1 deleted the update-vtk branch January 10, 2017 01:08

Update VTK #2146

Update VTK #2146

Conversation

danlipsa commented Dec 1, 2016

danlipsa commented Dec 1, 2016 • edited

danlipsa commented Dec 1, 2016 • edited

danlipsa commented Dec 1, 2016

danlipsa commented Dec 6, 2016

doutriaux1 commented Dec 6, 2016

danlipsa commented Dec 6, 2016

danlipsa commented Dec 6, 2016

danlipsa commented Dec 13, 2016

danlipsa commented Dec 13, 2016

aashish24 commented Dec 13, 2016

doutriaux1 commented Dec 14, 2016

aashish24 commented Dec 14, 2016

aashish24 commented Dec 14, 2016 • edited

danlipsa commented Dec 16, 2016

aashish24 commented Dec 19, 2016

aashish24 commented Dec 19, 2016

aashish24 commented Dec 22, 2016

danlipsa commented Dec 22, 2016 • edited

Create uvcdat repo and build the conda env

See the name of the created conda env

Delete the created conda env and create another one with the same name using conda-forge

Clone vcs repo and overwrite vcs in the conda env.

Rebuild a new VTK and instal it

Run the tests

aashish24 commented Dec 22, 2016

danlipsa commented Dec 22, 2016

aashish24 commented Jan 3, 2017

aashish24 commented Jan 4, 2017 • edited

aashish24 commented Jan 4, 2017

aashish24 commented Jan 4, 2017 • edited

danlipsa commented Jan 9, 2017 • edited

danlipsa commented Jan 9, 2017

danlipsa commented Jan 9, 2017

doutriaux1 commented Jan 9, 2017

aashish24 commented Jan 9, 2017

aashish24 commented Jan 9, 2017

doutriaux1 commented Jan 10, 2017

doutriaux1 commented Jan 10, 2017

danlipsa commented Jan 10, 2017

danlipsa commented Jan 10, 2017 • edited

aashish24 commented Jan 10, 2017

doutriaux1 commented Jan 10, 2017

doutriaux1 commented Jan 10, 2017

danlipsa commented Jan 10, 2017

aashish24 commented Jan 10, 2017

doutriaux1 commented Jan 10, 2017

danlipsa commented Jan 10, 2017

danlipsa commented Jan 10, 2017

doutriaux1 commented Jan 10, 2017

doutriaux1 commented Jan 10, 2017

danlipsa commented Jan 11, 2017

aashish24 commented Jan 11, 2017

doutriaux1 commented Jan 11, 2017

danlipsa commented Dec 1, 2016 •

edited

danlipsa commented Dec 1, 2016 •

edited

aashish24 commented Dec 14, 2016 •

edited

danlipsa commented Dec 22, 2016 •

edited

aashish24 commented Jan 4, 2017 •

edited

aashish24 commented Jan 4, 2017 •

edited

danlipsa commented Jan 9, 2017 •

edited

danlipsa commented Jan 10, 2017 •

edited