New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't load data in uvcdat 2.0.0 ! #843

Closed
jypeter opened this Issue Oct 24, 2014 · 29 comments

Comments

Projects
None yet
7 participants
@jypeter

jypeter commented Oct 24, 2014

I have tried to load the first time step of clt from sample_data/clt.nc and I got

Missing package: gov.llnl.uvcdat.cdms

See below for full traceback and screenshot

Notes:

  • I get this with any netcdf file, not just clt.nc
  • I only get the error message for the first Load and Close. Subsequent Load and Close do not do anything (nothing happens, not even an error, nothing is loaded)

Interestingly, I get a similar error (but slightly different traceback) if I try to do this by hand in the calculator! See the 2nd attached traceback+screenshot

Loading a var from the point-and-click GUI

Traceback (most recent call last):
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/gui/uvcdat/variable.py", line 594, in defineVarCloseClicked
    self.getUpdatedVarCheck()
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/gui/uvcdat/variable.py", line 630, in getUpdatedVarCheck
    self.getUpdatedVar(tid)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/gui/uvcdat/variable.py", line 729, in getUpdatedVar
    updatedVar = controller.create_exec_new_variable_pipeline(targetId)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/gui/uvcdat/project_controller.py", line 1185, in create_exec_new_variable_pipeline
    self.get_var_module(targetId, dummyCell, CDMSPipelineHelper)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/gui/uvcdat/project_controller.py", line 925, in get_var_module
    module = var.to_module(self.vt_controller)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/packages/uvcdat_cdms/init.py", line 163, in to_module
    module = Variable.to_module(self, controller, identifier)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/packages/uvcdat/init.py", line 57, in to_module
    reg.get_descriptor_by_name(pkg_identifier, self.__class__.__name__))
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/core/modules/module_registry.py", line 745, in get_descriptor_by_name
    raise MissingPackage(identifier)
MissingPackage: Missing package: gov.llnl.uvcdat.cdms

uvcdat-dataloadpb

Loading a var from the calculator

Traceback (most recent call last):
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/gui/uvcdat/commandLineWidget.py", line 597, in run_command
    tmp = prj_controller.create_exec_new_variable_pipeline(varname)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/gui/uvcdat/project_controller.py", line 1185, in create_exec_new_variable_pipeline
    self.get_var_module(targetId, dummyCell, CDMSPipelineHelper)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/gui/uvcdat/project_controller.py", line 939, in get_var_module
    varms.append(self.get_var_module(v, cell, helper, var_dict))
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/gui/uvcdat/project_controller.py", line 948, in get_var_module
    opvar)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 235, in build_variable_operation_pipeline
    op_module = op_class_inst.to_module(controller)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/packages/uvcdat_cdms/init.py", line 945, in to_module
    module = CDMSVariableOperation.to_module(self, controller)
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/packages/uvcdat_cdms/init.py", line 457, in to_module
    reg.get_descriptor_by_name(identifier, self.__class__.__name__))
  File "/home/share/unix_files/cdat/versions/cdat_install_uv-2.0.0_x86_64_gcc4_12/vistrails/vistrails/core/modules/module_registry.py", line 745, in get_descriptor_by_name
    raise MissingPackage(identifier)
MissingPackage: Missing package: gov.llnl.uvcdat.cdms

uvcdat-dataloadpb02

@ThomasMaxwell

This comment has been minimized.

Contributor

ThomasMaxwell commented Oct 24, 2014

Jean,
Did you build Uvcdat from source? I encountered this error in a recent build of the master branch on Mac OSX. Can someone else verify?

@jypeter

This comment has been minimized.

jypeter commented Oct 24, 2014

Yes I did build from source, tag 2.0.0. But was of the warning/error I got at the cmake stage and reported in #825 was related to vistrails and there are lots of references to vistrails in the traceback above. Could it be that I don't have vistrails, or not the correct vistrails? How can I check that?

@doutriaux1 doutriaux1 added this to the 2.1 milestone Oct 27, 2014

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Oct 27, 2014

@remram44 these are now the second and third independent reports of this. Can you take a look please? As @jypeter mentinos it might be due to checking out vistrails. @jypeter did you try to remove your ~/.uvcdat directory? It might help.

@jypeter

This comment has been minimized.

jypeter commented Oct 27, 2014

It does help, I can now load the clt data! Thanks!
I had tried to start from a clean slate and removed an old PCMDI_GRAPHICS directory, and an old .vistrail (or something like this) directory, but I had missed .uvcdat

Does all the configuration stuff go to .uvcdat? This should be documented...

Let me know if I have to restore the content of my old .uvcdat directory from our automated backup and send it to you, if you need to see what I had in there that prevented the correct loading of the data

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Oct 27, 2014

@mattben yes we should document this somewhere

@remram44

This comment has been minimized.

Contributor

remram44 commented Oct 27, 2014

UV-CDAT inherited the package manager & configuration from VisTrails. Packages that are disabled in the configuration won't be loaded (unless you enable it, but UV-CDAT has no GUI to do that). A package that fails to load (although that shouldn't happen) will display an error message and then get disabled in the configuration file, so it won't be loaded the next time.

@ThomasMaxwell

This comment has been minimized.

Contributor

ThomasMaxwell commented Oct 27, 2014

I'm seeing this problem in a MacOSX 10.8 build of the uvcdat master branch. Deleting the ~/.uvcdat file does not solve the problem. It is caused by the following error when I attempt to load a variable:

Traceback (most recent call last):
  File "/Developer/Projects/EclipseWorkspace/uvcdat/master/build/install/vistrails/vistrails/gui/uvcdat/variable.py", line 594, in defineVarCloseClicked
self.getUpdatedVarCheck()
  File "/Developer/Projects/EclipseWorkspace/uvcdat/master/build/install/vistrails/vistrails/gui/uvcdat/variable.py", line 630, in getUpdatedVarCheck
self.getUpdatedVar(tid)
  File "/Developer/Projects/EclipseWorkspace/uvcdat/master/build/install/vistrails/vistrails/gui/uvcdat/variable.py", line 722, in getUpdatedVar
axesOperations=str(axes_ops_dict))
  File "/Developer/Projects/EclipseWorkspace/uvcdat/master/build/install/vistrails/vistrails/packages/uvcdat_cdms/init.py", line 130, in __init__
Variable.__init__(self, filename, url, source, name, load)
TypeError: expected string or buffer

@remram44 remram44 removed their assignment Oct 27, 2014

@ThomasMaxwell

This comment has been minimized.

Contributor

ThomasMaxwell commented Oct 28, 2014

I'm now getting this same error in a CentOS build of the uvcdat branch master with vistrails branch uvcdat-master. It occurs when loading a variable from a file in the GUI. Is anyone else seeing this? Can someone check on this?

@williams13

This comment has been minimized.

Contributor

williams13 commented Oct 28, 2014

Yes, I am getting this too! How could this had happened?!

-Dean

@remram44

This comment has been minimized.

Contributor

remram44 commented Oct 28, 2014

I don't see this (on debian), I'm updating to master and rebuilding (2.0.0-25-g29394e5 -> 2.0.0-54-g4029ad4)

@aashish24

This comment has been minimized.

Contributor

aashish24 commented Oct 28, 2014

if this is a gui specific issue, then we don't test GUI as of now. Its very easy to get bad code into uvcdat-master.

@ThomasMaxwell I belive that command like still working?

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Oct 28, 2014

I don't think this is GUI specific because the tag 2.0.0 didn't change
uvcdat-master seems to still be at 2.0.0

@remram44

This comment has been minimized.

Contributor

remram44 commented Oct 28, 2014

This is annoying, because I see no way the line Variable.__init__(self, filename, url, source, name, load) could raise a TypeError, it's a pure-Python method call. I can't reproduce this here, I'll try on OS X later.

@remram44

This comment has been minimized.

Contributor

remram44 commented Oct 28, 2014

Sorry I don't see this on my OS X build either (2.0.0-25-g29394e5).

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Oct 28, 2014

and you built from master? I was able to see it on both my 10.8.5 and 10.9.5 macs. I load the "Standard" clt.nc variable clt and click "Load and Close"

@williams13

This comment has been minimized.

Contributor

williams13 commented Oct 29, 2014

From the splash screen, I am getting (2.0.0-54.g4029ad4).

This is a different version that what you hare getting below.

@williams13

This comment has been minimized.

Contributor

williams13 commented Oct 29, 2014

We can detect what has been added more recently to the repository? This will help us in two ways: (1) we can isolate the code and determine why it is affecting the GUI (then we can determine if we want to remove the code and or fix it); (2) help the team to submitted code properly and create a test for this in CTEST.

@remram44

This comment has been minimized.

Contributor

remram44 commented Oct 29, 2014

Alright, I'm getting this on OS X, if I build 4029ad4. I thought you were using 2.0.0...

Will investigate.

@remram44

This comment has been minimized.

Contributor

remram44 commented Oct 29, 2014

On this version, uvcdat_cdms doesn't even enable because of a VCS bug:

Traceback (most recent call last):
  File "/Users/remirampin/Documents/programming/uvcdat/build-2.0.0-54-g4029ad4/install/vistrails/vistrails/gui/preferences.py", line 314, in enable_current_package
    pm.late_enable_package(codepath)
  File "/Users/remirampin/Documents/programming/uvcdat/build-2.0.0-54-g4029ad4/install/vistrails/vistrails/core/packagemanager.py", line 397, in late_enable_package
    raise e
InitializationFailed: Package 'UVCDAT CDMS' failed to initialize because of the following exceptions:
Traceback (most recent call last):
  File "/Users/remirampin/Documents/programming/uvcdat/build-2.0.0-54-g4029ad4/install/vistrails/vistrails/core/modules/module_registry.py", line 1434, in initialize_package
    package.initialize()
  File "/Users/remirampin/Documents/programming/uvcdat/build-2.0.0-54-g4029ad4/install/vistrails/vistrails/core/modules/package.py", line 388, in initialize
    self._init_module.initialize()
  File "/Users/remirampin/Documents/programming/uvcdat/build-2.0.0-54-g4029ad4/install/vistrails/vistrails/packages/uvcdat_cdms/init.py", line 2248, in initialize
    gm = getattr(canvas,method_name)(gmname)
  File "/Users/remirampin/Documents/programming/uvcdat/build-2.0.0-54-g4029ad4/install/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/vcs/Canvas.py", line 1394, in gettaylordiagram
    return vcs.gettaylordiagram(Gtd_name_src)
  File "/Users/remirampin/Documents/programming/uvcdat/build-2.0.0-54-g4029ad4/install/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/vcs/manageElements.py", line 300, in gettaylordiagram
    for m in vcs.taylordiagrams:
AttributeError: 'module' object has no attribute 'taylordiagrams'
@durack1

This comment has been minimized.

Member

durack1 commented Oct 29, 2014

@doutriaux1 looks to me like folks aren't getting/testing the 2.0.0 tagged version.. "master" is no longer in sync with the 2.0.0 tag.. Your 2.0.0-XX-hash would suggest to me that master is 54 commits beyond 2.0.0?

@remram44

This comment has been minimized.

Contributor

remram44 commented Oct 29, 2014

Ok, got it. The other traceback (TypeError) is related to the uvcdat_cdms package failing to load. Because it fails, it gets disabled, thus unloaded, thus its globals get set to None. Variable is None when Variable.__init__(...) gets executed, so None.__init__ is called, and raises a TypeError.

Please fix the VCS error above.

@remram44 remram44 added VCS and removed UVCDAT GUI labels Oct 29, 2014

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Oct 29, 2014

pushing a fix for this along with a test case for it. Just hold on for 10minutes.

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Oct 29, 2014

should be fixed by #852 can someone review test and approve.Note this will NOT fix the case of loading uvcdat_cdms failing that will not be reloaded the next time around. I guess at that point you will have to remove your .uvcdat.

@remram44

This comment has been minimized.

Contributor

remram44 commented Oct 29, 2014

If uvcdat_cdms doesn't fail to load, it will be loaded the next time around.

The fact that the package gets used even though it was disabled is also a bug.

@williams13

This comment has been minimized.

Contributor

williams13 commented Oct 29, 2014

This is really a serious problem!

I just built v2.0.0: git clone --branch uvcdat-2.0.0 https://github.com/UV-CDAT/uvcdat.git

And I still cannot load a data variable from the GUI!

Jerry and I tested this before v2.0.0 was released. Are we sure that v2.0.0 got tagged properly?

-Dean

@ThomasMaxwell

This comment has been minimized.

Contributor

ThomasMaxwell commented Oct 29, 2014

I saw the problem in builds of a recent uvcdat master HEAD version but when I build the uvcdat-2.0.0 tagged version the problem did not appear. I got the tagged version of the code using the command:

git checkout tags/uvcdat-2.0.0

@remram44

This comment has been minimized.

Contributor

remram44 commented Oct 29, 2014

Didn't see that in 2.0.0 or 2.0.0-25-g29394e5, but we all had it on 2.0.0-54-g4029ad4.

@williams13

This comment has been minimized.

Contributor

williams13 commented Oct 29, 2014

Okay, I was able to get v2.0.0 to read in data… Charles and David reminded me that I had to remove the .uvcdat directory. Whew!!! :-)

-Dean

From: , Dean Williams <williams13@llnl.govmailto:williams13@llnl.gov>
Date: Wednesday, October 29, 2014 10:44 AM
To: UV-CDAT/uvcdat <reply@reply.github.commailto:reply@reply.github.com>, UV-CDAT/uvcdat <uvcdat@noreply.github.commailto:uvcdat@noreply.github.com>
Subject: Re: [uvcdat] Can't load data in uvcdat 2.0.0 ! (#843)

This is really a serious problem!

I just built v2.0.0: git clone --branch uvcdat-2.0.0 https://github.com/UV-CDAT/uvcdat.git

And I still cannot load a data variable from the GUI!

Jerry and I tested this before v2.0.0 was released. Are we sure that v2.0.0 got tagged properly?

-Dean

From: Aashish Chaudhary <notifications@github.commailto:notifications@github.com>
Reply-To: UV-CDAT/uvcdat <reply@reply.github.commailto:reply@reply.github.com>
Date: Tuesday, October 28, 2014 3:05 PM
To: UV-CDAT/uvcdat <uvcdat@noreply.github.commailto:uvcdat@noreply.github.com>
Cc: Dean Williams <williams13@llnl.govmailto:williams13@llnl.gov>
Subject: Re: [uvcdat] Can't load data in uvcdat 2.0.0 ! (#843)

if this is a gui specific issue, then we don't test GUI as of now. Its very easy to get bad code into uvcdat-master.

@ThomasMaxwellhttps://github.com/ThomasMaxwell I belive that command like still working?


Reply to this email directly or view it on GitHubhttps://github.com//issues/843#issuecomment-60840161.

@williams13

This comment has been minimized.

Contributor

williams13 commented Oct 29, 2014

Did you remove your .uvcdat directory? Once I did this, I was able to load data from the GUI.

@doutriaux1 doutriaux1 closed this Nov 10, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment