ED2 met process failing #1742
Assignees
Comments
|
The FLUXNET download a previous time and seemed to work but it appears to be crashing on met2model using h5 libs OR something is wrong with the data format? |
Generated nc files Looks accurate data: |
|
Just two ED2 files? |
Seem just to be empty? |
|
Also a problem because the previous failed file exists but is empty so it moved to the next month. Which means once this works, it may skip these two months, where are actually invalid. I will need to run this function and force re-build to purge these two bad files. Another option? |
|
I had a similar problem that was fixed by updating the hdf5 R package (which required a bunch of other updates too). Yes, also had to manually delete empty .h5 files during crashes like this. |
|
Thanks @ankurdesai I will give that a shot! |
|
Following these instructions for update: https://www.bioconductor.org/packages/devel/bioc/vignettes/rhdf5/inst/doc/rhdf5.pdf My problem is that it ins't using the HDF5 version that is loaded in my modules but instead finding an old HDF system lib....so that may cause me all sorts of problems. |
|
Yeah, trying to run again or manually causes me problems I think I need to try and install rhdf5 using the more up-to-date HDF5 I have installed but as a module. Its not yet clear how to get bioclite to install using a different lib location. I don't see any flags to do this. Perhaps there is a way to build from source? |
|
Here's what I did in rstudio: Then this worked: |
|
@ankurdesai I did the same, still fails. Part of my issue (I think) Thats the wrong HDF5 and I am trying to get it to use the correct one....so far failing to use a different HDF5 lib location I really hate this bioconductor / HDF5 dependency for ED2 |
|
Or is that somehow the HDF5 that ships with rhdf5? |
That was my own attempt at running that by "hand" after installing rhdf5 again |
|
Hmmm...hang on. If I run not using the remote function and just met2model directly, it seems to work except it tries to run years more than once? |
Files are being generated but why the warnings about the files already existing? |
It looks like we might be trying to interact with a file in a strange way during the process? |
|
Also, just to confirm. The way the function is trying to run in the web workflow using remotefunc() doesn't work But if I run PEcAn.ED2::met2model.ED2 directly it works OK, except with the warnings mentioned above. Also when running as I get one file created in the output directory but fails when populating.... Does anyone have any idea why the error when running as wrapped in the remotefunc? I can't see why that would matter? |
|
OK, weird new info. If I am trying to run PEcAn.ED2::met2model.ED2 by hand without first running library(rhdf5) it fails, but if I first load rhdf5 and then run the command it works fine?! Does anyone know what that would be? We are using the rhdf5::h5write() syntax in the function so shouldn't that be OK? |
|
Yeah, that seems to be the case: So why do I need to run library(rhdf5) first to get this to work if we are using rhdf5::h5write? |
|
OK, for whatever reason, I could only get this to work via web or locally by editing met2model.ED2 and adding library(rhdf5) to the top of the function? Here is output from teh web after doing this: |
|
Like so Why? |
|
That suggests there's one or more function calls that are missing the rhdf5 namespace |
|
Hmm....looking at the rhdf5 NAMESPACE, I can see the h5write being exported |
|
Possibly relevant: mschubert/r-h5store#5 |
|
Yes, looks like the same problem but that is from 2015 and it looks like the functions are in the NAMESPACE so not sure why I am still having that issue |
|
@serbinsh I didn't mean that rhdf5 wasn't exporting a function, I meant check to make sure that everywhere we're using the functions within the ed2 module that we're calling the functions by namespace. |
|
Hey @serbinsh I am also getting ED2 met processing failures for FLUXNET, but also AmerifluxLBL. I'm trying to see if it's for parallel reasons to you, but if not I can open another issue. |
|
@serbinsh Can confirm, I am getting the exact same problem with AmerifluxLBL |
serbinsh
pushed a commit
to serbinsh/pecan
that referenced
this issue
Nov 3, 2017
…roject#1742. Hard to test work around for more than one site due to other issues with metprocess
|
OK, based on a comment by @ashiklom I found this to work for me: This worked even with the library(rhdf5) removed from the function, as well as the importFrom's |
serbinsh
pushed a commit
to serbinsh/pecan
that referenced
this issue
Nov 3, 2017
|
Addressed in #1757 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm trying to setup runs at NOBS (1850 burn site, Manitoba Canada) and running into issues using the FLUXNET drivers
The text was updated successfully, but these errors were encountered: