Outfile Dimensions #303

wietsefranssen · 2015-10-19T09:13:14Z

We have added lat, lon and time variables in the NetCDF output files. One can choose between 1D and 2D lat/lon variables by setting COORD_DIMS_OUT in the global parameter file.

Adapted time dimension in NetCDF output:
Number of timesteps was based on MODEL_STEPS_PER_DAY in the global parameter file.
Now the number of timesteps is based on OUTPUT_STEPS_PER_DAY in the global parameter file.

…e/outfile_time * 'develop' of https://github.com/UW-Hydro/VIC: fix rounding / precision bug at midnight with subdaily timestep add python 3.5 to travis print traceback when log_err is called

…able in NetCDF output

jhamman · 2015-11-10T20:34:57Z

vic/drivers/image/include/vic_driver_image.h

@@ -63,6 +70,7 @@ typedef struct {
    size_t n_nx; /**< size of x-index; */
    size_t n_ny; /**< size of y-index */
    location_struct *locations; /**< locations structs for local domain */
+    location_grid_struct *locations_grid; /**< locations structs for full unmasked latlon domain */


I'm wondering if there is a better way to do this. Perhaps we could create a location_struct for all grid cells, even if they are not being run. We would just need to add an active (or something similar) member to the location_struct. When we do the decomposition, we would only split up active grid cells so nothing would change below the MASTER_PROC level.

That is possible, but keep in mind that in that case the indexes are changing. I mean the following:
In the case you suggest, location_struct is bigger because the unmasked gridcells are now included. This means that location_struct[n] refers to another lonlat combination. We have to be very careful that this will not effect other parts of the VIC code.
What shall we do now to solve it?

Okay, let's leave this as is here and we'll revisit it in #313.

jhamman · 2015-11-10T21:01:27Z

@wietsefranssen - good start here. I made a bunch of comments to keep this moving.

jhamman · 2015-11-10T23:14:12Z

vic/drivers/image/src/get_global_domain.c

+    global_domain->locations_grid = (location_grid_struct *)
+                                    malloc(
+        global_domain->n_ny * global_domain->n_nx *
+        sizeof(location_grid_struct));


see #257 and update the cast syntax.

I changed it into (made also a calloc of it):
global_domain->locations_grid = calloc( global_domain->n_ny * global_domain->n_nx, sizeof(*global_domain->locations_grid));
I just used the same syntax as a few lines earlier eg line 76. Why do these one still use the old cast syntax? Or just because you didn't push the last commit to github yet?

That PR is still in the works (see #257).

Okay I'll wait for a next commit :)

…ts (cast syntax).

bartnijssen · 2015-11-14T05:36:55Z

@wietsefranssen : Can you resolve the merge conflicts before I merge this? Since you started this branch a while ago, you probably need to merge the existing develop into your branch first. After that you probably won't have any merge conflicts.

jhamman · 2015-11-14T05:41:10Z

@bartnijssen - I don't think we're ready to merge here yet. We still need to resolve the calendar and time handling pieces.

bartnijssen · 2015-11-14T05:42:52Z

OK - sounds good - still need to resolve the conflict though.

derived time units from 'global_param' to output NetCDF. fixed time init by adding 'initialize_time()' to 'vic_init.c'

jhamman · 2015-11-17T16:24:09Z

vic/drivers/image/src/vic_init_output.c

+        sprintf(strUnit, "days");
+    }
+    else {
+        sprintf(strUnit, "-");


better to raise an error here if we don't know what the time units are.

jhamman · 2015-11-17T16:27:38Z

vic/drivers/image/src/vic_write.c


    grid_size = global_domain.n_ny * global_domain.n_nx;

    // allocate memory for variables to be stored
-    dvar = (double *) malloc(local_domain.ncells * sizeof(double));
+    dvar = calloc(local_domain.ncells, sizeof(*dvar));


did you mean to change this from malloc to calloc?

Yes, but it was a mistake of me while adapting the cast syntax you mentioned. I am used to use calloc in stead of malloc (calloc is a bit slower because it it zeroing out memory is'nt it?). anyway I changed it to:

dvar = malloc(local_domain.ncells * sizeof(*dvar));

Also some minor adjustments

jhamman · 2015-11-19T19:09:32Z

vic/drivers/image/src/get_global_domain.c

+initialize_location_grid(location_grid_struct *location_grid)
+{
+    location_grid->latitude = 0;
+    location_grid->longitude = 0;


can you use MISSING instead of 0 here?

jhamman · 2015-11-19T19:43:53Z

@wietsefranssen - almost there. I'm working on #312 today.

…e/outfile_time

…file

wietsefranssen · 2015-12-17T16:01:20Z

@jhamman Hi Joe, I just made some changes in order to retreive the number of dimensions from the domain file. I also removed the global_domain_grid struct and replaced them for ncells_total.
It is still perfect yet (need to tidy up and need fix one mpi error).
In case you find some time, it would be nice to have a quick look at it to give some suggestions.

jhamman · 2015-12-24T07:04:57Z

vic/drivers/image/src/get_nc_varndimensions.c

+ *****************************************************************************/
+int
+get_nc_varndimensions(char *nc_name,
+                 char *var_name)


the indentation looks off here.

jhamman · 2015-12-24T07:20:31Z

This looks pretty good. I had a few minor comments. Once you've addressed those, please ping @bartnijssen and @tbohn for their code reviews.

… number or coordinate dims. Also a minor bug fix in vic_write.c

wietsefranssen · 2016-01-08T15:36:46Z

@jhamman I just fixed the things Joe suggested. @tbohn and @bartnijssen Can you please have a look at the code? Thank you beforehand.

jhamman · 2016-01-23T20:54:26Z

@tbohn and @bartnijssen - last call for comments here.

@wietsefranssen - can you merge in the current develop branch?

I'll give this one last read through and then merge if there are no failing tests and no objections by others.

jhamman · 2016-01-25T01:16:17Z

this pr closes #297

…e/outfile_time

Outfile Dimensions

jhamman · 2016-01-25T18:43:54Z

Merging now. Thanks @wietsefranssen.

wietsefranssen added 4 commits October 13, 2015 13:17

Added: lat, lon and time output variable (still hardcoded!!!)

dcf67c6

Added lat, lon and time variables in NetCDF output

8397f93

Merge branch 'develop' of https://github.com/UW-Hydro/VIC into featur…

5dabee0

…e/outfile_time * 'develop' of https://github.com/UW-Hydro/VIC: fix rounding / precision bug at midnight with subdaily timestep add python 3.5 to travis print traceback when log_err is called

Adapted 1D and 2D lat/lon variables in NetCDF output. Added time vari…

e5a345c

…able in NetCDF output

jhamman reviewed Nov 10, 2015
View reviewed changes

uncrustified with version 0.61. Also changed some allocation statemen…

67d46c4

…ts (cast syntax).

wietsefranssen added 3 commits November 17, 2015 14:14

derived calendars from 'global_param' to output NetCDF.

d3071a3

derived time units from 'global_param' to output NetCDF. fixed time init by adding 'initialize_time()' to 'vic_init.c'

merged with latest upstream develop branch

2c9e4e9

fixed little mistake in some if statements

4e4cd6a

jhamman reviewed Nov 17, 2015
View reviewed changes

Added DOMAIN_LON_VAR and DOMAIN_LAT_VAR option in global config file

67fdb74

Also some minor adjustments

jhamman changed the title ~~Feature/outfile time~~ Outfile Dimensions Nov 19, 2015

jhamman reviewed Nov 19, 2015
View reviewed changes

wietsefranssen added 4 commits December 9, 2015 00:10

Merge branch 'develop' of https://github.com/UW-Hydro/VIC into featur…

4b0149d

…e/outfile_time

Merged and removed some conficts

77fb575

Removed from global parameter file. is now retreived from the domain …

5a2d56b

…file

uncrustified

8a7248f

jhamman reviewed Dec 24, 2015
View reviewed changes

jhamman added feature image driver labels Dec 24, 2015

jhamman added this to the 5.0 milestone Dec 24, 2015

jhamman assigned wietsefranssen Dec 24, 2015

wietsefranssen added 2 commits January 8, 2016 16:29

Add suggestion of Joe to make the dimension names conditional for the…

60c5e28

… number or coordinate dims. Also a minor bug fix in vic_write.c

fixed indentation

d0304fb

Merge branch 'develop' of https://github.com/UW-Hydro/VIC into featur…

57f8a60

…e/outfile_time

wietsefranssen mentioned this pull request Jan 25, 2016

Add a time variable in NetCDF output file #297

Closed

jhamman pushed a commit that referenced this pull request Jan 25, 2016

Merge pull request #303 from wietsefranssen/feature/outfile_time

60293f1

Outfile Dimensions

jhamman merged commit 60293f1 into UW-Hydro:develop Jan 25, 2016

jhamman mentioned this pull request Jan 25, 2016

Add all grid cells to global domain structure #313

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Outfile Dimensions #303

Outfile Dimensions #303

wietsefranssen commented Oct 19, 2015

jhamman Nov 10, 2015

wietsefranssen Nov 12, 2015

jhamman Nov 19, 2015

jhamman commented Nov 10, 2015

jhamman Nov 10, 2015

wietsefranssen Nov 12, 2015

jhamman Nov 12, 2015

wietsefranssen Nov 12, 2015

bartnijssen commented Nov 14, 2015

jhamman commented Nov 14, 2015

bartnijssen commented Nov 14, 2015

jhamman Nov 17, 2015

jhamman Nov 17, 2015

wietsefranssen Nov 19, 2015

jhamman Nov 19, 2015

jhamman commented Nov 19, 2015

wietsefranssen commented Dec 17, 2015

jhamman Dec 24, 2015

jhamman commented Dec 24, 2015

wietsefranssen commented Jan 8, 2016

jhamman commented Jan 23, 2016

jhamman commented Jan 25, 2016

jhamman commented Jan 25, 2016

Outfile Dimensions #303

Outfile Dimensions #303

Conversation

wietsefranssen commented Oct 19, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jhamman commented Nov 10, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bartnijssen commented Nov 14, 2015

jhamman commented Nov 14, 2015

bartnijssen commented Nov 14, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jhamman commented Nov 19, 2015

wietsefranssen commented Dec 17, 2015

Choose a reason for hiding this comment

jhamman commented Dec 24, 2015

wietsefranssen commented Jan 8, 2016

jhamman commented Jan 23, 2016

jhamman commented Jan 25, 2016

jhamman commented Jan 25, 2016