[BUG/ISSUE] Wrong timestamps in OFFLINE_BIOVOC/v2019-10 for Dec 29 2015 to Jan 1 2016

[LiamBindle]

Describe the bug

It looks like the timestamps in OFFLINE_BIOVOC/v2019-10 for the following dates are incorrect

• 2015-12-29
• 2015-12-30
• 2015-12-31
• 2016-01-01

For example

$ ncdump -h biovoc_025.20151229.nc
netcdf biovoc_025.20151229 {
dimensions:
	time = UNLIMITED ; // (24 currently)
	lon = 1152 ;
	lat = 721 ;
variables:
	double time(time) ;
		time:standard_name = "time" ;
		time:long_name = "Time" ;
		time:units = "hours since 2015-12-28 00:00:00 GMT" ;
		time:calendar = "gregorian" ;
		time:axis = "T" ;
.
.
.

This causes GCHP runs to crash if they run through this period.

It appears Dec 29-31 are copies of Dec 28, and Jan1 is a copy of Jan 2. See below

$ md5sum 2015/12/biovoc_025.20151228.nc 2015/12/biovoc_025.20151229.nc 2015/12/biovoc_025.20151230.nc 2015/12/biovoc_025.20151231.nc 2016/01/biovoc_025.20160101.nc 2016/01/biovoc_025.20160102.nc
f6a89dac5d28dc7c0d162028bd50a1bb  2015/12/biovoc_025.20151228.nc
f6a89dac5d28dc7c0d162028bd50a1bb  2015/12/biovoc_025.20151229.nc
f6a89dac5d28dc7c0d162028bd50a1bb  2015/12/biovoc_025.20151230.nc
f6a89dac5d28dc7c0d162028bd50a1bb  2015/12/biovoc_025.20151231.nc
074cbfc0a40d944d9878d06005ecc2e5  2016/01/biovoc_025.20160101.nc
074cbfc0a40d944d9878d06005ecc2e5  2016/01/biovoc_025.20160102.nc

To Reproduce

This can be reproduced by running a 1-day GCHP simulation starting at 2015-12-28 12:00.

Additional context

Thanks @lizziel for the help, and for tracking the root of the problem!

[msulprizio]

This is a known issue, but I don't believe it was recorded here on Github. @hongjianweng wrote:

There is a issue at the beginning and end of the dataset coverage period, which Melissa mentioned that both fixes would go into 12.7.0.

The data of 1/1, 12/28, 12/29, 12/30, and 12/31 in every year are wrong, and I temporarily replaced these error date with approaching day.

The fixes referred to above are the HEMCO fixes that went into GEOS-Chem 12.7.0, allowing us to properly interpolate (and officially turn on) the Yuan-processed MODIS LAI.

The plan was to reprocess those dates once fixes for the Yuan-processed MODIS LAI were in and GEOS-Chem 12.7.0 was approved. @hongjianweng If you haven't already, could you reprocess those dates at your earliest convenience?

[lizziel]

Does this cause a problem in GEOS-Chem Classic or only GCHP? It causes GCHP to crash when running on those dates so is an urgent problem to fix. I can update the time units string to prevent the crash in a new data directory for 12.8.1 and use those new files only in GCHP if GEOS-Chem Classic is unaffected.

[msulprizio]

It does not cause a problem in GEOS-Chem classic. In HEMCO_Config.rc a time cycle flag of "C" is used, so HEMCO will just use the closest date available.

[lizziel]

That makes sense. GCHP ExtData uses the time units string to determine the times in file, e.g. hours since 1985-01-01 00:00:00:. So those units being Dec 28 for Dec 29th means the Dec 29th data could not be found, causing MAPL to error out.

[hongjianweng]

Hi Melissa,

Sorry for the delay. I will reprocess those dates as soon as possible. Once it's settled, I'll let you know!

[sdeastham]

Hi all - I wanted to check in on this. I just attempted a simulation with a fresh download of the Compute Canada data, and my simulation failed on 2016-12-28 because of incorrect timestamping in the 2016-12-29 file. I saw from this thread that @hongjianweng had plans to regenerate the files with the new LAI, including producing files specific to these dates. Are these files available for download? Or should we look at fixing the broken files in the current inventory? Thanks!

[hongjianweng]

Hi @sdeastham ,

Sorry for the delayed update, I will update the wrong date files at the end of this month.

Hongjian

[yantosca]

@hongjianweng, is this updated now? If so we can close out the issue.

[yantosca]

I am going to close out this issue, as there hasn't been any recent activity. I assume the updated files are created and posted.

[msulprizio]

@hongjianweng @Jun-Meng Can either of you provide an update? Have these files been reprocessed and uploaded to Compute Canada?

[LiamBindle]

They have not been updated on Compute Canada. Unfortunately, that means it will affect most GCHP users.

[hongjianweng]

@msulprizio @LiamBindle ,
The files has not been uploaded due to some problems in processing the file.
Fortunately, these problems have been solved and I'll upload in a week.
I'm so sorry for the delay and reply in time.

[LiamBindle]

Thanks, @hongjianweng!

[LiamBindle]

Hi @hongjianweng, just checking in. Do you have any updates on this?

[hongjianweng]

@LiamBindle , thanks for waiting and I have updated the wrong timestamps files for OFFLINE_BIOVOC/v2019-10 for the following dates.

[lizziel]

A quick fix for this issue went into the files on Harvard ftp for the 12.8.1 release, as listed here.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. If there are no updates within 7 days it will be closed. You can add the "never stale" tag to prevent the Stale bot from closing this issue.

[stale]

Closing due to inactivity