Skip to content

Commit

Permalink
Remove the need for appending to instantaneous collections
Browse files Browse the repository at this point in the history 
(1) inst collections with multiple hourly time points.
    1st file time of the day = 01Z,
    last file time = next day, 00Z.
    No appending needed.

   For example, a 2-day run with hourly archiving saved to
   daily files creates this output:

     GEOSChem.SpeciesConc.20160101_0010z.nc4
     GEOSChem.SpeciesConc.20160102_0000z.nc4
     GEOSChem.SpeciesConc.20160102_0010z.nc4
     GEOSChem.SpeciesConc.20160103_0000z.nc4

(2) inst collections with one time point per file.
    The file name label uses the date/time when the data is written.
    No appending needed.

    A multi-day run with hourly archiving saved to hourly files
    now creates this output:

       GEOSChem.SpeciesConc_multifile.20160101_0100z.nc4
       GEOSChem.SpeciesConc_multifile.20160101_0200z.nc4
       GEOSChem.SpeciesConc_multifile.20160101_0300z.nc4
       ...
       GEOSChem.SpeciesConc_multifile.20160101_2000z.nc4
       GEOSChem.SpeciesConc_multifile.20160101_2100z.nc4
       GEOSChem.SpeciesConc_multifile.20160101_2200z.nc4
       GEOSChem.SpeciesConc_multifile.20160101_2300z.nc4
       GEOSChem.SpeciesConc_multifile.20160102_0000z.nc4
       GEOSChem.SpeciesConc_multifile.20160102_0100z.nc4
      ... etc ...

Signed-off-by: Bob Yantosca <yantosca@seas.harvard.edu>
  • Loading branch information
@yantosca
yantosca committed Apr 21, 2020
1 parent 81c7126 commit 94d78a7
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 182 deletions.
18 changes: 0 additions & 18 deletions History/histcontainer_mod.F90
View file
Expand Up @@ -141,17 +141,9 @@ MODULE HistContainer_Mod
LOGICAL :: IsFileOpen ! Is the netCDF file
! currently open?

INTEGER :: FileLabelYmd ! YMD & HMS that are
INTEGER :: FileLabelHms ! in the file name

! Leave commented for possible future expansion (bmy, 4/20/20)
! REAL(f8) :: FileLabelJd ! Corresponding Julday
! REAL(f8) :: FileLabelJsec ! Corresponding Julsec

!----------------------------------------------------------------------
! netCDF file identifiers and attributes
!----------------------------------------------------------------------
LOGICAL :: FirstInst ! 1st inst file write?
INTEGER :: FileId ! netCDF file ID
INTEGER :: xDimId ! X (or lon ) dim ID
INTEGER :: yDimId ! Y (or lat ) dim ID
Expand Down Expand Up @@ -714,16 +706,6 @@ SUBROUTINE HistContainer_Create( Input_Opt, Container, &
Container%Z0 = UNDEFINED_INT
Container%OnLevelEdges = .FALSE.

! If the collection is instantaneous, then set a flag to denote that
! first the netCDF file reference date/time should be the start-of-the-
! simulation time. This will ensure that all timestamps and filenames
! for instantaneous collections are consistent.
IF ( Container%Operation == COPY_FROM_SOURCE ) THEN
Container%FirstInst = .TRUE.
ELSE
Container%FirstInst = .FALSE.
ENDIF

!=======================================================================
! Initialize the alarms (elapsed seconds since start of run)
!=======================================================================
Expand Down
211 changes: 47 additions & 164 deletions History/history_netcdf_mod.F90
View file
Expand Up @@ -294,42 +294,8 @@ SUBROUTINE History_Netcdf_Define( Input_Opt, Container, RC )
! %%% INSTANTANEOUS COLLECTIONS %%%
!-----------------------------------------------------------------

! Test if the frequency is the same as the duration
! in that case, the file close time is the same as the write time
offset = Container%FileCloseIvalSec - Container%FileWriteIvalSec
IF ( offset < 1e-5_f8 ) offset = 0.0_f8

! Test if the collection has multiple time points
IF ( offset > 0.0_fp ) THEN

!--------------------------------------------------------------
! (a) Instantaneous collections with multiple time points
!--------------------------------------------------------------
IF ( Container%FirstInst ) THEN

! The first time we create output for this collection,
! make sure to set both the reference timestamp
! to the starting time of this simulation.
Container%ReferenceJsec = Container%EpochJsec

ELSE

! On every subsequent file write, set the reference
! timestamp to the time of file definition.
Container%ReferenceJsec = Container%CurrentJsec

ENDIF

ELSE

!--------------------------------------------------------------
! (b) Instantaneous collections w/ a single time point:
!--------------------------------------------------------------

! Always set the reference time to the current time
Container%ReferenceJsec = Container%CurrentJsec

ENDIF
! REFERENCE TIMESTAMP: Use the date/time when the file is created.
Container%ReferenceJsec = Container%CurrentJsec

ENDIF

Expand All @@ -339,16 +305,12 @@ SUBROUTINE History_Netcdf_Define( Input_Opt, Container, RC )
yyyymmdd = Container%ReferenceYmd, &
hhmmss = Container%ReferenceHms )

! Set file label date/time to the reference date/time
Container%FileLabelYmd = Container%ReferenceYmd
Container%FileLabelHms = Container%ReferenceHms

! Replace time and date tokens in the netCDF file name
! (Save the collection's file name in a temporary variable)
FileName = TRIM( Container%FileName )
CALL Expand_Date_Time( DateStr = FileName, &
yyyymmdd = Container%FileLabelYmd, &
hhmmss = Container%FileLabelHms, &
yyyymmdd = Container%ReferenceYmd, &
hhmmss = Container%ReferenceHms, &
MAPL_Style = .TRUE. )

!------------------------------------------------------------------------------
Expand Down Expand Up @@ -388,143 +350,64 @@ SUBROUTINE History_Netcdf_Define( Input_Opt, Container, RC )
IF ( Container%Operation == COPY_FROM_SOURCE .and. &
Container%UpdateIvalSec == Container%FileCloseIvalSec .and. &
Container%FileCloseAlarm == 0.0 ) THEN

! Set the first-time flag for this container to FALSE so that future
! instantaneous files will be written with the current timestamp.
Container%FirstInst = .FALSE.

! Exit without writing a file
RETURN

ELSE

!------------------------------------------------------------------
! Special handling for instantaneous collections to avoid
! clobbering existing data at e.g. 00:00 UTC of a new day
! Create the file and add global attributes
! Remain in netCDF define mode upon exiting this routine
!
! NOTE: Container%Reference is a global attribute lists the
! GEOS-Chem web and wiki page. It has nothing to do with the
! reference data/time computed by History_Set_RefDateTime.
!------------------------------------------------------------------
IF ( Container%Operation == COPY_FROM_SOURCE .and. &
.not. isRestart .and. &
Container%FirstInst ) THEN

! Test if the collection file already exists
INQUIRE( FILE=TRIM( fileName ), EXIST=appendToFile )

! If we are appending to an existing file, then we obviously
! don't need to re-enter netCDF define mode once again.
Container%IsFileDefined = appendToFile

! Echo info about the file we are creating
IF ( Input_Opt%amIRoot ) THEN
WRITE( 6, 100 ) TRIM( Container%Name ), &
Container%ReferenceYmd, &
Container%ReferenceHms
WRITE( 6, 110 ) TRIM( FileName )
ENDIF

! Append or create?
IF ( appendToFile ) THEN

!--------------------------------------------------------------
! Append to an existing file. This is necessary e.g. to
! preserve instantaneous values at the start of a day from
! being overwritten a run that gets restarted.
!--------------------------------------------------------------

! Echo info about the file we are creating
IF ( Input_Opt%amIRoot ) THEN
WRITE( 6, 105 ) TRIM( Container%Name ), &
Container%ReferenceYmd, &
Container%ReferenceHms
WRITE( 6, 110 ) TRIM( FileName )
ENDIF

! Append to the file and find out how many
! existing time slices are already present
CALL Nc_Append( fileName = fileName, &
fId = Container%fileId, &
nTime = Container%CurrTimeSlice )

! Trap potential errors
IF ( Container%CurrTimeSlice < 1 ) THEN
ErrMsg = 'Could not get number of times in existing ' // &
'file: ' // TRIM( fileName )
CALL GC_Error( ErrMsg, RC, ThisLoc )
CALL History_Netcdf_Close( Container, RC2 )
RETURN
ENDIF

! We should only append to files that were created at the end
! of a simulation, and thus have only one timestamp. Throw
! an error if this condition occurs. Otherwise, the appended
! file could potentially have multiple time points with the
! same timestamp, which should never happen!
IF ( Container%CurrTimeSlice > 1 ) THEN
ErrMsg = 'We are trying to append to netCDF file ' // &
TRIM(fileName) // ', which has more than one ' // &
'timestamp! This should never happen! ' // &
'Exiting simulation...'
CALL GC_Error( ErrMsg, RC, ThisLoc )
CALL History_Netcdf_Close( Container, RC2 )
RETURN
ENDIF

ELSE

!--------------------------------------------------------------
! Create the file and add global attributes
! Remain in netCDF define mode upon exiting this routine
!
! NOTE: Container%Reference is a global attribute lists the
! GEOS-Chem web and wiki page. It has nothing to do with the
! reference data/time computed by History_Set_RefDateTime.
!--------------------------------------------------------------

! Echo info about the file we are creating
IF ( Input_Opt%amIRoot ) THEN
WRITE( 6, 100 ) TRIM( Container%Name ), &
Container%ReferenceYmd, &
Container%ReferenceHms
WRITE( 6, 110 ) TRIM( FileName )
ENDIF

! Create the file
CALL Nc_Create( Create_Nc4 = .TRUE., &
NcFile = fileName, &
nLon = Container%nX, &
nLat = Container%nY, &
nLev = nLev, &
nIlev = nILev, &
nTime = NF_UNLIMITED, &
NcFormat = Container%NcFormat, &
Conventions = Container%Conventions, &
History = Container%History, &
ProdDateTime = Container%ProdDateTime, &
Reference = Container%Reference, &
Title = Container%Title, &
Contact = Container%Contact, &
StartTimeStamp = Container%StartTimeStamp, &
EndTimeStamp = Container%EndTimeStamp, &
fId = Container%FileId, &
TimeId = Container%tDimId, &
LevId = Container%zDimId, &
ILevId = Container%iDimId, &
LatId = Container%yDimId, &
LonId = Container%xDimId, &
KeepDefMode = .TRUE., &
Varct = VarCt )

ENDIF

!--------------------------------------------------------------------
! Denote that the file has been created and is open
!--------------------------------------------------------------------
Container%IsFileOpen = .TRUE.

! Create the file
CALL Nc_Create( Create_Nc4 = .TRUE., &
NcFile = fileName, &
nLon = Container%nX, &
nLat = Container%nY, &
nLev = nLev, &
nIlev = nILev, &
nTime = NF_UNLIMITED, &
NcFormat = Container%NcFormat, &
Conventions = Container%Conventions, &
History = Container%History, &
ProdDateTime = Container%ProdDateTime, &
Reference = Container%Reference, &
Title = Container%Title, &
Contact = Container%Contact, &
StartTimeStamp = Container%StartTimeStamp, &
EndTimeStamp = Container%EndTimeStamp, &
fId = Container%FileId, &
TimeId = Container%tDimId, &
LevId = Container%zDimId, &
ILevId = Container%iDimId, &
LatId = Container%yDimId, &
LonId = Container%xDimId, &
KeepDefMode = .TRUE., &
Varct = VarCt )

ENDIF

!--------------------------------------------------------------------
! Denote that the file has been created and is open
!--------------------------------------------------------------------
Container%IsFileOpen = .TRUE.

ENDIF

! Set the FirstInst flag to false, which will ensure that instantaneous
! files will use the current time as the reference time going forward
Container%FirstInst = .FALSE.

! Format strings for use above
100 FORMAT( ' - Creating file for ', a, '; reference = ',i8.8,1x,i6.6 )
105 FORMAT( ' - Appending file for ', a, '; reference = ',i8.8,1x,i6.6 )
110 FORMAT( ' with filename = ', a )

!=======================================================================
Expand Down

0 comments on commit 94d78a7

Please sign in to comment.