Skip to content

Monitoring Message Dictionary

Jump to bottom Edit New page
Sebastien Denvil edited this page Sep 9, 2016 · 43 revisions

The set of supported Simulation Monitoring message types are defined below. The message codes are defined within a controlled vocabulary.

Standard Fields are included in every message below.

0000 - Simulation initialisation

0100 - Simulation ends

1000 - Computing job starts

1001 - Computing job updates

1100 - Computing job ends

1900 - Computing jobs command failure

1999 - Computing job fails - FATAL simulation has been stopped due to an error

2000 - Post-processing job starts

2100 - Post-processing job ends

2900 - Post-processing jobs command failure

2999 - Post-processing job fails - FATAL post-processing has been stopped due to an error

3000 - Post-processing from checker job starts

3100 - Post-processing from checker job ends

3900 - Post-processing from checker jobs command failure

3999 - Post-processing from checker job fails - FATAL post-processing from checker has been stopped due to an error

7000 - Environment Metrics

7010 - Project Metrics

7011 - Jobs status

7100 - Simulation Metrics

8000 - Supervision : late job detection

8001 - Supervision : late job but ok

8100 - Supervision formatter

8200 - Supervision dispatcher

8300 - Supervision verification

8400 - Supervision mail

8888 - Cleanup

Custom fields associated with specific message:

#### 0000 - Simulation initialisation (FirstInitialize)
Standard Fields
Custom Fields

  • activity - simulation activity (validated against a controlled vocabulary)

  • centre - the computing centre where simulation is being run (validated against a controlled vocabulary)

  • startDate - date upon which the simulation starts

  • endDate - date upon which the simulation ends

  • experiment - experiment being run (validated against a controlled vocabulary)

  • jobuid - the job universally unique identifier

  • login - computing centre operator login under which the simulation was run (validated against a controlled vocabulary)

  • machine - computing centre machine upon which simulation was run (validated against a controlled vocabulary)

  • model - the climate model being executed (validated against a controlled vocabulary)

  • name - the operator assigned simulation name

  • space - the simulation output space (validated against a controlled vocabulary)

  • nesting - the execution depth of the current call tree

  • simuid - the simulation universally unique identifier

  • accountingProject - project providing computing hours

  • jobWarningDelay - requested job duration in seconds

  • jobSchedulerID - ID attributed by the scheduler to this job

  • jobSubmissionPath - Submit directory and job localisation

  • archivePath - is R_OUT variable in libIGCM. Output tree located on ARCHIVE.

  • storagePath - is R_BUF variable in libIGCM.Output tree located on STORAGE (files waiting treatment, or file location when SpaceName=TEST).

  • storageSmallPath - is R_FIG variable in libIGCM : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)

Attachments
  • config card - the simulation .ini configuration file
#### 0100 - Simulation end
Standard Fields
Custom Fields
#### 1000 - Computing job starts
Standard Fields
Custom Fields

  • jobuid - the job universally unique identifier

  • simuid - the simulation universally unique identifier

  • accountingProject - project providing computing hours

  • jobWarningDelay - requested job duration in seconds

  • jobSchedulerID - ID attributed by the scheduler to this job

  • jobSubmissionPath - Submit directory and job localisation

#### 1001 - Computing job updates
Standard Fields
Custom Fields
#### 1100 - Computing job ends
Standard Fields
Custom Fields
#### 1900 - PopStack Failure - an error occurred whilst executing a command
Standard Fields
Custom Fields
#### 1999 - FATAL - simulation has been stopped due to an error
Standard Fields
Custom Fields
#### 2000 - Post-processing job starts
Standard Fields
Custom Fields

  • jobuid - the job universally unique identifier

  • simuid - the simulation universally unique identifier

  • accountingProject - project providing computing hours

  • jobWarningDelay - requested job duration in seconds

  • jobSchedulerID - ID attributed by the scheduler to this job

  • jobSubmissionPath - Submit directory and job localisation

  • postProcessingName - (like : rebuild_fromWorkdir)

  • postProcessingDate - (like : 19801231)

  • postProcessingDimn - (like : Chunck3D)

  • postProcessingComp - (like : ATM)

  • postProcessingFile - (like : Post_1M_histmth)

#### 2100 - Post-processing job ends
Standard Fields
Custom Fields
#### 2900 - Post-processing job fails
Standard Fields
Custom Fields
#### 2999 - FATAL - post-processing has been stopped due to an error
Standard Fields
Custom Fields
#### 3000 - Post-processing from checker job starts
Standard Fields
Custom Fields
#### 3100 - Post-processing from checker job ends
Standard Fields
Custom Fields
#### 3900 - Post-processing from checker job fails
Standard Fields
Custom Fields
#### 3999 - FATAL - post-processing from checker has been stopped due to an error
Standard Fields
Custom Fields
#### 7000 - Environment metrics
Standard Fields
Custom Fields
#### 7010 - Project metrics
Standard Fields
Custom Fields
Attachments
  • consumption file - from the IDRIS cpt command or the TGCC ccc_myproject command
#### 7011 - Jobs status
Standard Fields
Custom Fields

  • jobuid - the job universally unique identifier

  • simuid - the simulation universally unique identifier

  • projectName - name of the project we describe

  • projectCentre - centre on which the project is active

  • projectMachine - machine we are descriibing

  • projectNodeType - machine node type when applicable

  • totalRunning - running jobs for this project

  • totalPending - pending jobs for this project

#### 7100 - Simulation metrics
Standard Fields
Custom Fields
  • metricsGroupName : group name metrics should be pushed in
Attachments
  • json files -
#### 8000 - Supervision : late job detection (internal message)
Standard Fields
#### 8001 - Supervision : the job is late but still running at HPC
Custom Fields
Standard Fields
#### 8100 - Supervision formatter (internal message)
Standard Fields
#### 8200 - Supervision dispatcher (internal message)
Standard Fields
#### 8300 - Supervision verification ok.
Standard Fields
#### 8400 - Supervision : mail to the user if the supervision fails (internal message) ?
Standard Fields
#### 8888 - CleanUp
Standard Fields
Add a custom footer
Add a custom sidebar

Clone this wiki locally