new section 4 template for further statistics on probabilities based on focal statistics #226

sebvi · 2023-10-17T10:50:47Z

Initial request

This proposal extends the existing template 4.112 to encode probabilities of anomalies and other parameters which depend on a reference period to describe a spatiotemporal processing based on focal (moving window) statistics as used in in the templates proposed in issue #.

A branch with an implementation of this template is accessible under:
https://github.com/ecmwf/eccodes/tree/feature/ECC-1705-probabilitiesWithFocalStats

Amendment details

ADD New Code Table 4.101 – Type of relationship to reference dataset

Code	Description
3	Climatology
22	Anomaly of probabilities (see note)

Note: Code number 22 is intended to be used with templates 4.112 and 4.123. Instead of having the probability of the exceedance of the anomaly (code number 0) of the limits specified in the message, it specifies probabilities which were calculated for the forecast and for the reference dataset with the given specification listed in the meta-data. The anomaly is calculated between the two derived probabilities.

Add Template 4.123: Probability forecasts from large ensembles with spatiotemporal processing based on focal (moving window) statistics in relation to a reference period at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval

Octet	Number of Octets	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Generating Process (see code table 4.3)
13	1	Background Process
14	1	Generating Process Identifier
15-16	2	Hours After Data Cut-off
17	1	Minutes After Data Cut-off
18	1	Indicator of Unit of Time Range (see code table 4.4)
19-22	4	Forecast Time
23	1	Type of First Fixed Surface (see code table 4.5)
24	1	Scale Factor of First Fixed Surface
25-28	4	Scaled Value of First Fixed Surface
29	1	Type of Second Fixed Surface (see code table 4.5)
30	1	Scale Factor of Second Fixed Surface
31-34	4	Scaled Value of Second Fixed Surface
35-36	2	Year of End of Overall Time Interval
37	1	Month of End of Overall Time Interval
38	1	Day of End of Overall Time Interval
39	1	Hour of End of Overall Time Interval
40	1	Minute of End of Overall Time Interval
41	1	Second of End of Overall Time Interval
42	1	Number of Time Range - NT
43-46	4	Number of Missing in Statistical Process
		Repeats NT times for nt=1:NT
47+(nt-1)*12	1	Type of Statistical Processing
48+(nt-1)*12	1	Type of Time Increment
49+(nt-1)*12	1	Indicator of Unit for Time Range
50+(nt-1)12 - 53+(nt-1)12	4	Length of Time Range
54+(nt-1)*12	1	Indicator of Unit for Time Increment (see code table 4.4)
55+(nt-1)12-58+(nt-1)12	4	Time Increment
		End of repeated NT sequence
59+(nt-1)*12	1	Type of ensemble forecast (see Code table 4.6)
60+(nt-1)12-63+(nt-1)12	4	Number of forecasts in ensemble
64 + (NT-1)*12	1	Forecast probability number
65 + (NT-1)*12	1	Total Number of Forecast Probabilities
66 + (NT-1)*12	1	Probability type (see code table 4.9.table)
67 + (NT-1)*12	1	Scale factor of lower limit
68 + (NT-1)12 - 71 + (NT-1)12	4	Scaled value of lower limit
72 + (NT-1)*12	1	Scale factor of upper limit
73 + (NT-1)12 - 76 + (NT-1)12	4	Scaled value of upper limit
77 + (NT-1)*12	1	Type of Reference Dataset (see code table 4.100)
78 + (NT-1)*12	1	Type of Relation to Reference Dataset (see code table 4.101)
79 + (NT-1)*12	1	Number of Additional Parameter for Reference Period – NA (see note 1)
		The next two entries are repeated NA times na=1:NA – can be 0 times if no parameters are necessary
80 + (NT-1)12 + (na-1)5	(1)	Scale Factor of Additional Parameter for Reference Period
81 + (NT-1)12-84 + (NT-1)12 + (na-1)*5	(4)	Scaled Value of Additional Parameter for Reference Period
		end of repeated NA sequence
80 + (NT-1)12 + NA5-81 + (NT-1)12 + NA5	2	Year of Start of Reference Period
82 + (NT-1)12 + NA5	1	Month of Start of Reference Period
83 + (NT-1)12 + NA5	1	Day of Start of Reference Period
84 + (NT-1)12 + NA5	1	Hour of Start of Reference Period
85 + (NT-1)12 + NA5	1	Minute of Start of Reference Period
86 + (NT-1)12 + NA5	1	Second of Start of Reference Period
87 + (NT-1)12 + NA5-90 + (NT-1)12 + NA5	4	Sample Size of Reference Period
91 + (NT-1)12 + NA5	1	Number of Reference Period Time Range – NR (see note 2)
		The next three entries are repeated NR times
92 + (NT-1)12 + NA5 + (nr-1)*6	1	Type of Statistical Processing of Reference Dataset (see code table 4.102)
93 + (NT-1)12 + NA5 + (nr-1)*6	1	Reference Period Indicator of Unit for Time Increment (see code table 4.4)
94 + (NT-1)12 + NA5 + (nr-1)6-97 + (NT-1)12 + NA5 + (nr-1)6	4	Length of Reference Period
		end of repeated NR sequence
98 + (NT-1)12 + NA5 + (nr-1)*6	1	Spatial Vicinity Type (see code table 4.103)
99 + (NT-1)12 + NA5 + (nr-1)*6	1	Number of Spatial Vicinity Values – NSV
		The next entry repeats nsv=1:NSV times
(100 +(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6–(103+(nsv-1)4) + (NT-1)12+ NA5 + (nr-1)6	4	Spatial Vicinity Value
104+(nsv-1)4 + (NT-1)12 + NA5 + (nr-1)6	1	Spatial Vicinity Processing (see code table 4.104)
(105+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6-(106+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6	2	Spatial Vicinity Processing Argument 1 (e.g., if previous is quantile)
(107+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6-(108+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6	2	Spatial Vicinity Processing Argument 2 (e.g., if previous is quantile)
109+(nsv-1)4 + (NT-1)12 + NA5 + (nr-1)6	1	Spatial Vicinity Missing Data (see code table 4.105)
(110+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6	1	Temporal Vicinity Processing (see code table 4.104)
(111+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6	1	Temporal Vicinity Unit (see code table 4.4)
(112+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6-(115+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6	4	Temporal Vicinity Towards Past
(116+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6-(119+(nsv-1)4) + (NT-1)12 + NA5 + (nr-1)6	4	Temporal Vicinity Towards Future

Comments

No response

Requestor(s)

Robert Osinski (ECMWF)
Sebastien Villaume (ECMWF)

Stakeholder(s)

ECMWF

Publication(s)

Example: Manual on Codes (WMO-No. 306), Volume I.2, GRIB template and code tables in section 4

Expected impact of change

None

Collaborators

No response

References

No response

Validation

No response

amilan17 · 2023-10-17T13:07:22Z

https://github.com/wmo-im/CCT/wiki/Teleconference.17.18.October.2023 notes:
Sebastien introduced the proposal; team has no questions yet

amilan17 · 2023-11-21T14:03:16Z

https://github.com/wmo-im/CCT/wiki/Teleconference.21.22.November.2023 notes:
this is an addition to 225 for more advanced statistics on climatology; These should be the same as the climatology templates from a couple fast-tracks past; @sebvi will update branch and provide samples

amilan17 · 2023-11-29T13:53:42Z

https://github.com/wmo-im/CCT/wiki/Teleconference.21.29.November.2023 notes:

extension of #225, so appendix can help with this too

sebvi · 2023-12-14T11:40:56Z

edited the proposal to add an escape character preventing the * operator to be interpreted as formatting character.

sebvi · 2023-12-14T14:41:28Z

branch updated

amilan17 · 2024-01-10T13:22:46Z

https://github.com/wmo-im/CCT/wiki/Teleconference.10.January.2024 notes:

Ready; @amilan17 validate branch

…sed on focal statistics (#235) * adding new templates, code tables entries and notes for issue #226 * finishing template 4.123 for issue #225 * Update GRIB2_Template_4_123_ProductDefinitionTemplate_en.csv fix CSV * editorial --------- Co-authored-by: Sebastien Villaume <sebastien.villaume@ecmwf.int>

* Update GRIB2_CodeFlag_4_2_CodeTable_en.csv updated per #229 * adding entries in code table 4.2 for issue #227 (#232) * adding entries in code table 4.2 for issue #227 * Update GRIB2_CodeFlag_4_2_CodeTable_en.csv fix CSV --------- Co-authored-by: Sebastien Villaume <sebastien.villaume@ecmwf.int> * Update GRIB2_CodeFlag_4_2_CodeTable_en.csv (#228) updating per #218 * 222 new ocean and ice parameters in code table 42 discipline 10 1 (#233) * Update CodeFlag_notes.csv updating per #222 * Update GRIB2_CodeFlag_4_2_CodeTable_en.csv updating per #222 * Update GRIB2_CodeFlag_4_5_CodeTable_en.csv updating per #222 --------- Co-authored-by: Jeff Ator <62722702+jbathegit@users.noreply.github.com> * 226 new section 4 template for further statistics on probabilities based on focal statistics (#235) * adding new templates, code tables entries and notes for issue #226 * finishing template 4.123 for issue #225 * Update GRIB2_Template_4_123_ProductDefinitionTemplate_en.csv fix CSV * editorial --------- Co-authored-by: Sebastien Villaume <sebastien.villaume@ecmwf.int> * 224 new probability templates for large ensemble (#234) * adding templates 4.119 and 4.120 and corresponding entries in code table 4.0 for issue #224 * adding missing entry in Code table 4.7 for issue #224 * editorial --------- Co-authored-by: Sebastien Villaume <sebastien.villaume@ecmwf.int> * 220 code table 42 adding new parameters (#239) * addingnew entries and note for issue #220 * removing the word -depth- from the new entry 20 and the existing entry 13 in code table 4.2 discipline 1 category 0 * editorial --------- Co-authored-by: Sebastien Villaume <sebastien.villaume@ecmwf.int> * editorial changes based on feedback from LSU (#230) #230 * editorial changes based on feedback from lsu * change ' - ' to ' to ' in octets where apppropriate * change ' - ' to ' to ' in octets where apppropriate * change ' - ' to ' to ' in octets where apppropriate * octet formating * octet formating * octet formating * octet formating * editorial * 221 encoding of drought indexes as defined by wmo (#237) * update for proposal #221 * fix typos for #221 * fix typos for #221 * removed word drought from entry in code table 4.10 * editorial * editorial for note 221b * Update notes/CodeFlag_notes.csv * editorial --------- Co-authored-by: Sebastien Villaume <sebastien.villaume@ecmwf.int> * 225 new templates for probability forecasts based on focal statistics (#236) * adding/modifying code tables for issue #225 * adding template 4.121 and 4.122 for issue #225 * Update GRIB2_Template_4_121_ProductDefinitionTemplate_en.csv * Update GRIB2_Template_4_122_ProductDefinitionTemplate_en.csv * Update GRIB2_CodeFlag_4_103_CodeTable_en.csv * Update GRIB2_CodeFlag_4_103_CodeTable_en.csv * Update GRIB2_CodeFlag_4_2_CodeTable_en.csv * adding notes for issue #225 * editorial * Update notes/CodeFlag_notes.csv * editorial --------- Co-authored-by: Sebastien Villaume <sebastien.villaume@ecmwf.int> * 173 updates to facilitate needs in modelling for nuclear protection (#241) * adding/modifying code tables and templates for issue #173 * adding tables for issue #173 * editorial changes * fix typo * Update GRIB2_CodeFlag_4_2_CodeTable_en.csv * Update GRIB2_CodeFlag_4_2_CodeTable_en.csv * fix CSV * editorial * Update GRIB2_Template_4_124_ProductDefinitionTemplate_en.csv * add missing content * remove irrelevant notes --------- Co-authored-by: Sebastien Villaume <sebastien.villaume@ecmwf.int> * Update GRIB2_CodeFlag_4_9_CodeTable_en.csv --------- Co-authored-by: Jeff Ator <62722702+jbathegit@users.noreply.github.com> Co-authored-by: Sebastien Villaume <sebastien.villaume@ecmwf.int>

amilan17 added this to Submitted in GRIB2 Amendments via automation Oct 17, 2023

amilan17 added this to the FT2024-1 milestone Oct 17, 2023

amilan17 moved this from Submitted to In discussion in GRIB2 Amendments Nov 21, 2023

amilan17 moved this from In discussion to In Validation in GRIB2 Amendments Nov 22, 2023

sebvi added a commit that referenced this issue Dec 14, 2023

adding new templates, code tables entries and notes for issue #226

361d7f5

amilan17 mentioned this issue Jan 4, 2024

226 new section 4 template for further statistics on probabilities based on focal statistics #235

Merged

amilan17 moved this from In Validation to Validated in GRIB2 Amendments Feb 23, 2024

amilan17 moved this from Validated to Ready for FT Approval Procedure in GRIB2 Amendments Feb 23, 2024

amilan17 closed this as completed Mar 5, 2024

sebvi mentioned this issue May 1, 2024

Additional Anomalies Templates #246

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new section 4 template for further statistics on probabilities based on focal statistics #226

new section 4 template for further statistics on probabilities based on focal statistics #226

sebvi commented Oct 17, 2023 •

edited

Loading

amilan17 commented Oct 17, 2023

amilan17 commented Nov 21, 2023

amilan17 commented Nov 29, 2023

sebvi commented Dec 14, 2023

sebvi commented Dec 14, 2023

amilan17 commented Jan 10, 2024

new section 4 template for further statistics on probabilities based on focal statistics #226

new section 4 template for further statistics on probabilities based on focal statistics #226

Comments

sebvi commented Oct 17, 2023 • edited Loading

Initial request

Amendment details

Comments

Requestor(s)

Stakeholder(s)

Publication(s)

Expected impact of change

Collaborators

References

Validation

amilan17 commented Oct 17, 2023

amilan17 commented Nov 21, 2023

amilan17 commented Nov 29, 2023

sebvi commented Dec 14, 2023

sebvi commented Dec 14, 2023

amilan17 commented Jan 10, 2024

sebvi commented Oct 17, 2023 •

edited

Loading