New Table D sequences for reporting biogeochemical measurements from Argo floats #155
Comments
|
25-April-meeting: |
|
As Jeff noted on the call, after checking I don't think we can use 2-07-YYY because it only enables you to multiply the existing (default) reference value by 10^YYY and we are defining/using descriptors with an existing reference value of zero (the physical minimum).
From: xchen ***@***.***>
Sent: 25 April 2023 14:24
To: wmo-im/BUFR4 ***@***.***>
Cc: Jon Turton ***@***.***>; Author ***@***.***>
Subject: Re: [wmo-im/BUFR4] New Table D sequences for reporting biogeochemical measurements from Argo floats (Issue #155)
You don't often get email from ***@***.******@***.***>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
This email was received from an external source. Always check sender details, links & attachments.
25-April-meeting:
(1) Jon will investigate on 2 07 to replace 2 03 014
(2) Jon is to provide the bufr sample data for validation before the beginning of June
(3) Marijana (ECMWF), Sibylle and Lei (CMA) will validate
(4) Anna to create a branch later
-
Reply to this email directly, view it on GitHub<#155 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A4X2BOHXZH542GNNVF5ZGQDXC7F77ANCNFSM6AAAAAAW33K5AM>.
You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>
|
Thanks, Jon. I was disconnected from the call for a while. I will check the recording and update the comment. Thanks for pointing it out. @JonTurton |
|
Correction - In the oxygen sequence 3-06-044 note the delayed replication of 8 descriptors should be given by 1 08 00 (not 1 09 00) |
|
Initial BUFR files for validation. For oxygen, nitrate, chlorophyll and seawater pH (BBP700 still to come). Each BGC sequence follows the core Argo sequence (3-15-003). BUFR files are .dat, my decodes are .txt. |
|
Dear all, I decoded the samples with INPE-mbufrtools software. The values are the same as the text provided. Follows decoded files. |
|
https://github.com/wmo-im/CCT/wiki/Teleconference.6.7.June.2023 notes: |
|
ph ( 0-41-006 discriptor) is a dimensionless variable. In the case of table B this is should be included as numeric |
|
I now attach a BUFR file (.dat) and my decode (.txt) for the profile of BBP700 (proposed sequence 3-06-048). If necessary I can generate the various BGC sequences for another float (but not until next week). Generated and decoded with my own code. |
|
Dear @amilan17 , FYI : @sergioh-pessoal and @JonTurton I just pushed the changes in the dedicated branch and modify 3 files, you can find the commit here : Bellow the diff with the master : Best to all, |
|
Dear @antionio Thank you for create the branch for me. |
|
Dear @JonTurton . I decoded the ArgoBBP700, with INPE decoder. I got the same result of the sample after fixed small error in the sequence provided. In the sequence 3-06-048 (BBP700 profile data), where is write 0 41 003 the correct descriptor is 0 41 007. I'll fix this in the branch |
|
Dear all, best regards |
no problem @sergioh-pessoal , only my tag is @antoinemerle :) cheers. |
|
https://github.com/wmo-im/CCT/wiki/Teleconference.13.July.2023 notes: Sergio validated the branch; @amilan17 will create PR; |
|
I think there's a typo in 3 06 044: "1 09 000 Delayed replication of 8 descriptors" Change to "108000" or change description to "Delayed replication of 9 descriptors"? According to BCO-DMO (https://www.bco-dmo.org/parameter/1776) the official "name" is bbp700 (lower case), but NERC (http://vocab.nerc.ac.uk/collection/R03/current/BBP700/) references it as BBP700. Do you have a preference? |
Updated branch to reflect this feedback and made all BBP references upper case. |
|
@JonTurton @marijanacrepulja @sergioh-pessoal I think there's an extra change reference value at the ends of 30644, 30645, 30646, and 306458 that is not needed. @david-i-berry agrees, but I wanted to get extra confirmation from the team or the proposer. Thanks.
|
|
I also agree and I think @amilan17 is right, 203000 at the end of the proposed sequences is not needed. |
|
Hi Sergio,
Thanks for spotting and correcting.
Jon
From: Sergkl - Sergio Ferreira ***@***.***>
Sent: Monday, July 10, 2023 5:59 PM
To: wmo-im/BUFR4 ***@***.***>
Cc: Jon Turton ***@***.***>; Mention ***@***.***>
Subject: Re: [wmo-im/BUFR4] New Table D sequences for reporting biogeochemical measurements from Argo floats (Issue #155)
This email was received from an external source. Always check sender details, links & attachments.
Dear @JonTurton<https://github.com/JonTurton> . I decoded the ArgoBBP700, with INPE decoder. I got the same result of the sample after fixed small error in the sequence provided. In the BBP700, where is write 0 41 003 the correct descriptor is 0 41 007.
The sample is valid with this small correction.
I'll fix this in the branch
Attached the decoded file
BUFR_4903456_025bbp700_decoded.txt.gz<https://github.com/wmo-im/BUFR4/files/12004529/BUFR_4903456_025bbp700_decoded.txt.gz>
-
Reply to this email directly, view it on GitHub<#155 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A4X2BOCCQB6F3XSD7RT5JPDXPQYF3ANCNFSM6AAAAAAW33K5AM>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
|
Thanks - I had noted that in the comments (May 10) as I could not work out how to edit the main text containing the sequences. I'm not very familiar with using GitHub.
Jon
From: Anna Milan ***@***.***>
Sent: Friday, July 14, 2023 4:02 PM
To: wmo-im/BUFR4 ***@***.***>
Cc: Jon Turton ***@***.***>; Mention ***@***.***>
Subject: Re: [wmo-im/BUFR4] New Table D sequences for reporting biogeochemical measurements from Argo floats (Issue #155)
This email was received from an external source. Always check sender details, links & attachments.
@JonTurton<https://github.com/JonTurton> I think there's a typo in 3 06 044.
"1 09 000 Delayed replication of 8 descriptors"
Change to "108000" or change description to "Delayed replication of 9 descriptors"?
-
Reply to this email directly, view it on GitHub<#155 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A4X2BOGXJOI7MIJDIVKGTV3XQFNM7ANCNFSM6AAAAAAW33K5AM>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
|
The reason I included the change reference value at the end of the sequence(s) to revert back to the default values was just in case a sequence (although it would not be for Argo BGC data) follows that might hold the same BGC variables but assumes the default reference values. It's possible although unlikely.
Jon
From: Sibylle Krebber ***@***.***>
Sent: Tuesday, July 18, 2023 10:25 AM
To: wmo-im/BUFR4 ***@***.***>
Cc: Jon Turton ***@***.***>; Mention ***@***.***>
Subject: Re: [wmo-im/BUFR4] New Table D sequences for reporting biogeochemical measurements from Argo floats (Issue #155)
This email was received from an external source. Always check sender details, links & attachments.
I also agree and I think @amilan17<https://github.com/amilan17> is right, 203000 at the end of the proposed sequences is not needed.
-
Reply to this email directly, view it on GitHub<#155 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A4X2BOHBWAZE6CUBHNVMJYLXQZJAPANCNFSM6AAAAAAW33K5AM>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
@marijanacrepulja can you please review this? |
|
@amilan17 I agree that we dont need 203 000 at the end. I believe the users should take care when encoding observations. |
|
Going back to my earlier comment, that would mean if for example you followed data for an Argo float with oxygen that has a changed (non-zero) reference value with ship CTD data (TM 3-15-007) that includes an oxygen profile, the changed reference would still apply to the CTD oxygen data which assumes the default zero reference value hence incorrect values would be decoded (even though the decoding itself would be correct). The manual states 'The operations specified by operator descriptors 2 01, 2 02, 2 03, 2 04, 2 07 and 2 08 remain defined until cancelled or until the end of the data subset' so I interpret that as those operations remain in force until the end of the Section 3 data. Keeping the 2-03-000 in place will ensure that anyone decoding the data can not use the wrong reference value for any subsequent data. The encoded BUFR file is unchanged, it just instructs the person decoding the data that the reference value defaults to the standard value. Hence I think it would be best preactice to leave the 2-03-000 in place at the end of the sequence(s). |
|
I think it's important to include the 2-03-000 at the end of the sequence. Otherwise, as Jon noted, and as Marijana noted earlier in the thread, the new reference value would still be in effect if a user later decided to add something else to Section 3 following the sequence in question. And while it's true that users should always take care when encoding sequences, the reality is that not all of them do and we can't always rely on them doing so, especially when we're talking about the 2-03 operator which can be confusing even to BUFR experts such as ourselves. So I would strongly urge to keep 2-03-000 at the end of the sequence. To me, this is no different than how we currently have 2-01-000, 2-02-000, 2-07-000, etc. at the end of many other existing sequences, to ensure that any such operators inside of those sequences are also properly closed and that no lingering changes are still in effect that could trip up someone who adds something to Section 3 after those sequences. |
|
@amilan17 Could you please revert changes and keep 203 000 at the end of BUFR templates as it was proposed. Many thanks. |
|
Please let me know if anyone wants any more BUFR files to decode for the validation procedure. I can probably find another BGC float and generate some BUFR files from the netCDF on the GDAC. Thanks. |
#155 new table d sequences for reporting biogeochemical measurements from argo floats
amilan17
moved this from In Validation
to Ready for FT Approval Procedure
in BUFR4 Amendments
Initial request
To review and approve five new Table D sequences for reporting biogeochemical data from Argo floats. For (i) dissolved oxygen, (ii) chlorophyll-A, (iii) nitrate, (iv) sea water pH and (v) BBP700 (particle backscattering at 700 nm).
The sequences would be used, where needed following template 3-15-003 (for temperature and salinity profile observed by sub-surface profiling floats) when additional biogeochemical measurements are made.
Amendment details
New Table D entries
(i) Dissolved oxygen profile from Argo floats
Change both number of bits and the reference value for 0-22-188
Table Reference Element Name
F X Y
3 06 044 Dissolved oxygen profile from floats
2 03 014 Change reference value (New reference value in 14 bits)
0 22 188 Change oxygen reference value to -5000 (given by 11001110001000)
2 03 255 Change reference value (Terminate reference value change)
1 09 000 Delayed replication of 8 descriptors
0 31 002 Extended delayed descriptor replication factor (Gives number of depths)
0 07 065 Water pressure
0 08 080 Qualifier for quality flag (= 10, indicates pressure at a level)
0 33 050 GTSPP quality flag
2 01 129 Change data width (Add 1 bit to oxygen data width)
0 22 188 Dissolved oxygen (μmol kg-1)
2 01 000 Change data width (Cancel change in oxygen data width)
0 08 080 Qualifier for quality flag (= 16, indicates dissolved oxygen at a level)
0 33 050 GTSPP quality flag
2 03 000 Change reference value (Reset oxygen reference value to Table B value)
(ii) Chlorophyll-A profile from Argo floats
New descriptor for chlorophyll-A 0 41 004, if 19 bits only need to change the reference value
Table Reference Element Name
F X Y
3 06 045 Chlorophyll-A profile from floats
2 03 011 Change reference value (New reference value in 11 bits)
0 41 004 Chlorophyll-A reference value of -1000 (given by 11111101000)
2 03 255 Change reference value (Terminate reference value change)
1 06 000 Delayed replication of 6 descriptors
0 31 002 Extended delayed descriptor replication factor (Gives number of depths)
0 07 065 Water pressure
0 08 080 Qualifier for quality flag (= 10, indicates pressure at a level)
0 33 050 GTSPP quality flag
0 41 004 Chlorophyll-A (mg m-3)
0 08 080 Qualifier for quality flag (= 21, indicates chlorophyll-A at a level)
0 33 050 GTSPP quality flag
2 03 000 Change reference value (Reset chlorophyll-A reference value to Table B default)
(iii) Nitrate profile from Argo floats
Only need to change the reference value
Table Reference Table References Element Name
F X Y
3 06 046 Dissolved nitrate profile data from floats
2 03 015 Change reference value (New reference value in 15 bits)
0 41 003 Nitrate reference value of -15000 (given by 111101010011000)
2 03 255 Change reference value (Terminate reference value change)
1 06 000 Delayed replication of 6 descriptors
0 31 002 Extended delayed descriptor replication factor (Gives number of depths)
0 07 065 Water pressure
0 08 080 Qualifier for quality flag (= 10, indicates pressure at a level)
0 33 050 GTSPP quality flag
0 41 003 Dissolved nitrate (in μmol kg-1)
0 08 080 Qualifier for quality flag (= 22, indicates nitrate at a level)
0 33 050 GTSPP quality flag
2 03 000 Change reference value (Reset nitrate reference value to Table B default)
(iv) Sea water pH profile from Argo floats
A new descriptor recommended for seawater pH to cover possible range from 7 to 8.8 to O(0.0001) 0-41-006.
Table Reference Element Name
F X Y
3 06 047 Sea water pH profile data from floats
1 06 000 Delayed replication of 6 descriptors
0 31 002 Extended delayed descriptor replication factor (Gives number of depths)
0 07 065 Water pressure
0 08 080 Qualifier for quality flag (= 10, indicates pressure at a level)
0 33 050 GTSPP quality flag
0 41 006 Sea water pH (dimensionless)
0 08 080 Qualifier for quality flag (= 23, indicates pH at a level)
0 33 050 GTSPP quality flag
(v) BBP700 (Particle backscattering at 700 nm) profile from Argo floats
New descriptor for BBP 0-41-007
Table Reference Element Name
F X Y
3 06 048 BBP700 profile data
2 03 009 Change reference value (New reference value in 9 bits)
0 41 003 BBP reference value of -250 (given by 111111010)
2 03 255 Change reference value (Terminate reference value change)
1 06 000 Delayed replication of 6 descriptors
0 31 002 Extended delayed descriptor replication factor (Gives number of depths)
0 07 065 Water pressure
0 08 080 Qualifier for quality flag (= 10, indicates pressure at a level)
0 33 050 GTSPP quality flag
0 41 007 BBP In parts per billion
0 08 080 Qualifier for quality flag (= 24, indicates BBP at a level)
0 33 050 GTSPP quality flag
2 03 000 Change reference value (Reset BBP reference value to Table B default)
New Table B entries
Descriptor Name Units Scale Reference Width
0-41-004 Chlorophyll-A mg m-3 4 0 19
0-41-006 Sea water pH 4 70000 15
0-41-007 BBP700 m-1 7 0 20
Additions to Code Tables
0-08-080 Qualifier for GTSPP quality flag
21 Chlorophyll-A at a level
22 Nitrate at a level
23 pH at a level
24 BBP700 at a level
Comments
The sequences here have been developed in discussion with the Argo Data Management Team experts on BGC Argo, who have advised on the range and precision required for the various BGC variables. The Argo BGC BUFR formatted data would in virtually all cases be generated from Argo netCDF bio-files. A key point to note is that negative values for some BGC variable concentrations are allowed in the netCDF files (from which the BUFR would be generated). Although negative concentrations are physically impossible, the reason for allowing them in the Argo netCDF, where they may be flagged as ‘good’ data, is because the values reported are not reality but estimates of reality. These estimates come with an uncertainty, so any "true" zero value will have an envelope of uncertainty that spans negative values (even after an adjustment process is performed). In BGC-Argo, negative values are allowed with the understanding that changing all negative values to zero would result in left-censoring of the data and skewing any global statistics. (Ken Johnson touches on this in section 4.2 of his publication on SOCCOM sensor performance, https://doi.org/10.1002/2017JC012838.)
As such the following sequences have been designed to allow for such negative values so the data are the same as in the netCDF and has not been debased, where the decision on how to handle any negative values will be down to the data users. Where existing descriptors are used the 2-XX-YYY data description operators are used to change the reference value, scaling and number of bits where required. Where new descriptors are suggested the reference values are set to zero for the minimum physical value and a 2-03-YY data description operator used to allow for negative values to be consistent with the Argo netCDF. A simplification is that only pressure is needed for the vertical coordinate as depth is not reported from floats.
Requestor(s)
Jon Turton on behalf of the Argo Data Management Team
Stakeholder(s)
Argo Data Management Team
Operational ocean forecasting centres
Publication(s)
Example: Manual on Codes (WMO-No. 306), Volume I.2, BUFR code table 0 20 009
Expected impact of change
None
Collaborators
Argo Data Management Team experts on BGC Argo data
References
No response
Validation
No response
The text was updated successfully, but these errors were encountered: