Table D sequence for reporting trajectory profile data from marine animal tags

[david-i-berry]

Branch

https://github.com/wmo-im/BUFR4/tree/issue-17-ft

Authors

• Bill Woodward (NOAA), ATN Network coordinator
• David Berry (NOC), JCOMM TDC coordinator

Summary and purpose

The US Integrated Ocean Observing System (IOOS) Animal Telemetry Network (ATN) aims to start distributing temperature, depth and salinity measurements from marine animal tags over the GTS. A draft Table D sequence to meet the requirements to do so is presented for review and comment.

Action proposed

The team is kindly asked to review and provide comment on the contents of this proposal for future consideration of validation status approval.

Discussions

The US Integrated Ocean Observing System (IOOS) Animal Telemetry Network (ATN) Data Assembly Center (DAC) receives and curates real time and delayed mode data from tags attached to marine animals. These tags typically provide measurements of the sea surface temperature along a trajectory whist the animal is on the surface and periodic profiles from dives as the animals forage for food. The profiles include measurements of the temperature, pressure, and salinity.

A small subset of this data is distributed by the community over the GTS, but currently in the TAC TESAC format (FM-64). The IOOS ATN aims to start distributing more of this data over the GTS in real time and the draft template seeks to meet the requirements for doing so. Coordination within the community is ongoing.

Detailed proposal

New Table D Entries

Category 15: Oceanographic report sequences

Table Reference	Table References	Element Name	Description
3-15-013		Sequence for reporting trajectory profile data from marine animal tags
	3-01-150	WIGOS Identifier
	0-01-087	WMO marine observing platform extended identifier	WMO number where assigned
	2-08-032	Change width of CCITT IA5	change width to 32 characters
	0-01-019	Ship or mobile land station identifier	Platform ID, e.g. ct145-933-BAT2-18 (max 32 characters)
	2-08-000	Change width of CCITT IA5	Cancel change width
	0-03-001	Surface station type	11 (Marine animal)
	0-22-067	Instrument type of water temperature and / or salinity measurement	set to 995 (attached to marine animal)
	1-14-000	Delayed replication of 14 descriptors
	0-31-001	Delayed descriptor replication factor
	0-08-021	Time significance	set to 26, time of last known position
	3-01-011	(Year, month, day)
	3-01-012	(Hour, minute)
	3-01-021	(Latitude / longitude (high accuracy))
	0-01-051	Platform transmitter ID number	e.g. Argos PTT
	0-02-148	Data collection and / or location system
	0-01-012	Direction of motion of moving observing platform
	0-01-014	Platform drift speed (high precision)
	0-33-022	Quality of buoy satellite transmission
	0-33-023	Quality buoy location
	0-33-027	Location quality class (range of radius of 66% confidence)
	0-07-063	Depth below sea / water surface
	0-22-045	Sea / water temperature
	0-08-021	Time significance	Set to missing / cancel
	1-07-000	Delayed replication of 7 descriptors
	0-31-001	Delayed descriptor replication factor
	3-01-011	(Year, month, day)
	3-01-012	(Hour, minute)
	3-01-021	(Latitude / longitude (high accuracy))
	0-01-079	Unique identifier for the profile	Profile ID
	0-01-023	Observation sequence number	Upcast number
	0-22-056	Direction of profile	Set to 0 (ascending / upwards)
	3-06-035	Temperature and salinity profile

Additions to code tables

0-03-001
Surface station type

Code figure	Meaning
0	Land station (synoptic network)
1	Shallow water station (fixed to sea/lake floor)
2	Ship
3	Rig/platform
4	Moored buoy
5	Drifting buoy (or drifter)
6	Ice buoy
7	Land station (local network)
8	Land vehicle
9	Autonomous marine vehicle
10	Reserved
* 11	Tag attached to marine animal
12 - 30	Reserved for future use
31	Missing value

New entry (11) indicated by *.

Attachments

Excel document showing expanded sequences
atn_bufr_template_draft.xlsx

[david-i-berry]

Hi Enrico,

Sorry, I meant to flag this sooner. After the call I've been in conversation with the people behind the template and we'd like to aim for the next FT, not FT-2021-1. Would that be possible.

Thanks,

Dave.

[david-i-berry]

Updated to reflect changes as part of validation, move of PTT ID field and transmitter ID / location method out of first replication block. Revised sequence below, changes flagged using *** in description column:

New Table D Entries

Category 15: Oceanographic report sequences

Table Reference	Table References	Element Name	Description
3-15-013		Sequence for reporting trajectory profile data from marine animal tags
	3-01-150	WIGOS Identifier
	0-01-087	WMO marine observing platform extended identifier	WMO number where assigned
	2-08-032	Change width of CCITT IA5	change width to 32 characters
	0-01-019	Ship or mobile land station identifier	Platform ID, e.g. ct145-933-BAT2-18 (max 32 characters)
	2-08-000	Change width of CCITT IA5	Cancel change width
	0-03-001	Surface station type	11 (Marine animal)
	0-22-067	Instrument type of water temperature and / or salinity measurement	set to 995 (attached to marine animal)
	0-01-051	Platform transmitter ID number	e.g. Argos PTT *** modified (move)
	0-02-148	Data collection and / or location system	*** modified (move)
	1-12-000	Delayed replication of 12 descriptors	*** modified (change from 14 to 12 descriptors
	0-31-001	Delayed descriptor replication factor
	0-08-021	Time significance	set to 26, time of last known position
	3-01-011	(Year, month, day)
	3-01-012	(Hour, minute)
	3-01-021	(Latitude / longitude (high accuracy))
	0-01-012	Direction of motion of moving observing platform
	0-01-014	Platform drift speed (high precision)
	0-33-022	Quality of buoy satellite transmission
	0-33-023	Quality buoy location
	0-33-027	Location quality class (range of radius of 66% confidence)
	0-07-063	Depth below sea / water surface
	0-22-045	Sea / water temperature
	0-08-021	Time significance	Set to missing / cancel
	1-07-000	Delayed replication of 7 descriptors
	0-31-001	Delayed descriptor replication factor
	3-01-011	(Year, month, day)
	3-01-012	(Hour, minute)
	3-01-021	(Latitude / longitude (high accuracy))
	0-01-079	Unique identifier for the profile	Profile ID
	0-01-023	Observation sequence number	Upcast number
	0-22-056	Direction of profile	Set to 0 (ascending / upwards)
	3-06-035	Temperature and salinity profile

[david-i-berry]

Sample file and decoded values attached within zip archive.

Sample file (example-profile.bufr) encoded by Axiom using custom encoder and tables.
Output from two different decoders

• eccodes (example-profile_eccodes.json). Table D definition file (sequence.def, renamed to sequence.txt for upload) attached.
• python decoder using attached B and D tables (example-profile.csv)

Expected values in attached Excel spread sheet, decoded values in agreement.

1. sample and decoded values Archive.zip
2. expected values example-profile.xlsx
3. table B BUFRCREX_TableB_en.txt
4. table D BUFR_TableD_en.txt
5. table D for eccodes sequence.txt

[david-i-berry]

I've believe the sample validated as it's been successfully decoded using two independent pieces of software but both have been run by me.

[david-i-berry]

@chenxiaoxia2019 Please use the BUFRCREX_CodeFlag_en_03.csv and BUFR_TableD_en_15.csv from the attached zip file when creating the branch. There are no new Table B entries.

Archive.zip

[chenxiaoxia2019]

@DavidBerryNOC Hi, David, a new branch for this issue has been created. Could you please check it? Thanks.

[david-i-berry]

@chenxiaoxia2019 the branch has been checked (both visually and in decoding the test data) and is correct, many thanks.

[amilan17]

Please check validation status
@SibylleK

[david-i-berry]

This was accepted as validated back in June. It missed the inclusion in FT-2020-2 drafting and I failed to pick that up (it's possible the card hadn't been moved to the correct column in the project board). I had sent an email on this a couple of weeks ago that I've been meaning to follow up on (and whether we could use the sequence pre-operationally).

[SibylleK]

Just to confirm the validation, here is the output of the DWD BUFR software using the branch for issue-17.
example-profile.bufr_DWDout.txt
The values correspond to the expected values.

[amilan17]

@DavidBerryNOC 

https://github.com/wmo-im/BUFR4/tree/issue-17-FT 

Can you verify that the modification in this issue are correct and, if so, move to 'validated' column?  

(branch issue-17 is really out of sync with Master, this was easier for me than trying to fix conflicts found while trying to rebase)

[david-i-berry]

Branch

https://github.com/wmo-im/BUFR4/tree/issue-17-ft

is correct (updated on the 22nd Jan). The other branch (https://github.com/wmo-im/BUFR4/tree/issue-17-FT, updated last November) is not correct and does not include the new sequence.

[david-i-berry]

I'm unable to move to the validated column, this may be a permissions issue.

[amilan17]

@DavidBerryNOC  - Thanks. I will delete issue-17-FT. Do the codes in issue-17-ft branch need revalidation?

[SibylleK]

I got the same result using the entries of branch issue-17-ft for the decoding of Davids example from last June.

[david-i-berry]

Thanks Sibylle!

[jitsukoh]

@amilan17 @efucile Do you have the approximate date for table publication and the implementation date for FT2021-1? As Enrico suggested at the meeting last week, if the new tables are published earlier, probably there is no need for using the pre-operational status. It was unfortunate that the issue fell into the gap during the transition of our team and I feel that we need to make this new sequence usable as soon as possible.