EnumValues fields often have the wrong size

[alexphredorg]

As an example this field comes from PGN 129038:
{
"Order":15,
"Id":"navStatus",
"Name":"Nav Status",
"BitLength":8,
"BitOffset":200,
"BitStart":0,
"Type":"Lookup table",
"Signed":false,
"EnumValues":[
{"0": "Under way using engine"},
{"1": "At anchor"},
{"2": "Not under command"},
{"3": "Restricted manoeuverability"},
{"4": "Constrained by her draught"},
{"5": "Moored"},
{"6": "Aground"},
{"7": "Engaged in Fishing"},
{"8": "Under way sailing"}]},

The BitLength here is 8, but the table only requires 4 bits to store the 9 values. Simrad and Raymarine will pad the un-specified bits with 1, so the raw value for this will show up as 240 (0b11110000) when it should be 0. The top 4 bits are reserved, and NMEA 2000 recommends that reserved fields keep the bit set to 1.

I've found this to be true of almost all EnumValues in the system, here is another example that is 1 bit with 1 bit reserved, but pgns.json has it as two bits (from pgn 127250):
{
"Order":5,
"Id":"reference",
"Name":"Reference",
"BitLength":2,
"BitOffset":56,
"BitStart":0,
"Type":"Lookup table",
"Signed":false,
"EnumValues":[
{"0": "True"},
{"1": "Magnetic"}]}]},

I've worked around this in my own application by computing the most bits necessary to store all enum values, and assuming that any higher bits are reserved.

[keesverruijt]

@alexphredorg

You do realise that the source data is (a) reverse engineered, so we don't know what field sizes are and whether there are any hidden reserved fields until we see them in actual datagrams in the field and (b) the actual source is in the C definitions.

So if you want to contribute, please show your point PGN by PGN by uploading a single raw message in hex format as understood by analyzer, along with the output of analyzer -raw -data and explain which fields you see that are wrong.

Even better, send me a pull request with an updated pgn.h.

[alexphredorg]

Here is the example for 129038:
2015-04-02-18:28:22.000 4 001 255 129038 : a0 1b c1 78 cf e8 15 64 2015-04-02-18:28:22.000 4 001 255 129038 : a1 75 13 b7 f7 74 5a 1c 2015-04-02-18:28:22.000 4 001 255 129038 : a2 b0 b9 29 00 00 00 00 2015-04-02-18:28:22.000 4 1 255 129038 AIS Class A Position Report: C1 78 CF E8 15 64 75 13 B7 F7 74 5A 1C B0 B9 29 00 00 00 00 00 4F 77 00 00 F0 FE 2015-04-02-18:28:22.000 4 1 255 129038 AIS Class A Position Report: . x . . . d u . . . t Z . . . . . . . . . O w . . . . 2015-04-02-18:28:22.000 4 1 255 129038 AIS Class A Position Report: decode Message ID offset=0 startBit=0 bits=6 bytes=1:(en f=Message ID,sb=0,b=0) (d=c1,bib=6,fb=0,msk=3f,v=1,mag=0) (v=1,m=3f) Message ID = 1 decode Repeat Indicator offset=0 startBit=6 bits=2 bytes=1:(en f=Repeat Indicator,sb=6,b=6) (d=c1,bib=2,fb=6,msk=c0,v=3,mag=0) (v=3,m=3) ; Repeat Indicator = Unknown decode User ID offset=1 startBit=0 bits=32 bytes=4:(en f=User ID,sb=0,b=0) (d=78,bib=8,fb=0,msk=ff,v=78,mag=0) (d=cf,bib=8,fb=0,msk=ff,v=cf,mag=8) (d=e8,bib=8,fb=0,msk=ff,v=e8,mag=10) (d=15,bib=8,fb=0,msk=ff,v=15,mag=18) (v=15e> decode Longitude offset=5 startBit=0 bits=32 bytes=4:(-1223461532) ; Longitude = -122.3461532 decode Latitude offset=9 startBit=0 bits=32 bytes=4:(475690231) ; Latitude = 47.5690231 decode Position Accuracy offset=13 startBit=0 bits=1 bytes=1:(en f=Position Accuracy,sb=0,b=0) (d=b0,bib=1,fb=0,msk=1,v=0,mag=0) (v=0,m=1) ; Position Accuracy = Low decode RAIM offset=13 startBit=1 bits=1 bytes=1:(en f=RAIM,sb=1,b=1) (d=b0,bib=1,fb=1,msk=2,v=0,mag=0) (v=0,m=1) ; RAIM = not in use decode Time Stamp offset=13 startBit=2 bits=6 bytes=1:(en f=Time Stamp,sb=2,b=2) (d=b0,bib=6,fb=2,msk=fc,v=2c,mag=0) (v=2c,m=3f) ; Time Stamp = 44 decode COG offset=14 startBit=0 bits=16 bytes=2:(en f=COG,sb=0,b=0) (d=b9,bib=8,fb=0,msk=ff,v=b9,mag=0) (d=29,bib=8,fb=0,msk=ff,v=29,mag=8) (v=29b9,m=ffff) ; COG = 61.2 deg decode SOG offset=16 startBit=0 bits=16 bytes=2:(en f=SOG,sb=0,b=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=8) (v=0,m=ffff) ; SOG = 0.00 m/s decode Communication State offset=18 startBit=0 bits=19 bytes=3:(en f=Communication State,sb=0,b=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=8) (d=0,bib=3,fb=0,msk=7,v=0,mag=10) (v=0,m=7ffff) ; Communica> decode AIS Transceiver information offset=20 startBit=3 bits=5 bytes=1:(en f=AIS Transceiver information,sb=3,b=3) (d=0,bib=5,fb=3,msk=f8,v=0,mag=0) (v=0,m=1f) ; AIS Transceiver information = Channel A VDL reception decode Heading offset=21 startBit=0 bits=16 bytes=2:(en f=Heading,sb=0,b=0) (d=4f,bib=8,fb=0,msk=ff,v=4f,mag=0) (d=77,bib=8,fb=0,msk=ff,v=77,mag=8) (v=774f,m=ffff) ; Heading = 175.0 deg decode Rate of Turn offset=23 startBit=0 bits=16 bytes=2:(en f=Rate of Turn,sb=0,b=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=8) (v=0,m=7fff) ; Rate of Turn = 0.000 deg/s decode Nav Status offset=25 startBit=0 bits=8 bytes=1:(en f=Nav Status,sb=0,b=0) (d=f0,bib=8,fb=0,msk=ff,v=f0,mag=0) (v=f0,m=ff) ; Nav Status = 240 decode Reserved for Regional Applications offset=26 startBit=0 bits=8 bytes=1:(en f=Reserved for Regional Applications,sb=0,b=0) (d=fe,bib=8,fb=0,msk=ff,v=fe,mag=0) (v=fe,m=ff) ; Reserved for Regional Applications = ERROR

You'll see that NavStatus = 240 (0xF0, or 0b11110000). If you process this as 3 bits instead you get 0, which is "Under Way Using Engines".

Here is another with Nav Status = 248:
2015-04-02-18:28:22.000 2 032 255 128259 : 61 3b 01 ff ff ff ff ff 2015-04-02-18:28:22.000 4 001 255 129038 : 40 1b c1 aa eb e0 15 25 2015-04-02-18:28:22.000 4 001 255 129038 : 41 20 0e b7 c5 3b 68 1c 2015-04-02-18:28:22.000 4 001 255 129038 : 42 b0 97 0b 00 00 00 00 2015-04-02-18:28:22.000 4 1 255 129038 AIS Class A Position Report: C1 AA EB E0 15 25 20 0E B7 C5 3B 68 1C B0 97 0B 00 00 00 00 00 FF FF FF 7F F8 FE 2015-04-02-18:28:22.000 4 1 255 129038 AIS Class A Position Report: . . . . . . . . . . . h . . . . . . . . . . . . . . . 2015-04-02-18:28:22.000 4 1 255 129038 AIS Class A Position Report: decode Message ID offset=0 startBit=0 bits=6 bytes=1:(en f=Message ID,sb=0,b=0) (d=c1,bib=6,fb=0,msk=3f,v=1,mag=0) (v=1,m=3f) Message ID = 1 decode Repeat Indicator offset=0 startBit=6 bits=2 bytes=1:(en f=Repeat Indicator,sb=6,b=6) (d=c1,bib=2,fb=6,msk=c0,v=3,mag=0) (v=3,m=3) ; Repeat Indicator = Unknown decode User ID offset=1 startBit=0 bits=32 bytes=4:(en f=User ID,sb=0,b=0) (d=aa,bib=8,fb=0,msk=ff,v=aa,mag=0) (d=eb,bib=8,fb=0,msk=ff,v=eb,mag=8) (d=e0,bib=8,fb=0,msk=ff,v=e0,mag=10) (d=15,bib=8,fb=0,msk=ff,v=15,mag=18) (v=15e> decode Longitude offset=5 startBit=0 bits=32 bytes=4:(-1223811035) ; Longitude = -122.3811035 decode Latitude offset=9 startBit=0 bits=32 bytes=4:(476593093) ; Latitude = 47.6593093 decode Position Accuracy offset=13 startBit=0 bits=1 bytes=1:(en f=Position Accuracy,sb=0,b=0) (d=b0,bib=1,fb=0,msk=1,v=0,mag=0) (v=0,m=1) ; Position Accuracy = Low decode RAIM offset=13 startBit=1 bits=1 bytes=1:(en f=RAIM,sb=1,b=1) (d=b0,bib=1,fb=1,msk=2,v=0,mag=0) (v=0,m=1) ; RAIM = not in use decode Time Stamp offset=13 startBit=2 bits=6 bytes=1:(en f=Time Stamp,sb=2,b=2) (d=b0,bib=6,fb=2,msk=fc,v=2c,mag=0) (v=2c,m=3f) ; Time Stamp = 44 decode COG offset=14 startBit=0 bits=16 bytes=2:(en f=COG,sb=0,b=0) (d=97,bib=8,fb=0,msk=ff,v=97,mag=0) (d=b,bib=8,fb=0,msk=ff,v=b,mag=8) (v=b97,m=ffff) ; COG = 17.0 deg decode SOG offset=16 startBit=0 bits=16 bytes=2:(en f=SOG,sb=0,b=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=8) (v=0,m=ffff) ; SOG = 0.00 m/s decode Communication State offset=18 startBit=0 bits=19 bytes=3:(en f=Communication State,sb=0,b=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=8) (d=0,bib=3,fb=0,msk=7,v=0,mag=10) (v=0,m=7ffff) ; Communica> decode AIS Transceiver information offset=20 startBit=3 bits=5 bytes=1:(en f=AIS Transceiver information,sb=3,b=3) (d=0,bib=5,fb=3,msk=f8,v=0,mag=0) (v=0,m=1f) ; AIS Transceiver information = Channel A VDL reception decode Heading offset=21 startBit=0 bits=16 bytes=2:(en f=Heading,sb=0,b=0) (d=ff,bib=8,fb=0,msk=ff,v=ff,mag=0) (d=ff,bib=8,fb=0,msk=ff,v=ff,mag=8) (v=ffff,m=ffff) ; Heading = Unknown decode Rate of Turn offset=23 startBit=0 bits=16 bytes=2:(en f=Rate of Turn,sb=0,b=0) (d=ff,bib=8,fb=0,msk=ff,v=ff,mag=0) (d=7f,bib=8,fb=0,msk=ff,v=7f,mag=8) (v=7fff,m=7fff) ; Rate of Turn = Unknown decode Nav Status offset=25 startBit=0 bits=8 bytes=1:(en f=Nav Status,sb=0,b=0) (d=f8,bib=8,fb=0,msk=ff,v=f8,mag=0) (v=f8,m=ff) ; Nav Status = 248 decode Reserved for Regional Applications offset=26 startBit=0 bits=8 bytes=1:(en f=Reserved for Regional Applications,sb=0,b=0) (d=fe,bib=8,fb=0,msk=ff,v=fe,mag=0) (v=fe,m=ff) ; Reserved for Regional Applications = ERROR

Drop the top 4 bits and you have 8, or "Under Way Sailing".

As I've mentioned in other places I'm consuming pgns.json, not using the full canboat suite. I didn't even have a way to run the canboat analyzer until the recent checkin to make it compatible with other versions of candump.

[keesverruijt]

What AIS are you using? My NAIS-400 produces a full 8 bit value and not a four bit value, so it looks okay to me with my device:

2011-04-25-09:39:18.877 4   8 255 129038 AIS Class A Position Report:  01 B2 39 95 0E 49 2B 27 03 EE 43 69 1F 73 00 00 00 00 20 86 A1 51 5C 91 1C 00 FE
2011-04-25-09:39:18.877 4   8 255 129038 AIS Class A Position Report:   .  .  9  .  .  I  .  .  .  .  C  i  .  s  .  .  .  .  .  .  .  Q  .  .  .  .  .
2011-04-25-09:39:18.877 4   8 255 129038 AIS Class A Position Report:
decode Message ID offset=0 startBit=0 bits=6 bytes=1:(en f=Message ID,sb=0,b=0) (d=1,bib=6,fb=0,msk=3f,v=1,mag=0) (v=1,m=3f)   Message ID = 1
....
decode Rate of Turn offset=23 startBit=0 bits=16 bytes=2:(en f=Rate of Turn,sb=0,b=0) (d=91,bib=8,fb=0,msk=ff,v=91,mag=0) (d=1c,bib=8,fb=0,msk=ff,v=1c,mag=8) (v=1c91,m=7fff) ; Rate of Turn = 13.094 deg/s
decode Nav Status offset=25 startBit=0 bits=8 bytes=1:(en f=Nav Status,sb=0,b=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=0) (v=0,m=ff) ; Nav Status = Under way using engine
decode Reserved for Regional Applications offset=26 startBit=0 bits=8 bytes=1:(en f=Reserved for Regional Applications,sb=0,b=0) (d=fe,bib=8,fb=0,msk=ff,v=fe,mag=0) (v=fe,m=ff) ; Reserved for Regional Applications = ERROR

This is the sort of thing that happens -- manufacturers produce slightly out of spec data, and I can only correct this if you report each wrong field, with the data as above and device type. Please use a new issue for every field that you find wrong.

[alexphredorg]

Simrad RS35

The Lowrance Link8 runs similar if not identical firmware.

alex

---

From: Kees Verruijt notifications@github.com
Sent: Thursday, April 2, 2015 1:45 PM
To: canboat/canboat
Cc: Alex Wetmore
Subject: Re: [canboat] EnumValues fields often have the wrong size (#36)

What AIS are you using? My NAIS-400 produces a full 8 bit value and not a four bit value, so it looks okay to me with my device:

2011-04-25-09:39:18.877 4 8 255 129038 AIS Class A Position Report: 01 B2 39 95 0E 49 2B 27 03 EE 43 69 1F 73 00 00 00 00 20 86 A1 51 5C 91 1C 00 FE
2011-04-25-09:39:18.877 4 8 255 129038 AIS Class A Position Report: . . 9 . . I . . . . C i . s . . . . . . . Q . . . . .
2011-04-25-09:39:18.877 4 8 255 129038 AIS Class A Position Report:
decode Message ID offset=0 startBit=0 bits=6 bytes=1:(en f=Message ID,sb=0,b=0) (d=1,bib=6,fb=0,msk=3f,v=1,mag=0) (v=1,m=3f) Message ID = 1
....
decode Rate of Turn offset=23 startBit=0 bits=16 bytes=2:(en f=Rate of Turn,sb=0,b=0) (d=91,bib=8,fb=0,msk=ff,v=91,mag=0) (d=1c,bib=8,fb=0,msk=ff,v=1c,mag=8) (v=1c91,m=7fff) ; Rate of Turn = 13.094 deg/s
decode Nav Status offset=25 startBit=0 bits=8 bytes=1:(en f=Nav Status,sb=0,b=0) (d=0,bib=8,fb=0,msk=ff,v=0,mag=0) (v=0,m=ff) ; Nav Status = Under way using engine
decode Reserved for Regional Applications offset=26 startBit=0 bits=8 bytes=1:(en f=Reserved for Regional Applications,sb=0,b=0) (d=fe,bib=8,fb=0,msk=ff,v=fe,mag=0) (v=fe,m=ff) ; Reserved for Regional Applications = ERROR

This is the sort of thing that happens -- manufacturers produce slightly out of spec data, and I can only correct this if you report each wrong field, with the data as above and device type. Please use a new issue for every field that you find wrong.

Reply to this email directly or view it on GitHubhttps://github.com//issues/36#issuecomment-89040636.

[keesverruijt]

On 02 Apr 2015, at 22:48, alexphredorg notifications@github.com wrote:

Simrad RS35

Ah. It seems it uses a completely different implementation than the SRT base that most AIS devices that produce NMEA 2000. So we learned something today.

On a more positive note, I just noticed that the ITU has opened up considerably and that (amongst others) ITU 1371-4 is now free to download, so we can reliably determine all nav statuses now — a few years ago when most of this data was derived this was behind a paywall. See https://www.itu.int/dms_pubrec/itu-r/rec/m/R-REC-M.1371-4-201004-S!!PDF-E.pdf https://www.itu.int/dms_pubrec/itu-r/rec/m/R-REC-M.1371-4-201004-S!!PDF-E.pdf