Key/value not found error for keys that exists

[liamtill]

Hi,

I've recently starting implementing pdbufr in my BUFR decoding. It has been working well until testing on a larger sample of files. I have encountered the error:

gribapi.errors.KeyValueNotFoundError: Key/value not found

Whilst trying to extract using:

pdbufr.read_bufr(bufr_file, columns=('stationNumber', 'timePeriod', 'maximumWindGustDirection', 'maximumWindGustSpeed'))

The data are BUFR files provided by the Met Office. Using bufr_dump on (one of) the BUFR files I can see that the keys do exists and have determined that is is the timePeriod key causing the error:

A sample from bufr_dump showing the keys existing. The keys exist for all stations within the file. :

[

                                                                    {
                                                                      "key" : "timePeriod",
                                                                      "value" : -10,
                                                                      "units" : "min"
                                                                    },
                                                                    {
                                                                      "key" : "maximumWindGustDirection",
                                                                      "value" : 190,
                                                                      "units" : "deg"
                                                                    },
                                                                    {
                                                                      "key" : "maximumWindGustSpeed",
                                                                      "value" : 4.9,
                                                                      "units" : "m/s"
                                                                    }
                                                                  ],
                                                                  [

                                                                    {
                                                                      "key" : "timePeriod",
                                                                      "value" : -60,
                                                                      "units" : "min"
                                                                    },
                                                                    {
                                                                      "key" : "maximumWindGustDirection",
                                                                      "value" : 190,
                                                                      "units" : "deg"
                                                                    },
                                                                    {
                                                                      "key" : "maximumWindGustSpeed",
                                                                      "value" : 5.8,
                                                                      "units" : "m/s"
                                                                    }
                                                                  ],
                                                                  [

                                                                    {
                                                                      "key" : "timePeriod",
                                                                      "value" : -180,
                                                                      "units" : "min"
                                                                    },
                                                                    {
                                                                      "key" : "maximumWindGustDirection",
                                                                      "value" : 210,
                                                                      "units" : "deg"
                                                                    },
                                                                    {
                                                                      "key" : "maximumWindGustSpeed",
                                                                      "value" : 8.4,
                                                                      "units" : "m/s"
                                                                    }
                                                                  ],

I noted issue #24 stating a similar issue, however I am using build of eccodes 2.29.0 built from source with pdbufr 0.9.0. The user in #24 stated updating solved their issue, but in this case I was already using 2.16.0 and updated to 2.29.0 to see if that resolved the issue.

Any ideas what might be causing this error given keys exists in the file?

Thanks

[sandorkertesz]

Hi Liam,

Thank you for reporting this issue. Please can I ask you to share either the input data (at least a few messages) or the full bufr dump for a given message with us?

Many thanks,
Sandor

[liamtill]

Hi Sandor,

Thanks for your reply. I have attached the BUFR file causing the issue. I use both UK and EU data and there are a few files where, as with this file, the extraction fails for the same Key/Value not found. I have renamed the file from .dat to .txt to upload here.

syWEU_BUFR_0705.txt

Thanks,
Liam

[sandorkertesz]

Hi Liam,

Thank you for the example file. I can conclude that the error comes from message 51 (if we count from 1) where one of the timePeriod values cannot be accessed, so this is probably a badly encoded BUFR message. Unfortunately, pdfbufr crashes without correctly reporting the real nature of the problem.

Thanks,
Sandor

[liamtill]

Hi Sandor,

OK great, thanks for investigating. Yes, I think still being able to extract all other messages and even all the other data for a given station message even if a key is missing would be great. Also having a way to raise this as an exception such that other data can be extracted but we know the which key is missing and can decide on what to do by catching the exception.

Thanks for your great work.

Liam