-
Notifications
You must be signed in to change notification settings - Fork 7
FAQ
- How do I use the API to code a delayed replication?
- What version of Tables B and D do I have? Where can I get the latest?
- Can you make a Red Hat (Centos, Suse...) package?
- Is there a preferred Linux distribution for libecbufr?
- How can I download a tarball (tar.gz) of libecbufr?
- How do I create a BUFR message with random values?
- Is libecbufr thread-safe?
- What development packages do I need to compile libECBUFR from source?
- How do we contact the libECBUFR team?
- Is there documentation for libECBUFR?
- Running libECBUFR experimentally under Microsoft Windows (R)
- How do I install libecbufr?
- How do I get the BUFR decoding/encoding utilities to work?
The following link shows how to do it in C:
http://bazaar.launchpad.net/~libecbufr-dev/libecbufr/trunk/view/head:/Examples/encode_delayed_repl.c
And this one how to do it in Perl:
Roughly speaking, the approach is:
- create the template
- create the dataset from the template
- create a datasubset from the dataset
- expand the datasubset (if necessary, such as the replication descriptor being buried in a Table C)
- find the delayed replication descriptor in the datasubset
- set the value of the replication descriptor to your replication $count
- expand the datasubset (again). Note that if you didn't mess with anything else, the result of the previous find will still be valid, and now there will be $count blank and expanded replicated elements just after that position.
- set the values for the replicated descriptors.
Thanks to Chris Beauregard for providing the source material
The latest version of the Tables available for libECBUFR is found in the Tables directory of the trunk code tree, at http://bazaar.launchpad.net/~libecbufr-dev/libecbufr/trunk/files/head:/Tables/ . You can update the Tables in your installation by replacing the existing table_b_bufr and table_d_bufr files with the new ones.
To find out which version you currently have in your own installation, you need to look into the first few lines of file /usr/share/libecbufr/table_b_bufr. The first portion of the file is a changelog. Look for the topmost instance of the string "Update to Version N" (probably on line 6 of the file). N will be the current version.
Here is some background information I recommend you read, to get a better understanding of the meaning of the version number. Tables B and D are always updated at the same time and share the same version number. This is more formally referred to as the "Master Table version number".
The BUFR encoding and decoding process makes reference to Tables B and D (as defined in the BUFR regulations). Broadly speaking, these tables contain "constant" metadata about the BUFR descriptors and so help keep the messages compact by avoiding having to transmit repetitive business metadata with each bulletin. Just as importantly, Tables B and D can be updated and augmented without requiring changes to BUFR software. This is one of the defining features of the WMO Table-Driven Code Forms, and of BUFR in particular. This is useful because it makes it possible to increase the business content of the Tables and create new data products without requiring software modifications. This was not the case with the Traditional Alphanumeric Code Forms.
With this approach, for the recipient of a BUFR message to be able to decode it, she needs her version of the Tables B and D to contain the same descriptor definitions as the sender's version. The Table version number used for encoding is to be specified in Section 1 of the BUFR message, and ideally, this number would be read by the decoder and same version of the Tables would be used at decoding. If both instances of the tables are error-free, this would ensure correct decoding of the message. In practice, this has been a difficult point of implementation for BUFR. Far too frequently, the version number stated in Section one is unreliable.
In historical practice, every effort has been made to keep successive versions of the Tables backward-compatible. This makes it so that a message encoded with descriptors valid in, say, Table version 15 will remain unchanged and decodable through subsequent versions. For instance, a message encoded with version 15 of the Tables will be decodable if you operate with version 23 of the Tables. However, if your decoder only has version 15 of the Tables, there is no guarantee you will be able to decode a message encoded with version 23.
There was (only) one episode of a breakdown of backward compatibility. For a few element descriptors, there was a break of compatibility between versions 13 and 14 of the Tables. Unfortunately, this can affect BUFR SYNOP messages and we strongly recommend always encoding SYNOP data with version 14 or later of the Tables.
The latest version of the Tables is published by the WMO in May and November each year. There is a bit of a time lag as we (the MSC), transcribe the information and test the new Tables in our systems, and then publish it in the libECBUFR source code tree.
I'm afraid not. We are able to generate Debian and Ubuntu packages because that is what Environment Canada uses internally. We have no access to other distributions. To use libecbufr on other distributions, you will need to compile the source code. Instructions to download the code and compile it can be found in the other FAQs.
Not really. Environment Canada uses Ubuntu, but that should not be taken as an endorsement or a statement of preference when it comes to libecbufr. Libecbufr's dependencies are straightforward and can be met by all mainstream distributions. Under casual testing, done outside of Environment Canada, libecbufr has performed identically well with Ubuntu, Debian, Red Hat, Centos and Gentoo.
Having said this, it is up to each organization to decide what constitutes appropriate testing and integration of libecbufr into their own systems.
A source tarball can be downloaded for any bazaar revision from the web interface.
Go to https://code.launchpad.net/libecbufr and click on the latest commit number (e.g. 286) for the branch you wish to download. If unsure of which branch you want, choose "trunk". This will lead you to a page with a "download tarball" link near the top of the page.
For testing or development work, it can sometimes be useful to create a BUFR message from an existing Table D sequence, with only "bogus" or random values. The command to do this is:
bufr_encoder -sequence <sequence_number> -def_values -nbsubset -outbufr
Where sequence_number is a valid Table D entry, such as "307080".
The switch -nbsubset indicates the desired number of subsets and may be omitted if only one subset is required.
In general, no.
For the most part libecbufr can be considered "reentrant", but the API itself is not thread-safe.
About the only assurance we can make for multi-threading is that once bufr_begin_api() is called (by a single thread), multiple threads should be capable of using the library, and that (read-only) BUFR table instances could be shared by multiple threads for processing messages.
However, actions like processing a single message using multiple threads is almost certainly a bad idea without some form of external locking.
In addition to the basic GNU C package (which under a debian-like system should be named "build-essential") you will also need to install automake, autoconf and libtools.
gettext development libraries (gettext-devel) and utilities are needed to process the internationalization information.
Autopoint, a development component of gettext, needs to be installed separately on Debian/Ubuntu (apt-get install autopoint). On Redhat/Fedora, if the initial configuration (reconf) fails and tells you that autopoint is not found, install gettext-devel.
You will also need doxygen to process and install the documentation. If doxygen is not installed, but all the other required packages are present, the "make" and "make install" will stop prematurely but the binaries will be installed properly.
You may leave a question on this site, or, if you prefer, send an e-mail to: yves.pelletier@ec.gc.ca .
If you install libecbufr from source or from the debian package, you may point your browser locally to: file:///usr/share/doc/libecbufr/html-en/index.html .
Otherwise, a documentation package is available in the download area.
Currently, documentation still needs work, but it is a project priority that is seeing some concrete progress. If you are looking for ways to help the project, this may be an area where a little extra help could make a significant difference.
Supporting a Windows version of libECBUFR is beyond the current scope of Environment Canada requirements for libECBUFR.
However, experimentally, libECBUFR and utilities have been successfully compiled in the MinGW environment (Minimalist GNU for Windows). BUFR encoding and decoding have been consistently successful under this environment but the testing we have done so far is relatively minimal and does not cover all possible cases.
If you wish to try this, you will need to download the libECBUFR source code into an existing MinGW environment. Then, following the usual steps for compiling the library and utilities should do the trick. Make sure to use MinGW Installation Manager to install all the packages needed for libECBUFR to compile. Follow the link to "related question", on the upper right of this screen, to get information the steps that led to a successful compilation under MinGW (thanks to Charlie Martin).
libecbufr is distributed as a binary debian package (produced on Debian and Ubuntu). Installing the package should be straightforward, using the Debian-style package manager you like best. Be on the lookout for error messages and broken dependencies. If all goes well you should find the library and utilities in /usr/lib and /usr/bin.
If you wish to compile the source code, you will need the GNU toolchain, which is available for most Unix-like environments today. Aside from the basic GNU binary utilities, you will need to install libtool and automake. The documentation package requires "doxygen". More details here: https://answers.launchpad.net/libecbufr/+faq/1182
To install from debian packages:
-
Download the most appropriate package for your system. Currently only .deb files are available.
-
Install via "dpkg -i"
To compile:
-
Download the source code from Launchpad using bzr (e.g. "bzr branch lp:libecbufr"); change to the "libecbufr" directory.
-
Type "./reconf" and return. This will call the necessary automake and libtool commands to adapt the package to your system.
-
Type "./configure" and return. This will detect system parameters and create the makefiles.
-
Type "make" to trigger compilation of the library and utilities, and to generate the documentation.
-
As the superuser, type make install. This will install the library, executables, and documentation in the system directories.
-
To build a Debian package, type "make pkg". This will (hopefully) create a Debian package.
libECBUFR includes two utilities: bufr_encoder and bufr_decoder.
Make sure to set the environment variable BUFR_TABLES to the directory containing tables B and D. If you installed libECBUFR using the downloadable debian package, the proper command under the most commonly used Unix shells would be:
export BUFR_TABLES=/usr/share/libecbufr
Assuming you have a BUFR file named data.bufr, the basic decoding command is:
bufr_decoder -inbufr data.bufr
By default, the data dump includes the units and definition of the descriptors, to increase human readability.
Now to produce an ascii dump and template suitable for re-encoding back to BUFR:
bufr_decoder -inbufr data.bufr -output data.txt -dump -otemplate template.txt
The -dump switch tells the decoder to omit descriptor name and unit. The -otemplate switch tells the decoder to produce an ASCII dump of the descriptor sequence in section 3. The command will produce two files: data.txt and template.txt. These files may be edited, values altered, etc. This can be useful, for instance, when prototyping a new template or a new BUFR product.
Then, to re-encode, use the following command:
bufr_encoder -datafile data.txt -outbufr newdata.bufr -template template.txt -compress
The switch -compress may be omitted. In that case the BUFR data will be uncompressed.
To view all the available switches, just enter "bufr_decoder" or "bufr_encoder".