v2.0.2

Full Changelog: v2.0.1...v2.0.2

v2.0.1: Using JSON files to untangle file messiness

What's Changed

• Pushing production changes back to original codebase by @rsmith013 in #3
• Use JSON files to untangle messiness of data - https://breezy.badc.rl.ac.uk/rsmith013/cci_tagger_json
• Download and use a cache of the vocabs to reduce SPARQL lookups. Local cache uses dictionary lookups.

New Contributors

• @rsmith013 made their first contribution in #3

Full Changelog: v1.0.4...v2.0.1

Updated to use NERC vocab terms

v1.0.4

update version number

Improve use of defaults file

Allow multiple values in the defaults file

MOLES intergration

It is now possible to update the data in MOLES directly

Extra ESGF data and checks

Release Notes

• Providemtime and file size values in the json output for ESGF
• Added a check to see if the time variable in the NetCDF file is a scalar. If it is then reject the file

First release

CCI Tagger

This is the first release of this package

Overview

This package provides a command line tool moles_esgf_tag to generate dataset
tags for both MOLES and ESGF.

Installation

Create a Python virtual environment:

virtualenv tagger
source tagger/bin/activate

Find the latest release of the code and install it in the virtual environment, i.e. for version 1.0.0:

pip install https://github.com/cedadev/cci-tagger/archive/v1.0.0.tar.gz

Usage

moles_esgf_tag [-h] (-f FILE | -d DATASET | -s) [--file_count FILE_COUNT] [--no_check_sum] [-v]

You can tag an individual dataset, or tag all the datasets listed in a file. By default a check sum will be produces for each file.

Arguments:

-h, --help            show help message and exit

-f FILE, --file FILE  the name of the file containing a list of datasets to process. This option is used for
                      tagging one or more datasets.

-d DATASET, --dataset DATASET
                      the full path to the dataset that is to be tagged. This option is used to tag a single
                      dataset.

-s, --show_mappings   show the local vocabulary mappings

-m, --use_mappings    use the local vocabulary mappings. This will map a number of non compliant terms to
                      allowed terms.

--file_count FILE_COUNT
                      how many .nt files to look at per dataset

--no_check_sum        do not produce a check sum for each file

-v, --verbose         increase output verbosity

Output

A number of files are produced as output:

• esgf_drs.json contains a list of DRS and associated files and check sums
• moles_tags.csv contains a list of dataset paths and vocabulary URLs
• moles_esgf_mapping.csv contains mappings between dataset paths and DRS
• error.txt contains a list of errors

Examples

moles_esgf_tag -d /neodc/esacci/cloud/data/L3C/avhrr_noaa-16 -v
moles_esgf_tag -f datapath --file_count 2 --no_check_sum -m -v
moles_esgf_tag -s