NWC-SAFer

An awesome CLI tool simplifying NetCDF data extraction of NWC-SAF products and transformation to different file formats!

Report Bug · Request Feature

📔 Table of Contents

• 📔 Table of Contents
  • 🌟 About the Project
    • 📚 Tech Stack
  • 🧰 Getting Started
    • 🔧 Prerequisites
    • ⚙️ Installation
      • PyPi (recommended)
      • Source
    • 🏃 Run Locally
  • 📑 Usage
    • Watch directory and convert incoming files
    • Manual file conversion
    • Check compatible products
    • Launch Github repository
  • 🧭 Roadmap
  • ⚠️ License
  • 🤝 Contact

🌟 About the Project

This is a modern and simplified Command Line Interface tool created to extract info and metadata from NetCDF files and convert them into different file formats such as Csv, Excel, plain text, etc.

Its aim is to simplify data extraction from NWC SAF products and their transformation to different file formats, more readable and easier to access and edit. ΝWC-SAF is an initiative of EUMETSAT to provide operational services ensuring the optimum use of meteorological satellite data in Nowcasting and Very Short Range Forecasting.

📚 Tech Stack

The tech stack of this project consists of the following main technologies/frameworks/tools:

• 

• 

• 

• 

• Python Queues and thread handling is applied to enhance user experience

• NetCDF4, Pandas, xarray libraries are used for file manipulation

🧰 Getting Started

🔧 Prerequisites

You just need to have installed Python language preferrably > 3.10 version and a Python package/dependency management tool like Pip, Poetry, Conda etc.

The tool is compatible with all popular OS.

Better used along with CLIs that support emojis and Unicode characters for a better user experience.

⚙️ Installation

PyPi (recommended)

• Install it from PyPi using pip or any other python package manager

    pip install --user nwc-safer

• Enable auto-completion by executing the following command

    nwc-safer --install-completion

Source

• Clone the Git repo

    git clone https://github.com/fotioudim/nwc-safer.git

• Install using poetry in the corresponding directory

    poetry install

🏃 Run Locally

Just run it using nwc-safer. Get help for the available commands

  nwc-safer --help

📑 Usage

Run nwc-safer or nwc-safer -- help in your CLI to get a better description of the available commands and options.

Watch directory and convert incoming files

Constantly watch a directory for incoming NetCDF files, in order to extract the desired data and export them in the form of Csv/Excel/Text files.

  nwc-safer watch [OPTIONS] [INPUT_PATH] [OUTPUT_PATH]

Arguments:

• input_path: The path (relative/absolute) for the directory desired to be watched [default: .]
• output_path: The path (relative/absolute) for the output directory [default: .\output]

Options:

• --lat -la: The latitude coordinate range in ascending order, given in the form of two int values (min & max)
• --lon -lo: The longitude coordinate range in ascending order, given in the form of two int values (min & max)
• --format -f: The output file format [default: csv]
• --recursive -r: Watch for incoming files recursively in all the subdirectories of the specified directory
• --existing -e: Convert pre-existing files in the specified directory

Manual file conversion

Process a single or multiple NetCDF file(s), by extracting the desired data and exporting them in a new file format (eg. Csv, Excel).

  nwc-safer convert [OPTIONS] [FILE_PATHS]...

Arguments:

• file_paths: The path(s) (relative/absolute) for the file(s) desired to be converted [default: None] [required]

Options:

• --lat -la: The latitude coordinate range in ascending order, given in the form of two int values (min & max)
• --lon -lo: The longitude coordinate range in ascending order, given in the form of two int values (min & max)
• --output -o: The path (relative/absolute) for the output directory [default: .\output]
• --format -f: The output file format [default: csv]

Check compatible products

Check which NWC-SAF products are currently supported by the tool

  nwc-safer compatibility

Launch Github repository

Launch NWC-SAF NetCDF Data Exporter Github repository in the web browser

  nwc-safer repo

🧭 Roadmap

• Add continuous watching & conversion operation
• Single conversion operation
• OS agnostic functionality
• Multiple conversion operation
• Subset dataset based on coordinates range
• Support 3 output file formats (.csv, .xlsx, txt)
• Generalize solution for a bigger set of NWC SAF products

⚠️ License

Distributed under the MIT License. See LICENSE.md for more information.

🤝 Contact

You can reach me at:

• Personal website: https://www.dfotiou.gr
• StackOverflow profile: https://stackoverflow.com/users/11680294/fotiou-d
• Linkedin profile: https://www.linkedin.com/in/dimitris-fotiou-4141a8197/
• Email: fotioudimitris93@gmail.com