TG toolset golang

CLI tools and re-usable libraries for interacting and managing various ICT services provided by the DCCN technical group, written in Golang.

Code structure

Currently, the whole package is divided into a few parts, each provides a set of CLI tools. They are:

• dataflow contains tools and libraries for automatic MEG/MRI dataflow.
  • lab_bookings: a CLI for retrieving lab-booking events from the calendar system. It is utilized on the MEG console to provide information for structuring MEG raw data in the project storage.
  • pacs_getstudies: a CLI for retrieving MRI studies from the Orthanc PACS server.
  • pacs_streamdata: a CLI for (re-)streaming data from the Orthanc PACS server.
• project contains tools and libraries for project storage management.
  • prj_getacl: a CLI for getting ACLs of a project storage and translating it to data-access roles (e.g. manager, contributor, viewer).
  • prj_setacl: a CLI for setting ACLs on a project storage to implement data-access roles.
  • prj_delacl: a CLI for deleting ACLs from a project storage to remove data-access roles.
  • prj_mine: a CLI for retrieving the current user's data-access roles in all projects.
  • pdbutil: a project database utility for performing actions such as provisioning storage resource or changing storage quota of project.
• repository contains libraries for repository data management. See dr-tools for repository tools.
• metrics contains tools and libraries for collecting metrics.
  • hpc-metrics-appusage-collector: a daemon collects count on module load on HPC into Prometheus metrics. See README for more detail.
  • lab_usage_report: a CLI to report lab usage to OpenTSDB database (and displayed on https://grafana.dccn.nl)

Various CLIs take a YAML-based configuration file (via the -c option) for setting up connections to, e.g., project database, filers, etc.. An example YAML file is provided here; and the codes that "objectize" the YAML file are located in the pkg/config directory.

Most of the re-usable libraries are written to support the CLI tools listed above. Those libraries are organised in various pkg directories:

• pkg: common libraries shared between the sub-modules.
• dataflow/pkg: libraries for the CLI tools of the MEG/MRI dataflow management.
• project/pkg: libraries for the CLI tools of the project management.
• repository/pkg: libraries for the CLI tools of the repository data management.

Build

To build the CLIs, simply run:

make

After the build, the executable binaries are located in ${GOPATH}/bin directory. If ${GOPATH} is not set in the environment, default is ${HOME}/go.

Release

The Makefile has a target to build a GitHub release with an RPM package as the release asset. For making a GitHub release X.Y.Z, one does:

VERSION=X.Y.Z make github-release

The GitHub personal access token is required for interacting with the GitHub APIs.

Run

All CLI commands have a -h option to print out a brief usage of the command.