Basic Python libraries used by UGent's HPC group
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
doc
examples
lib/vsc
test
.gitignore
.travis.yml
Jenkinsfile
LICENSE
README.md
setup.py

README.md

vsc-base

Build Status

  • Python 2.6 : Build Status
  • Python 2.7 : Build Status

Description

Common tools used within our organization. Originally created by the HPC team of Ghent University (http://ugent.be/hpc).

Documentation

https://jenkins1.ugent.be/job/vsc-base-python26/Documentation/

Namespaces and tools

lib/utils

python utilities to be used as libraries

  • fancylogger: an extention of the default python logger designed to be easy to use and have a couple of fancy features.

  • custom specifiers for mpi loggin (the mpirank) with autodetection of mpi

  • custom specifier for always showing the calling function's name

  • rotating file handler

  • a default formatter.

  • logging to an UDP server (logdaemon.py f.ex.)

  • easily setting loglevel

  • daemon.py : Daemon class written by Sander Marechal (http://www.jejik.com) to start a python script as a daemon.

  • missing.py: Small functions and tools that are commonly used but not available in the Python (2.x) API.

  • cache.py : File cache to store pickled data identified by a key accompanied by a timestamp. (moved to vsc-utils)

  • generaloption.py : A general option parser for python. It will fetch options (in this order) from config files, from environment variables and from the command line and parse them in a way compatible with the default python optionparser. Thus allowing a very flexible way to configure your scripts. It also adds a few other useful extras.

  • affinity.py : Linux cpu affinity.

  • Based on sched.h and bits/sched.h,

  • see man pages for sched_getaffinity and sched_setaffinity

  • also provides a cpuset class to convert between human readable cpusets and the bit version Linux priority

  • Based on sys/resources.h and bits/resources.h see man pages for getpriority and setpriority

  • asyncprocess.py : Module to allow Asynchronous subprocess use on Windows and Posix platforms

  • Based on a python recipe by Josiah Carlson

  • added STDOUT handle and recv_some

  • daemon.py : A generic daemon class by Sander Marechal

  • dateandtime.py : A module with various convenience functions and classes to deal with date, time and timezone.

  • nagios.py : This module provides functionality to cache and report results of script executions that can readily be interpreted by nagios/icinga.

  • run.py : Python module to execute a command, can make use of asyncprocess, answer questions based on a dictionary

  • supports a whole lot of ways to input, process and output the command. (filehandles, PIPE, pty, stdout, logging...)

  • mail.py : Wrapper around the standard Python mail library.

  • Send a plain text message

  • Send an HTML message, with a plain text alternative

bin

A collection of python scripts, these are examples of how you could use fancylogger to log to a daemon, but should not be used directly.

  • logdaemon.py: A daemon that listens on a port for udp packets and logs them to file, works toghether with fancylogger.
  • startlogdaemon.py : Script that will start the logdaemon for you and set environment variables for fancylogger.

License

vsc-base is made available under the GNU Library General Public License (LGPL) version 2 or any later version.

Acknowledgements

vsc-base was created with support of Ghent University, the Flemish Supercomputer Centre (VSC), the Flemish Research Foundation (FWO), and the Department of Economy, Science and Innovation (EWI).