Skip to content

scikit-hep/scikit-hep-testdata

Repository files navigation

scikit-hep-testdata

Scikit-HEP PyPI version Conda latest release

Github Actions badge Code Coverage pre-commit.ci status Code style: black

A common package to provide example files (e.g. ROOT) for testing and developing packages against. The sample of files is representative of typical files found "in the wild".

In addition to including some root files directly, this package adds some simple helper methods to get larger files from common open-access data repositories.

Installing and usage

To install:

python -m pip install scikit-hep-testdata

Once installed, absolute file paths can be resolved using the helper methods:

from skhep_testdata import data_path

filename = data_path("some_file.root")

By default, if an unknown file is requested an exception is raised but this can be skipped by passing the above method raise_missing=False:

filename = data_path("unknown_file.root", raise_missing=False)

The files are not stored on PyPI, so if installed from SDist/wheel, the "local" files will not be present, but will be downloaded from GitHub and cached in the ~/.local/skhepdata directory. If you make an editable install from the Git repo, or if you set SKHEP_DATA=1 when building/installing from the Git repo, you will have the data files locally.

You can see all "local" files with skhep_testdata.known_files, and you can download all files at once with skhep_testdata.download_all(), optionally selecting the download cache directory.

Remote vs. Local files

Some files, particularly large ones, for example, are not stored within this package and instead live on a remote server; we call these "remote files". To obtain these use the same data_path method as above, however this will trigger the code to download and configure the remote file. This might be slow the first time round but will subsequently be as fast as for a local file. WARNING: the local file caching system has not yet been applied to remote files.

Command-line invocation

You can also interact with this package from the command-line:

# Print a path (download if needed)
python -m skhep_testdata cms_hep_2012_tutorial/data.root

# Show all "local" files
python -m skhep_testdata --list

# Download all files to an existing directory
python -m skhep_testdata --all --dir local

You can also use pipx run scikit-hep-testdata to access the above CLI without installing.

Adding new files

We're on the look out for new, interesting files!

  • Large files: If the file is particularly large, for example > 25 MB, it might be worth adding to an external open access data repository and adding a configuration here so that the internal helper methods can pull this down.
  • Experiment data policies: Please make sure you have permissions to add the file to this collection, and that any private or sensitive data has been appropriately masked, salted, or scrambled.

List of files

The following lists describe the files known by this package.

Files stored in this package

Known remote files

Contributors

We hereby acknowledge the contributors that made this project possible (emoji key):

benkrikler
benkrikler

πŸ’» πŸ“–
Jim Pivarski
Jim Pivarski

🚧 πŸ”£ πŸ“–
Henry Schreiner
Henry Schreiner

🚧 πŸ”£ πŸ’» πŸ“–
Eduardo Rodrigues
Eduardo Rodrigues

🚧 πŸ”£ πŸ’»
Matthew Feickert
Matthew Feickert

πŸ”£ πŸ’»
Pratyush Das
Pratyush Das

πŸ”£ πŸ’»
Jerry Ling
Jerry Ling

πŸ”£ πŸ’»
Jonas Eschle
Jonas Eschle

πŸ’»
Giordon Stark
Giordon Stark

πŸ”£ πŸ’»
Dmitry Kalinkin
Dmitry Kalinkin

πŸ”£
Michele Peresano
Michele Peresano

πŸ”£
Luis Antonio Obis Aparicio
Luis Antonio Obis Aparicio

πŸ”£
Oksana Shadura
Oksana Shadura

πŸ”£
Nicholas Smith
Nicholas Smith

πŸ”£
Beojan Stanislaus
Beojan Stanislaus

πŸ”£
Lukas
Lukas

πŸ”£
Johannes Schumann
Johannes Schumann

πŸ”£
Elliott Kauffman
Elliott Kauffman

πŸ”£
Tom Eichlersmith
Tom Eichlersmith

πŸ”£
Alexander Puck Neuwirth
Alexander Puck Neuwirth

πŸ”£
ioanaif
ioanaif

πŸ”£

This project follows the all-contributors specification.

Acknowledgements

  • Many of the files collected directly within this package were collated originally by Jim Pivarski for uproot

Running the tests

This package uses pytest to run the unit tests. Install with pip install scikit-hep-testdata[test] or pip install -e .[test] (dev) to get the testing requirements. then run:

pytest