by Ned Thaddeus Taylor
ATHENA (Adaptive Training for High Efficiency Neural network Applications) is a Fortran library for developing and handling neural networks (with a focus on convolutional neural networks).
This repository has been migrated from the University of Exeter GitLab to GitHub to facilitate community interaction and support. The latest version, updates, and collaboration now take place on this GitHub repository.
GitLab Repository (Archived): https://git.exeter.ac.uk/hepplestone/athena
It was decided that this project should be migrated to allow for better community support (i.e. allowing community users to raise issues). All information has been ported over where possible. Issues have not been migrated over, these can be found in the old repository. Releases prior to 1.2.0 have not been migrated over, but they can still be found as tags in this repository.
ATHENA is distributed with the following directories:
| Directory | Description |
|---|---|
| example/ | A set of example programs utilising the ATHENA library |
| src/ | Source code |
| tools/ | Additional shell script tools for automating learning |
| test/ | A set of test programs to check functionality of the library works after compilation |
For extended details on the functionality of this library, please check out the wiki
NOTE: There currently exists no manual document. This will be included at a later date
The ATHENA library can be obtained from the git repository. Use the following commands to get started:
git clone https://github.com/nedtaylor/athena.git
cd athena
The library has the following dependencies:
The library has been developed and tested using the following compilers:
- gfortran -- gcc 13.2.0
- ifort -- Intel 2021.10.0.20230609
- ifx -- IntelLLVM 2023.2.0
The library is set up to work with the Fortran Package Manager (fpm).
Run the following command in the repository main directory:
fpm build --profile release
To check whether ATHENA has installed correctly and that the compilation works as expected, the following command can be run:
fpm test
This runs a set of test programs (found within the test/ directory) to ensure the expected output occurs when layers and networks are set up.
Run the following commands in the directory containing CMakeLists.txt:
mkdir build
cd build
cmake [-DCMAKE_BUILD_TYPE="optim;mp"] ..
make install
This will build the library in the build/ directory. All library files will then be found in:
${HOME}/.local/athena
Inside this directory, the following files will be generated:
include/athena.mod
lib/libathena.a
To check whether ATHENA has installed correctly and that the compilation works as expected, the following command can be run:
ctest
This runs a set of test programs (found within the test/ directory) to ensure the expected output occurs when layers and networks are set up.
After the library has been installed, a set of example programs can be compiled and run to test the capabilities of ATHENA on the MNIST dataset. Some of the examples can be run as-is, and do not require external databases. For those that require the MNIST (a set of 60,000 hand-written numbers for training and 10,000 for testing, 0-9) dataset (i.e. 'example/mnist_' directories ), the dataset must first be downloaded. The example program has been developed to accept a text-based format of the MNIST dataset. The .txt database that these examples have been developed for can be found here: https://github.com/halimb/MNIST-txt/tree/master
The link to the original MNIST database is: http://yann.lecun.com/exdb/mnist/
NOTE: For the mnist examples, the MNIST dataset must be downloaded. By default, the database is expected to be found in the directory path ../../DMNIST. However, this can be chaned by editing the following line in the example/mnist[_VAR]/test_job.in file to point to the desired path:
dataset_dir = "../../DMNIST"
Using fpm, the examples are built alongside the library. To list all available examples, use:
fpm run --example --list
To run a particular example, execute the following command:
fpm run --example [NAME]
where [NAME] is the name of the example found in the list.
To compile and run the examples, run the following commands in the directory containing CMakeLists.txt:
cd example/mnist
make build optim [FC=FORTRAN-COMPILER]
./bin/athena_test -f test_job.in
After the example program is compiled, the following directories will also exist:
| Directory | Description |
|---|---|
| example/mnist/bin/ | Contains binary executable |
| example/mnist/obj/ | Contains module/object files (non-linked binary files) |
The example will perform a train over the MNIST dataset. Once complete, it will print its weights and biases to file, and test the trained network on the training set. The output from this can then be compared to the file expected_output_COMPILER.txt.
In the tools/ directory, there exist scripts that take utilise the wandb python package (Weights and Biases, a machine learning data tracker). Wandb is a Python module and, as such, a Python interface has been provided to call and run the Fortran example. The Python interface then reads the Fortran output files and logs the results to the wandb project.
Example wandb project link: https://wandb.ai/ntaylor/cnn_mnist_test/overview?workspace=user-ntaylor
To call/reference the ATHENA library in a program, include the following use statement at the beginning of the necessary Fortran file: use athena
During compilation, include the following flags in the compilation (gfortran) command:
-I${HOME}/.local/athena/include -L${HOME}/.local/athena/lib -lathena
- Ned Thaddeus Taylor
Please note that this project adheres to the Contributing Guide. If you are interested in contributing to this project, please contact Ned Taylor.
This work is licensed under an MIT license.
Automated reporting on unit test code coverage in the README is achieved through utilising the cmake-modules and dynamic-badges-action projects.
| Source file | Description |
|---|---|
| src/athena.f90 | the module file that imports all necessary user-accessible procedures |
| src/lib/mod_accuracy.f90 | accuracy calculation procedures |
| src/lib/mod_activation.f90 | generic node activation (transfer) setup |
| src/lib/mod_activation_[NAME].f90 | [NAME] activation method |
| src/lib/mod_base_layer.f90 | abstract layer construct type |
| src/lib/mod_base_layer_sub.f90 | base layer submodule |
| src/lib/mod_clipper.f90 | gradient clipping procedures |
| src/lib/mod_constants.f90 | a set of global constants used in this code |
| src/lib/mod_container.f90 | layer container construct for handling multiple layers in a network |
| src/lib/mod_container_sub.f90 | layer container submodule |
| src/lib/mod_[NAME]_layer.f90 | [NAME] layer-type |
| src/lib/mod_initialiser.f90 | generic kernel (and bias) initialiser setup |
| src/lib/mod_initialiser_[NAME].f90 | [NAME] kernel initialisation method |
| src/lib/mod_loss.f90 | loss and corresponding derivatives calculation procedures |
| src/lib/mod_lr_decay.f90 | learning rate decay procedures |
| src/lib/mod_metrics.f90 | training convergence metric derived type and procedures |
| src/lib/mod_misc.f90 | miscellaneous procedures |
| src/lib/mod_misc_ml.f90 | miscellaneous machine learning procedures |
| srcs/lib/mod_network.f90 | neural network derived type and procedures |
| srcs/lib/mod_network_sub.f90 | network submodule |
| src/lib/mod_normalisation.f90 | data normalisation procedures |
| src/lib/mod_optimiser.f90 | learning optimisation derived type and procedures |
| src/lib/mod_random.f90 | random number procedures |
| src/lib/mod_tools_infile.f90 | tools to read input files |
| src/lib/mod_types.f90 | neural network-associated derived types |
| Additional file | Description |
|---|---|
| CHANGELOG | human-readable athena codebase version history |
| CMakeLists.txt | the makefile used for compiling the library |
| CONTRIBUTING.md | Guidelines for organisation of athena codebase |
| fpm.toml | Fortran Package Manager (fpm) compilation file |
| LICENSE | licence of ATHENA code |
| README.md | a readme file with a brief description of the code and files |
| TODO | todo-list in addition to useful machine learning and fortran references |
| cmake/CodeCoverage.cmake | cmake-modules file to automate unit test coverage reporting |
| example/example_library | Utility library shared between the examples |
| example/[NAME]/expected_output.txt | expected output from executing [NAME] example program |
| example/[NAME]/test_job.in | input file for [NAME] example program |
| example/[NAME]/src | source directory for [NAME] example program |
| test/test_[NAME]__.f90_ | [NAME] test program to check library expected functionality |
| tools/coverage_badge.py | script to extract code coverage percentage from GitHub Action |
| tools/sweep_init.py | script to initialise wandb sweep |
| tools/sweep_train.py | script to perform training and log learning to wandb |
| tools/template.in | input file for program in test/bin/ (once compiled) |
| tools/wandb-metadata.json | metadata defining default plots on wandb website |