Journal – GPEM 2023

This repository provides a means to profile (i.e., benchmark) the evaluation methodologies given by some genetic programming (GP) tools. The evolutionary mechanisms provided by the GP tools are not included when profiling, only mechanisms for calculating "fitness."

This repository was created for the journal paper "Using FPGA Devices to Accelerate the Evaluation Phase of Tree-Based Genetic Programming: An Extended Analysis", which was invited to the Genetic Programming and Evolvable Machines (GPEM) Journal Special Issue titled "Highlights of Genetic Programming 2023 Events." This paper compared the evaluation performance of an initial FPGA-based GP hardware accelerator with that of the GP software tools DEAP (version 1.3), TensorGP (Git revision 09e6d04), and Operon (Git revision 9e7ee4e).

Included Tools

A means for profiling for the following GP tools is given:

• DEAP - for the original paper, click here.
• TensorGP - for the original paper, click here.
• Operon - for the original paper, click here.

Source code for the FPGA accelerator is not provided at this time, although the architecture is described at length in the aforementioned paper, "Using FPGA Devices to Accelerate the Evaluation Phase of Tree-Based Genetic Programming: An Extended Analysis."

Profiling

By default, the repository already contains the results published in the relevant journal paper. These results can be viewed with the experiment/results.ipynb Jupyter Notebook file.

If so desired, after successfully completing installation (as described below), you may run the entire profiling suite by executing the following within a shell program, after having navigated to the repository directory within the shell:

cd experiment
bash run.sh

After the run.sh script fully executes, to view some relevant statistics, run the Jupyter Notebook file given by the path experiment/results.ipynb.

NOTE: Running all experiments will likely take several days.

Installation instructions

The following has been verified via RHEL 8.8 and Ubuntu 20.04. It is likely that other Linux distributions are supported, but it is unlikely that Windows and MacOS operating systems are readily supported.

Prerequisites

• Ensure that some Conda package management system (e.g., Miniconda) is installed on the relevant machine.
• Download the latest software release from GitHub, available here. Extract and paste the contents of the data.zip file (i.e., the results folder) into the experiment folder of the repository.

Upon performing the above, navigate to the root directory of the repository and set up the relevant tool environment with the following:

conda env create -f environment.yml
conda activate journal-gpem-2023
pip install -r requirements.txt
bash install.sh

NOTE: After copying the contents of the data.tar.gz file to the experiment/results folder, you may need to change file permissions for the relevant .pkl files. One way of doing so is by executing the following:

chmod 755 experiment/results/*.pkl

NOTE: If not using a CPU from the Intel Skylake series (like in the journal paper), then you may need to specify a native CPU architecture in the compilation settings for Operon before running the bash install.sh command listed above. One way of doing so is to comment out line 501 in experiment/tools/operon/custom/CMakeLists.txt and uncomment line 502.

NOTE: If using an Nvidia GPU (like in the journal paper), you may need to ensure that tensorflow can successfully utilize a GPU within the conda environment by prepending the following CUDA paths (or something similar) to the $LD_LIBRARY_PATH environment variable:

export LD_LIBRARY_PATH=$CONDA_PREFIX_1/pkgs/cudatoolkit-11.2.2-hbe64b41_10/lib:$CONDA_PREFIX_1/envs/tensorgp-test/lib:$LD_LIBRARY_PATH

If the above export command is executed, you will likely need to restart your shell to reset the $LD_LIBRARY_PATH environment variable after running any experiments.

NOTE: Due to the general stochasticity of CPU execution, some of the results reported in the paper will be irreproducible, e.g., runtime and energy measurements. In addition, if there are discrepancies in hardware/firmware, it is possible that there may exist other differences in results, e.g., due to differing implementations of the IEEE-754 floating-point standard.