Diversity Preservation in Minimal Criterion Coevolution through Resource Limitation

This is the code accompanying the GECCO 2020 paper, Diversity Preservation in Minimal Criterion Coevolution through Resource Limitation.

The project that runs the published MCC experiments is titled "MCC_Executor". This is the contained within the source code zip/tar files. For a description of each execution parameter, refer to MCC Parameters. The following exemplar executes a single run of MCC resource limition with a configuration identical to that in the paper. It assumes that the experiment is being run from the solution directory (i.e. one level up from the MCC_Executor project). Seed agents are given to ensure that each run starts from a common baseline, though a random population can be generated by setting the GenerateAgentPopulation parameter to true. Note that the seed agents are themselves randomly generated, so the bootstrap process (which evolves competent agents that can satisfy the non-trivial MC of maze navigation) must execute regardless.

MCC_Executor.exe maze_navigation \
ExperimentSource=file \
NumRuns=1 \
ExperimentConfigDirectory=./MCC_Executor/MazeNavigation/ExperimentConfigurations/ \
OutputFileDirectory=../ExperimentResults/ \
SeedAgentFile=./MCC_Executor/MazeNavigation/SeedAgents/SeedAgents_Acyclic_20_Genomes_10_Size_2_Waypoints_2_Walls_5_LR.xml \
SeedMazeFile=./MCC_Executor/MazeNavigation/SeedAgents/GeneratedMaze_10_Genomes_10_Height_10_Width_2_Waypoints_2_Walls.xml \
ExperimentNames="MCC-LimitedResources"
StartFromRun=1
IsDistributedExecution=false

MCC Parameters

The below table enumerates all MCC experiment parameters and their function/purpose.

Parameter	Description
ExperimentSource	Source for the experiment configuration (file or database). The default selection is file; however, MCC can also be executed from a configuration persisted to an RDBMS (the schema is contained within the ExperimentEntities project).
NumRuns	Number of runs of MCC to execute.
ExperimentConfigDirectory	Directory containing experiment configuration files (these are the XML files under /MCC_Executor/MazeNavigation/ExperimentConfigurations). This must be specified if ExperimentSource is set to file (which is the default).
OutputFileDirectory	The directory in which to persist experiment CSV output files containing experiment results.
SeedAgentFile	XML file containing seed agent genomes. If this parameter is not specified, a random population will be generated.
SeedMazeFile	XML file containing seed maze genomes. Mazes are not generated by default, so this must be specified.
ExperimentNames	Comma-delimited list of experiments to execute (or just single experiment). The experiment name must match the name of the experiment configuration file.
StartFromRun	Run number at which to begin execution. This is useful when executing multiple runs of the same MCC experiment in parallel.
IsDistributedExecution	This should almost always be set to false. It enables behavior very specific to the HPC environment in which the initial MCC experiments were executed (such as additional logic for interfacing with Slurm, interfacing with external processes for moving data off the cluster, etc.) and is likely of little use in any other environment.

1.3: Merge branch 'master' of https://github.com/jbrant/MinimalCriterionCo…

…evolution

# Conflicts:
#	ExperimentEntities/App.Config
#	MCC_Domains/app.config
#	MCC_Executor/MCC_Executor.csproj
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/MCC Control.config.xml
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/MCC Path Evo 1.config.xml
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/MCC-PathEvolution-3.config.xml
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/MCC-PathEvolution-FM-2.config.xml
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/MCC-PathEvolution-FM-3.config.xml
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/MCC-PathEvolution-Test.config.xml
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/MCC-Template.config.xml
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/McsExperiments/Coevolution/Coevolution MCS - Speciation Control - Single MC 4.config.xml
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/OriginalConfigs/MCC Control.config.xml
#	MCC_Executor/MazeNavigation/ExperimentConfigurations/OriginalConfigs/MCC Speciation.config.xml
#	MCC_Executor/MazeNavigation/MazeNavigationExperimentExecutor.cs
#	MCC_Executor/app.config
#	MCC_Projects.sln
#	MazeNavigationEvaluator/App.config

Benchmarking Open-Endedness in Minimal Criterion Coevolution

This is the code accompanying the GECCO 2019 paper, Benchmarking Open-Endedness in Minimal Criterion Coevolution.

The project that runs the published MCC experiments is titled "MCC_Executor". This is the contained within the source code zip/tar files. For a description of each execution parameter, refer to MCC Parameters. The following exemplar executes MCC with a configuration identical to that in the paper. It
assumes that the experiment is being run from the solution directory (i.e. one level up from the MCC_Executor project). Seed agents are given to ensure that each run starts from a common baseline, though a random population can be generated by setting the GenerateAgentPopulation parameter to true. Note that the seed agents are themselves randomly generated, so the bootstrap process (which evolves competent agents that can satisfy the non-trivial MC of maze navigation) must execute regardless.

MCC_Executor.exe maze_navigation \
ExperimentSource=file \
NumRuns=20 \
ExperimentConfigDirectory=./MCC_Executor/MazeNavigation/ExperimentConfigurations/ \
OutputFileDirectory=../ExperimentResults/ \
SeedAgentFile=./MCC_Executor/MazeNavigation/SeedAgents/ \
SeedMazeFile=./MCC_Executor/MazeNavigation/SeedAgents/GeneratedMaze_10_Genomes_10_Height_10_Width_2_Waypoints_2_Walls.xml \
ExperimentNames="MCC-PathEvolution-2"

MCC Parameters

The below table enumerates all MCC experiment parameters and their function/purpose.

Parameter	Description
ExperimentSource	Source for the experiment configuration (file or database). The default selection is file; however, MCC can also be executed from a configuration persisted to an RDBMS (the schema is contained within the ExperimentEntities project).
NumRuns	Number of runs of MCC to execute.
ExperimentConfigDirectory	Directory containing experiment configuration files (these are the XML files under /MCC_Executor/MazeNavigation/ExperimentConfigurations). This must be specified if ExperimentSource is set to file (which is the default).
OutputFileDirectory	The directory in which to persist experiment CSV output files containing experiment results.
SeedAgentFile	XML file containing seed agent genomes. If this parameter is not specified, a random population will be generated.
SeedMazeFile	XML file containing seed maze genomes. Mazes are not generated by default, so this must be specified.
ExperimentNames	Comma-delimited list of experiments to execute (or just single experiment). The experiment name must match the name of the experiment configuration file.
StartFromRun	Run number at which to begin execution. This is useful when executing multiple runs of the same MCC experiment in parallel.
IsDistributedExecution	This should almost always be set to false. It enables behavior very specific to the HPC environment in which the initial MCC experiments were executed (such as additional logic for interfacing with Sun Grid Engine, interfacing with external processes for moving data off the cluster, etc.) and is likely of little use in any other environment.

MCC Version 1.1

This is a maintenance release of the core MCC algorithm published in GECCO 2017 proceedings. It's functionally identical to the original, but unused features/configurations have been removed and the project structure revamped to be more intuitive.

Like the original, by default, the post hoc analysis code (i.e. code in the "MazeNavigationEvaluator" project) assumes an underlying relational data store conforming to the schema bindings in the ExperimentEntities project. The DDL for such a data store is contained in the MCC_Experiment_DB_DDL.sql file. Though some of the commands therein are Transact-SQL specific, the tables and views are all ANSI-SQL compliant, so porting them to another RDBMS should be fairly straightforward. Alternatively, there is some limited capability for running said analysis from flat files; however, it's legacy code that would likely need to be updated to support analysis of the published experiments.

In addition to the source code, an executable along with the requisite assemblies are included in the "MCC_Binaries.zip" file. This can also be executed on linux using mono.

Execution instructions are on the MCC Wiki.

GECCO 2017 Paper Supporting Code

This release contains the tagged code supporting experiments and results that were presented in the GECCO 2017 publication, Minimal Criterion Coevolution: A New Approach to Open-Ended Search.

Please note that a persistent, relational data store is assumed by the post hoc analyses code contained in the "NavigatorMazeMapEvaluator" project. The DDL for such a data store is contained in the MCC_Experiment_DB_DDL.sql file. Though some of the commands therein are Transact-SQL specific, the tables and views are all ANSI-SQL compliant, so porting them to another RDBMS should be fairly straightforward.