Neural evolution algorithms implementation (CNE, NEAT, HyperNEAT) #686
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -487,7 +487,6 @@ class NEAT { | |
|
|
||
| Species newSpecies = Species(); | ||
| newSpecies.AddGenome(genome); | ||
| newSpecies.StaleAge(0); | ||
| population.AddSpecies(newSpecies); | ||
| } | ||
|
|
@@ -501,19 +500,6 @@ class NEAT { | |
| } | ||
| } | ||
|
|
||
| // Aggregate population's genomes. | ||
| void AggregateGenomes(Population& population, std::vector<Genome>& genomes) { | ||
| genomes.clear(); | ||
|
|
@@ -529,14 +515,6 @@ class NEAT { | |
| std::sort(genomes.begin(), genomes.end(), Species::CompareGenome); | ||
| } | ||
|
|
||
| // Get genome index in a genomes vector. | ||
| ssize_t GetGenomeIndex(std::vector<Genome>& genomes, ssize_t id) { | ||
| for (ssize_t i=0; i<genomes.size(); ++i) { | ||
|
|
@@ -546,12 +524,11 @@ class NEAT { | |
| return -1; | ||
| } | ||
|
|
||
| // Calculate species' average rank in population by fitness. Bigger is better. | ||
| void CalcSpeciesAverageRank(Population& population, std::vector<double>& speciesAverageRank) { | ||
| std::vector<Genome> genomes; | ||
| AggregateGenomes(population, genomes); | ||
| SortGenomes(genomes); | ||
| speciesAverageRank.clear(); | ||
|
|
||
| for (ssize_t i=0; i<population.aSpecies.size(); ++i) { | ||
|
|
@@ -698,22 +675,19 @@ class NEAT { | |
| // keep best genome. | ||
| Genome lastBestGenome = aPopulation.BestGenome(); | ||
|
|
||
| // Remove weak genomes in each species. | ||
| CullSpecies(aPopulation, aCullSpeciesPercentage); | ||
|
There was a problem hiding this comment. I'm not sure, is this a technique to avoid local minima? Truncating each species by 50% sounds like an counterproductive idea. Maybe I missed a reference in the NEAT paper? There was a problem hiding this comment. I referred to this implementation which seems performs good on Mario game: There was a problem hiding this comment. Thanks for the information, maybe it's a good idea, but it sounds counterproductive to me. I think it should work without, since RemoveStaleSpecies should remove old species anyway. There was a problem hiding this comment. My current plan is keep the functions, but when we do the Reproduce(), we need further decision about whether to use it. Organizing these steps in Reproduce() are flexible. There was a problem hiding this comment. Sounds good, since there is a parameter (aCullSpeciesPercentage) it's super easy to disable the function. |
||
|
|
||
| // Remove stale species, weak species. | ||
| if (aPopulation.aSpecies.size() > 10) { | ||
|
There was a problem hiding this comment. As for the number of stale species, we should also provide a parameter for the number of weak species. |
||
| RemoveStaleSpecies(aPopulation); | ||
| RemoveWeakSpecies(aPopulation); | ||
| } | ||
|
|
||
| // Breed children in each species. | ||
| std::vector<double> speciesAverageRank; | ||
| CalcSpeciesAverageRank(aPopulation, speciesAverageRank); | ||
|
There was a problem hiding this comment. Regarding https://groups.yahoo.com/neo/groups/neat/conversations/topics/2203 we don't use the adjusted fitness. There was a problem hiding this comment. So you mean, as we are calculation the average fitness of species, we doesn't need to use the adjustedFitness, because it likes we average it for twice, right? There was a problem hiding this comment. Yes, I think that's what Stanley is trying to explain. There was a problem hiding this comment. I now removed all adjustedFitness, even in the genome class. |
||
|
|
||
| //DEBUGGING!!!!!!!!! | ||
| printf("Species average rank are: "); | ||
| for (ssize_t s=0; s<speciesAverageRank.size(); ++s) { | ||
|
|
@@ -729,7 +703,7 @@ class NEAT { | |
| ssize_t numBreed = std::floor(speciesAverageRank[i] * aPopulationSize / totalAverageRank) - 1; | ||
|
There was a problem hiding this comment. That's the right position, I would go with offspring here. |
||
| ssize_t numBreedSuccess = 0; | ||
| while (numBreedSuccess < numBreed) { | ||
| Genome genome; | ||
| bool hasBaby = BreedChild(aPopulation.aSpecies[i], genome, aCrossoverRate); | ||
| if (hasBaby) { | ||
| childGenomes.push_back(genome); | ||
|
|
@@ -739,14 +713,17 @@ class NEAT { | |
| } | ||
|
|
||
| // Keep the best in each species. | ||
|
|
||
| //DEBUGGING!!!!!!!!! | ||
| printf("before, species sizes are: "); | ||
| for (ssize_t s=0; s<aPopulation.aSpecies.size(); ++s) { | ||
| std::cout<< aPopulation.aSpecies[s].aGenomes.size() << " "; | ||
| } | ||
| printf("\n"); | ||
| //DEBUGGING!!!!!!!!! | ||
|
|
||
| CullSpeciesToOne(aPopulation); // NOTICE: I found that this makes XOR test converges fast. | ||
|
|
||
| //DEBUGGING!!!!!!!!! | ||
| printf("after, species sizes are: "); | ||
| for (ssize_t s=0; s<aPopulation.aSpecies.size(); ++s) { | ||
|
|
@@ -759,7 +736,7 @@ class NEAT { | |
| childGenomes.push_back(lastBestGenome); | ||
|
There was a problem hiding this comment. We can't just add the lastBestGenome, because it's possible that at the time we add the genome it doesn't exist anymore. Also the population already contains the genome, so we would add the genome twice. There was a problem hiding this comment. childGenomes.push_back(lastBestGenome); There was a problem hiding this comment. ah, right I thought the lastBestGenome is a reference but it's a copy, so it does exist. |
||
| while (childGenomes.size() + aPopulation.aSpecies.size() < aPopulationSize) { | ||
| ssize_t speciesIndex = mlpack::math::RandInt(0, aPopulation.aSpecies.size()); | ||
| Genome genome; | ||
| bool hasBaby = BreedChild(aPopulation.aSpecies[speciesIndex], genome, aCrossoverRate); | ||
| if (hasBaby) childGenomes.push_back(genome); | ||
| } | ||
|
|
@@ -779,7 +756,8 @@ class NEAT { | |
| aPopulation.ReassignGenomeId(); | ||
| } | ||
|
|
||
| // Evaluate genomes in population. | ||
| // Set genomes' fitness, species' and population's best fitness and genome. | ||
| void Evaluate() { | ||
| for (ssize_t i=0; i<aPopulation.aSpecies.size(); ++i) { | ||
| for (ssize_t j=0; j<aPopulation.aSpecies[i].aGenomes.size(); ++j) { | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -97,7 +97,6 @@ BOOST_AUTO_TEST_CASE(NEGenomeTest) | |
| ssize_t numInput = 3; | ||
| ssize_t numOutput = 1; | ||
| double fitness = -1; | ||
| std::vector<NeuronGene> neuronGenes; | ||
| std::vector<LinkGene> linkGenes; | ||
|
|
||
|
|
@@ -134,8 +133,7 @@ BOOST_AUTO_TEST_CASE(NEGenomeTest) | |
| linkGenes, | ||
| numInput, | ||
| numOutput, | ||
| fitness); | ||
|
|
||
| // Test seed genome. | ||
| std::vector<std::vector<double>> inputs; // TODO: use arma::mat for input. | ||
|
|
@@ -190,7 +188,6 @@ BOOST_AUTO_TEST_CASE(NECneXorTest) | |
| ssize_t numInput = 3; | ||
| ssize_t numOutput = 1; | ||
| double fitness = -1; | ||
| std::vector<NeuronGene> neuronGenes; | ||
| std::vector<LinkGene> linkGenes; | ||
|
|
||
|
|
@@ -227,8 +224,7 @@ BOOST_AUTO_TEST_CASE(NECneXorTest) | |
| linkGenes, | ||
| numInput, | ||
| numOutput, | ||
| fitness); | ||
|
|
||
|
There was a problem hiding this comment. I think it would be a good idea to set a default initial genome inside the CNE method. The authors start with a genome where each input unit is connected with the output units. But, we could also use a hidden units as you did. This way, we can directly start by evolving the network. There was a problem hiding this comment. @zoq Yeah, I will put it in the default constructor. It is just for the testing, I use this style to demonstrate how to use and test the parametric constructor. There was a problem hiding this comment. Ah, okay, good idea, just checked the CNE method, and couldn't find a default genome. There was a problem hiding this comment. @zoq Yeah, I will set seedGenome in default constructor. Currently I haven't add the default constructor. I am planning to add it after the tests passed. |
||
| // Specify task type. | ||
| TaskXor<ann::MeanSquaredErrorFunction> task; | ||
|
|
@@ -273,7 +269,6 @@ BOOST_AUTO_TEST_CASE(NENeatXorTest) | |
| ssize_t numInput = 3; | ||
| ssize_t numOutput = 1; | ||
| double fitness = -1; | ||
| std::vector<NeuronGene> neuronGenes; | ||
| std::vector<LinkGene> linkGenes; | ||
|
|
||
|
|
@@ -310,8 +305,7 @@ BOOST_AUTO_TEST_CASE(NENeatXorTest) | |
| linkGenes, | ||
| numInput, | ||
| numOutput, | ||
| fitness); | ||
|
|
||
| // Specify task type. | ||
| TaskXor<ann::MeanSquaredErrorFunction> task; | ||
|
|
@@ -350,28 +344,27 @@ BOOST_AUTO_TEST_CASE(NENeatCartPoleTest) | |
| Parameters params; | ||
| params.aPopulationSize = 500; | ||
| params.aMaxGeneration = 500; | ||
| params.aCoeffDisjoint = 1.0; | ||
| params.aCoeffWeightDiff = 0.5; | ||
| params.aCompatThreshold = 1.0; | ||
| params.aStaleAgeThreshold = 15; | ||
| params.aCrossoverRate = 0.75; | ||
| params.aCullSpeciesPercentage = 0.5; | ||
| params.aMutateWeightProb = 0.2; | ||
| params.aPerturbWeightProb = 0.5; | ||
| params.aMutateWeightSize = 0.1; | ||
| params.aMutateAddLinkProb = 0.3; | ||
| params.aMutateAddRecurrentLinkProb = 0; | ||
| params.aMutateAddLoopLinkProb = 0; | ||
| params.aMutateAddNeuronProb = 0.1; | ||
| params.aMutateEnabledProb = 0.1; | ||
| params.aMutateDisabledProb = 0.1; | ||
|
|
||
| // Set seed genome for cart pole task. | ||
| ssize_t id = 0; | ||
| ssize_t numInput = 5; | ||
| ssize_t numOutput = 1; | ||
| double fitness = -1; | ||
| std::vector<NeuronGene> neuronGenes; | ||
| std::vector<LinkGene> linkGenes; | ||
|
|
||
|
|
@@ -420,8 +413,7 @@ BOOST_AUTO_TEST_CASE(NENeatCartPoleTest) | |
| linkGenes, | ||
| numInput, | ||
| numOutput, | ||
| fitness); | ||
|
|
||
| // Construct NEAT instance. | ||
| NEAT<TaskCartPole> neat(task, seedGenome, params); | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't we have to use the fitness of the genomes to calculate the rank?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, but I am not clear what this means.
The average rank is the average of each genome's rank in this species. And
the rank of each genome is using the fitness of genome.
2016-07-16 16:10 GMT-06:00 Marcus Edel notifications@github.com:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought it would be easier to rank the genomes, directly by the fitness value instead of checking at which position the genome is after sorting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure, but after we sort with fitness, isn't the position is
exactly the rank of genomes?
2016-07-17 15:25 GMT-06:00 Marcus Edel notifications@github.com: