Loa algorithm #351

AljoM · 2021-09-24T15:07:47Z

I added 2 files ('loa.py', 'run_loa.py'). An implementation of lion optimization algorithm and an example of how to run it. Algorithm steps and formulas are found in an article found on (https://doi.org/10.1016/j.jcde.2015.06.003).

First I created a 'Lion' class that inherits from 'Individual'. Lions are distributed into groups called prides and a group called nomad lions.

Steps of algorithm and progress so far:

Hunting: Completed.
Moving towards safe place: completed.
Roaming: For pride lions is completed, for nomad lions is completed.
Mating: Creating new pride offspring is completed, mutation on the pride offspring is completed. Creating new nomad offspring is completed, mutation on the nomad offspring is completed.
Defense: Attacking amongst males in prides is completed, nomad lions attacking the prides is completed
Migration: Completed
Population equilibrium: Completed

Function that checks if lion's position has improved since last iteration ('data_correction') is also completed.

Main function ('run_iteration') is also completed.

Added lion optimization algorithm implementation, named 'loa.py'

Added example for lion optimization algorithm, named 'run_loa.py'

firefly-cpp · 2021-09-24T15:14:34Z

Hello @AljoM!

Thanks for all the hard work!

I assigned @zStupan to review your code.

First comments from my side:

Update Algorithms.md with new algorithm
Update niapy/algorithms/basic/init.py
Update niapy/util/factory.py

Added a new row (alphabetically sorted) with Lion Optimization Algorithm's name, acronym and reference.

Added Lion Optimization Algorithm to __init__.py file

Added lion optimization algorithm to factory.py

AljoM · 2021-09-24T15:43:00Z

Hello @AljoM!

Thanks for all the hard work!

I assigned @zStupan to review your code.

First comments from my side:

Update Algorithms.md with new algorithm

Update niapy/algorithms/basic/init.py

Update niapy/util/factory.py

Thank you for first comments. I updated the specified files.

Fixed some warnings from Scrutinizer test.

zStupan

The main problems are:

Use of the random library and np.random functions. Those functions should be called from the rng of the algorithm so the results can be reproduced if a seed is provided in the constructor.
Using a for loop instead of task.repair
With the random functions for example self.integers(0, 4, 1)[0] or self.random(1),The size argument 1 can be left out (self.integers(0, 4) will return a single integer between 0 and 4) with size=1 it returns an array of size 1.

Also, could you ad a unit test in niapy/tests? Just copy the BatAlgorithm test in test_ba.py and change the names and imports. That way we can see if it actually runs. It would also be good to compare the results to an existing implementation or to the results in the article, to see if it produces similar results.

zStupan · 2021-09-24T15:34:24Z

niapy/algorithms/basic/loa.py

+                        for i,x in enumerate(lion.current_x):
+                            if x > task.upper[i]:
+                                lion.current_x[i] = task.upper[i]
+                            elif x < task.lower[i]:
+                                lion.current_x[i] = task.lower[i]


This can be replaced with lion.current_x = task.repair(lion.current_x)

zStupan · 2021-09-24T15:36:35Z

niapy/algorithms/basic/loa.py

+                    for i, x in enumerate(lion.current_x):
+                        if x > task.upper[i]:
+                            lion.current_x[i] = task.upper[i]
+                        elif x < task.lower[i]:
+                            lion.current_x[i] = task.lower[i]


This can be replaced with lion.current_x = task.repair(lion.current_x)

zStupan · 2021-09-24T15:37:02Z

niapy/algorithms/basic/loa.py

+                    for i, x in enumerate(lion.current_x):
+                        if x > task.upper[i]:
+                            lion.current_x[i] = task.upper[i]
+                        elif x < task.lower[i]:
+                            lion.current_x[i] = task.lower[i]


This can be replaced with lion.current_x = task.repair(lion.current_x)

zStupan · 2021-09-24T15:37:13Z

niapy/algorithms/basic/loa.py

+                        for i, x in enumerate(prey_x):
+                            if x > task.upper[i] :
+                                prey_x[i] = task.upper[i]
+                            elif x < task.lower[i]:
+                                prey_x[i] = task.lower[i]


This can be replaced with lion.current_x = task.repair(lion.current_x)

zStupan · 2021-09-24T15:41:38Z

niapy/algorithms/basic/loa.py

+                    * pride_size (numpy.ndarray): Pride and nomad sizes.
+                    * gender_distribution (numpy.ndarray): Pride and nomad gender distributions.
+        """
+        nomad_size = (int)(round(self.nomad_ratio * self.population_size))


Replace with int(self.nomad_ratio * self.population_size) or round(self.nomad_ratio * self.population_size). Round returns int already so no conversion is required. The first option will always round down.

zStupan · 2021-09-24T15:59:16Z

niapy/algorithms/basic/loa.py

+                        r_two[rand_index] = 1
+                    else:
+                        #Gram-Schmidt process to find orthogonal vector r_two.
+                        random_vec = np.random.randn(len(r_one))


replace with self.standard_normal(len(r_one))

zStupan · 2021-09-24T16:08:34Z

niapy/algorithms/basic/loa.py

+                        num_of_mating_males = random.randint(1, num_of_males)
+                        mating_males_x = random.sample(males_x, num_of_mating_males)


Replace with:

num_of_mating_males = self.integers(1, num_of_males) mating_males_x = self.rng.choice(males_x, num_of_mating_males)

zStupan · 2021-09-24T16:09:39Z

niapy/algorithms/basic/loa.py

+                        num_of_mating_males = random.randint(1, num_of_males)
+                        mating_males_x = random.sample(males_x, num_of_mating_males)
+
+                        beta =  np.random.normal(0.5, 0.1)


Use self.normal

zStupan · 2021-09-24T16:14:48Z

niapy/algorithms/basic/loa.py

+            rand = self.uniform(0, 1, 1)
+            # If roaming treshold is met, move lion according to reference paper formula.
+            if not rand > roaming_probability:


if self.random() <= roaming_probability:

zStupan · 2021-09-24T16:21:38Z

niapy/algorithms/basic/loa.py

+                counter -= 1
+            else:
+                nomad_females_to_keep = np.append(nomad_females_to_keep, objects_to_array([lion_copy]))
+        np.random.shuffle(nomad_females_to_move)


Use self.rng.shuffle

- Removed "for loops" for checking if x is in limits, using task.repair/task.evaluate instead - Fixed and added some comments

- Removed use of random library and np.random function, instead these functions are called from rng of the algorithm. - Newly used self.rng.choice function was causing some warnings, fixed parts of codes so that the warning does not appear. - Removed size argument where it's not needed when calling random functions - Fixed tournament selection to calculate correct tournament size

Added test file 'test_loa.py' for Lion Optimization Algorithm.

zStupan

Looks good.

- Removed use of int() function where round() function was applied - Fixed some scrutinizer warnings

- Replaced one of the placeholder function args descriptions with correct one - Replaced use of np.max() function with max() function

AljoM · 2021-09-25T15:43:17Z

The main problems are:

Use of the random library and np.random functions. Those functions should be called from the rng of the algorithm so the results can be reproduced if a seed is provided in the constructor.

Using a for loop instead of task.repair

With the random functions for example self.integers(0, 4, 1)[0] or self.random(1),The size argument 1 can be left out (self.integers(0, 4) will return a single integer between 0 and 4) with size=1 it returns an array of size 1.

Also, could you ad a unit test in niapy/tests? Just copy the BatAlgorithm test in test_ba.py and change the names and imports. That way we can see if it actually runs. It would also be good to compare the results to an existing implementation or to the results in the article, to see if it produces similar results.

Thank you very much for that comment. I have since then fixed some of the problems:

I removed use of random and np.random function. Instead these functions are now called from rng of the algorithm. I have tested these changes, 2 instances of algorithm with same seed now produce same results.
I removed those for loops, instead task.repair and task.evaluate are used.
I removed size arguments where they're not needed, I swapped np.max with max function.

Extra:

There was also some scrutinizer error fixing; comment updating and some minor logical code fixes (there was a problem when tournament size of the tournament selection was bigger than the sample size)
I added test_loa.py file. I've ran setUp(), test_griewank() and test_algorithm_run() functions. They didn't throw any errors.

TO-DO:

I will do a bit of comparing with other algorithms. Is it okay if I post results as a comment here? Or should I upload them/share them somehow else?

zStupan · 2021-09-25T16:12:06Z

The main problems are:

Use of the random library and np.random functions. Those functions should be called from the rng of the algorithm so the results can be reproduced if a seed is provided in the constructor.

Using a for loop instead of task.repair

With the random functions for example self.integers(0, 4, 1)[0] or self.random(1),The size argument 1 can be left out (self.integers(0, 4) will return a single integer between 0 and 4) with size=1 it returns an array of size 1.

Also, could you ad a unit test in niapy/tests? Just copy the BatAlgorithm test in test_ba.py and change the names and imports. That way we can see if it actually runs. It would also be good to compare the results to an existing implementation or to the results in the article, to see if it produces similar results.

Thank you very much for that comment. I have since then fixed some of the problems:
1. I removed use of random and np.random function. Instead these functions are now called from rng of the algorithm. I have tested these changes, 2 instances of algorithm with same seed now produce same results.

2. I removed those for loops, instead task.repair and task.evaluate are used.

3. I removed size arguments where they're not needed, I swapped np.max with max function.
Extra:
* There was also some scrutinizer error fixing; comment updating and some minor logical code fixes (there was a problem when tournament size of the tournament selection was bigger than the sample size)

* I added test_loa.py file. I've ran setUp(), test_griewank() and test_algorithm_run() functions. They didn't throw any errors.
TO-DO:
* I will do a bit of comparing with other algorithms. Is it okay if I post results as a comment here? Or should I upload them/share them somehow else?

Great! You can post the results as a comment here.

Two more things. With self,rng.choice replace=False should be added as an argument in all cases, otherwise, an element can be selected multiple times. Also max([2, int(np.ceil(num_of_improvements / 2))]) can be simplified to max(2, int(np.ceil(num_of_improvements / 2))) (just the 2 arguments instead of an array).

Minor fixes on some function calls

AljoM · 2021-09-25T17:39:11Z

The main problems are:

Use of the random library and np.random functions. Those functions should be called from the rng of the algorithm so the results can be reproduced if a seed is provided in the constructor.

Using a for loop instead of task.repair

With the random functions for example self.integers(0, 4, 1)[0] or self.random(1),The size argument 1 can be left out (self.integers(0, 4) will return a single integer between 0 and 4) with size=1 it returns an array of size 1.

Also, could you ad a unit test in niapy/tests? Just copy the BatAlgorithm test in test_ba.py and change the names and imports. That way we can see if it actually runs. It would also be good to compare the results to an existing implementation or to the results in the article, to see if it produces similar results.

Thank you very much for that comment. I have since then fixed some of the problems:
1. I removed use of random and np.random function. Instead these functions are now called from rng of the algorithm. I have tested these changes, 2 instances of algorithm with same seed now produce same results.

2. I removed those for loops, instead task.repair and task.evaluate are used.

3. I removed size arguments where they're not needed, I swapped np.max with max function.
Extra:
* There was also some scrutinizer error fixing; comment updating and some minor logical code fixes (there was a problem when tournament size of the tournament selection was bigger than the sample size)

* I added test_loa.py file. I've ran setUp(), test_griewank() and test_algorithm_run() functions. They didn't throw any errors.
TO-DO:
* I will do a bit of comparing with other algorithms. Is it okay if I post results as a comment here? Or should I upload them/share them somehow else?
Great! You can post the results as a comment here.

Two more things. With self,rng.choice replace=False should be added as an argument in all cases, otherwise, an element can be selected multiple times. Also max([2, int(np.ceil(num_of_improvements / 2))]) can be simplified to max(2, int(np.ceil(num_of_improvements / 2))) (just the 2 arguments instead of an array).

I fixed those 2 issues.

I ran some comparissons. I was checking only for the average value. I used population size of 50, dimension of 60 and max evals of 30000. I ran each of the algorithms 10 times. I compared it to 3 algorithms: Camel algorithm (CA), ParticleSwarmOptimization (PSO) and FireFlyAlgorithm (FA). I ran them against 5 problems: Sphere, Ackley, Bent cigar, Griweank and Rastrigin.

Here are the results:

Problem - Sphere
LOA - 3.49
CA - 0.59
PSO - 1.64
FA - 395.49

Problem - Ackley
LOA - 7.22
CA - 13.05
PSO - 19.95
FA - 20.87

Problem - Bent cigar
LOA - 1.35 x 10^9
CA - 1.6 x 10^8
PSO - 1.35 x 10^11
FA - 1.44 x 10^11

Problem - Griewank
LOA - 1.44
CA - 1.05
PSO - 37.57
FA - 37.36

Problem - Rastrigin
LOA - 389.50
CA - 376.28
PSO - 227.57
FA - 924.18

Based on these quick results, I would say LOA is comparable to other algorithms

firefly-cpp · 2021-09-25T19:46:58Z

Thank you @zStupan for insightful comments/questions. Algorithm is now ready to be merged.

Thank you @AljoM for your revision.

@AljoM -- Have you already compared the results of your implementation against the results in original publication?

firefly-cpp · 2021-09-25T19:48:27Z

@AljoM -- Are there any referential implementations of this algorithm available online for comparison?

AljoM · 2021-09-25T21:25:45Z

@AljoM -- Are there any referential implementations of this algorithm available online for comparison?

Original publication got their results by using algorithm against functions that were part of CEC 2014 competition. I made a quick check and as far as I can see (correct me if I'm wrong please), NiaPy only contains the basic functions that were part of CEC 2014. So for example, I can find Bent Cigar function in the library but I can't find the Rotated Bent Cigar function, which is what was used in competition. Same goes with some of the other functions. I also couldn't find any of the hybrid or composition functions.
So due to original publication testing the algorithm against problems that are not part of NiaPy library and due to the original publication not providing source code of the algorithm, I cannot compare the results.

Now, about comparing it with other implementations: I was checking a little and besides the article that was my reference point, I couldn't even find any other official articles that would be testing out this algorithm, let alone providing an implementation in any of the programming languages.

It's not a very well researched algorithm, which is part of why I chose to implement it and compare it with others as part of my bachelor's thesis.

firefly-cpp · 2021-09-26T12:26:03Z

@AljoM -- you may also try to explore the following repository. CEC 2014 and other variations are linked to niapy library. If there is a chance, please run your algorithm using these benchmarks and compare with results in official papers.

AljoM · 2021-09-26T20:15:34Z

@AljoM -- you may also try to explore the following repository. CEC 2014 and other variations are linked to niapy library. If there is a chance, please run your algorithm using these benchmarks and compare with results in official papers.

I will check into it and try to get some testing done when I figure it out.

I have seen that there was merge being done but that there were some unsuccessful checks. Are those errors of significant meaning that would require me to fix the code? And is the merge going to be cancelled or proceeded?

zStupan · 2021-09-26T20:26:04Z

There is a linter error. Please fix it so the rest of the tests can get executed. Once all the checks have passed, the code can be merged.

Fixed linter error.

zStupan · 2021-09-27T17:25:14Z

Can one of the maintainers please approve this, so the checks can execute?

Fixed errors raised from docstyle check.

Fixed errors caused by scrutinizer check

AljoM · 2021-09-27T18:22:45Z

Fixed some error caused by docstyle check. Fixing them raised errors from scrutinizer check (lovely) so I fixed those as well.

Now I'd need another maintainer approval to execute checks.

firefly-cpp · 2021-09-27T20:32:13Z

Merging, thanks @AljoM & @zStupan.

AljoM added 2 commits September 24, 2021 16:28

Added loa.py

2d7f1d4

Added lion optimization algorithm implementation, named 'loa.py'

Added run_loa.py

8788848

Added example for lion optimization algorithm, named 'run_loa.py'

firefly-cpp requested a review from zStupan September 24, 2021 15:10

firefly-cpp added the feature label Sep 24, 2021

AljoM added 3 commits September 24, 2021 17:26

Added an algorithm

8a323c0

Added a new row (alphabetically sorted) with Lion Optimization Algorithm's name, acronym and reference.

Update __init__.py

f9d0bd8

Added Lion Optimization Algorithm to __init__.py file

Update factory.py

e3e1012

Added lion optimization algorithm to factory.py

Update loa.py

87c406b

Fixed some warnings from Scrutinizer test.

zStupan requested changes Sep 24, 2021

View reviewed changes

AljoM added 3 commits September 25, 2021 14:57

Update loa.py

63cf2ac

- Removed "for loops" for checking if x is in limits, using task.repair/task.evaluate instead - Fixed and added some comments

Add test_loa.py

393b495

Added test file 'test_loa.py' for Lion Optimization Algorithm.

zStupan approved these changes Sep 25, 2021

View reviewed changes

zStupan previously approved these changes Sep 25, 2021

View reviewed changes

Update loa.py

393679a

- Removed use of int() function where round() function was applied - Fixed some scrutinizer warnings

AljoM dismissed zStupan’s stale review via 393679a September 25, 2021 15:24

zStupan previously approved these changes Sep 25, 2021

View reviewed changes

Update loa.py

0849450

- Replaced one of the placeholder function args descriptions with correct one - Replaced use of np.max() function with max() function

AljoM dismissed zStupan’s stale review via 0849450 September 25, 2021 15:31

Update loa.py

6af463a

Minor fixes on some function calls

zStupan previously approved these changes Sep 25, 2021

View reviewed changes

Merge branch 'master' into loa_algorithm

3e23d5d

Fix error loa.py

905a54f

Fixed linter error.

AljoM dismissed zStupan’s stale review via 905a54f September 26, 2021 20:51

zStupan approved these changes Sep 26, 2021

View reviewed changes

zStupan previously approved these changes Sep 26, 2021

View reviewed changes

Fix error loa.py

fd36376

Fixed errors raised from docstyle check.

AljoM dismissed zStupan’s stale review via fd36376 September 27, 2021 18:07

Fix error loa.py

ef77b5e

Fixed errors caused by scrutinizer check

firefly-cpp merged commit 1ef5b4a into NiaOrg:master Sep 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Loa algorithm #351

Loa algorithm #351

AljoM commented Sep 24, 2021

firefly-cpp commented Sep 24, 2021

AljoM commented Sep 24, 2021

zStupan left a comment

zStupan Sep 24, 2021

zStupan Sep 24, 2021 •

edited

Loading

zStupan Sep 24, 2021

zStupan Sep 24, 2021

zStupan Sep 24, 2021

zStupan Sep 24, 2021

zStupan Sep 24, 2021

zStupan Sep 24, 2021

zStupan Sep 24, 2021

zStupan Sep 24, 2021

zStupan left a comment

AljoM commented Sep 25, 2021

zStupan commented Sep 25, 2021

AljoM commented Sep 25, 2021

firefly-cpp commented Sep 25, 2021

firefly-cpp commented Sep 25, 2021

AljoM commented Sep 25, 2021

firefly-cpp commented Sep 26, 2021

AljoM commented Sep 26, 2021

zStupan commented Sep 26, 2021

zStupan commented Sep 27, 2021

AljoM commented Sep 27, 2021

firefly-cpp commented Sep 27, 2021

		num_of_mating_males = random.randint(1, num_of_males)
		mating_males_x = random.sample(males_x, num_of_mating_males)

Loa algorithm #351

Loa algorithm #351

Conversation

AljoM commented Sep 24, 2021

firefly-cpp commented Sep 24, 2021

AljoM commented Sep 24, 2021

zStupan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zStupan Sep 24, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zStupan left a comment

Choose a reason for hiding this comment

AljoM commented Sep 25, 2021

zStupan commented Sep 25, 2021

AljoM commented Sep 25, 2021

firefly-cpp commented Sep 25, 2021

firefly-cpp commented Sep 25, 2021

AljoM commented Sep 25, 2021

firefly-cpp commented Sep 26, 2021

AljoM commented Sep 26, 2021

zStupan commented Sep 26, 2021

zStupan commented Sep 27, 2021

AljoM commented Sep 27, 2021

firefly-cpp commented Sep 27, 2021

zStupan Sep 24, 2021 •

edited

Loading