Implement Constrained Objective Functions

[ljvmiranda921]

Context

For the v.0.2.0 release (#5), we're planning to add two new major features, and one of them is constrained optimization via PSO (#8). As with PySwarms, we don't only provide primitives for optimizers, but also built-in objective functions to test various methods (such as in the single-objective case). In this Issue, we will be implementing constrained objective functions.

What you'll do

• Write a module pyswarms.utils.functions.contrained_obj containing two constrained objective functions (let's do two for now).
• Create unit tests for the said functions in the module tests.utils.functions.test_constrainedobj.

What makes this different from the single_obj module?

Single objective functions takes an input vector, and computes the fitness of that vector given an equation. So if your objective function is the sphere function, you simply supply it by an input vector, say [0,0], and it gives you its corresponding fitness i.e., 0.

Constrained objective functions are different in a sense that you don't only have a method that computes a vector's fitness, but also a constraint where these vectors are limited. These are useful in applications where you want to optimize with respect to a certain limit, say, in mechanical engineering where the yield stress should not go overboard a value x. Of course, you can have multiple constraints in a given problem, and we will consider that here.

What should the data structure look like?

Unlike single-objective functions where we only have one method per function, constrained optimizations will take on a more object-oriented approach. Before, we have something like this:

def objective_function(X):
    # Compute a fitness given X, say, just (X-5)**2
    J = (X-5)**2
    return J

Now, we will have something like:

class MyConstrainedObjective(object):
    """My constrained objective function

    Most constrained optimization problems are defined in the following manner:
    min J(X) = something something, such that
        equality constraint: h(x), g(x)
        inequality constraint: i(x), j(x)
    """

    @classmethod
    def objective_func(cls, X):
        """Contains the actual fitness function that computes for an input X"""
        # Compute for J
        return J

    @classmethod
    def ineq_cons(cls):
        """Returns all inequality constraints of the given problem"""
        # Assuming we have two inequality constraints
        return (cls.ineq_1, cls.ineq_2)

    @classmethod
    def eq_cons(cls):
       """Returns all equality constraints of the given problem"""
      # Assuming we only have one equality constraint
      return cls.eq_1

    @staticmethod
    def ineq_1(X):
       """Inequality constraint 1 that checks X"""
       return X <= 5 # for example

    # the same goes for ineq_2 and eq_1

If you think there's a better data model to such, feel free to suggest a better way of handling these constraints. This means that when we're performing a constrained optimization problem, we only need to call the objective function, and the inequality and equality constraints if possible:

cons_pso = ConstrainedPSO()
cons_pso.optimize(objective_func=MyConstrainedObjective.objective_func, 
                  ineq_cons=MyConstrainedObjective.ineq_cons
                  eq_cons=MyConstrainedObjective.eq_cons)

A good resource for good constrained optimization problems can be in this link. It's Wikipedia I know, but this one is pretty accurate. For starters, you can try the first two methods.

Writing tests

You can check on how we write tests in the tests module of this package. Currently, we're using the unittest package to do these tests. Some of the things we'd like to check are the following:

• Does the objective function approximately return the minima when fed with argmin (the argument that gives the optimal value)?
• Do the methods raise an Error when fed with values outside of bounds, or of wrong type etc?
• Other things you can think of that increases test coverage. Meaning, all methods you write should be tested.

Other things you need to know

• PySwarms is pretty much a documentation junkie so please put docstrings in all classes, methods, and modules you'll create. This is important so that others can easily pick-up what you're working on. You can refer to the numpy documentation for a primer on how we do things.
• We also follow PEP8 rules for styling Python code. If you wish to check your code against PEP8, please run a flake8 test on your file.
• This whole Issue may be quite long and overwhelming at first, but don't worry! I also wanted to help you make your first contribution! And hope we both learn in the process. 👍
• Currently, I'm building a prototype of a constrained optimizer. Ideally, I hope that we can test how the optimizer I'm building will interface in the objective functions you are building 😄

Getting Started

• To help you get started, please check the Contributing guidelines in the documentation.
• When creating a git commit, please make sure that it follows this prescribed format:

Capitalized, short (50 chars or less) summary

More detailed explanatory text, if necessary.  Wrap it to about 72
characters or so.  In some contexts, the first line is treated as the
subject of an email and the rest of the text as the body.  The blank
line separating the summary from the body is critical (unless you omit
the body entirely); tools like rebase can get confused if you run the
two together.

Write your commit message in the imperative: "Fix bug" and not "Fixed bug"
or "Fixes bug."  This convention matches up with commit messages generated
by commands like git merge and git revert.

Further paragraphs come after blank lines.

- Bullet points are okay, too

- Typically a hyphen or asterisk is used for the bullet, followed by a
  single space, with blank lines in between, but conventions vary here

- Use a hanging indent

Author: your-github-username
E-mail: your-email

[ljvmiranda921]

Hi @taylorspencer , you might be interested in this 👍

[bishwascg]

Hi Miranda!
I would like to work on this

[ljvmiranda921]

Since @taylorspencer hasn't replied here for more than 3 days, I'll give this one to @bishwascg .

Hi @bishwascg ! Awesome! Okay, just ping me if you ran into any problems/questions!

[bishwascg]

@ljvmiranda921 what should the behavior be when the constraints are not satisfied?

[ljvmiranda921]

Hi @bishwascg , do you mean no solution was found? If that's the case, just print that no solutions were found via logger.info 👍

[bishwascg]

@ljvmiranda921 Just to clarify, as an example, if I have the Rosenbrock function constrained to a disk then the constraint is $$X^2+Y^2 < 2$$. So, if the constraint is not satisfied, i should print that no solutions were found. Am I right?
Or should I raise a ValueError similar to the one when I have the input variables outside the search domain?

[ljvmiranda921]

Oops, I apologize for my confusion. I've been thinking of the constrained optimizer 😕

Yes, you're correct, please raise a ValueError instead 👍

[bishwascg]

Hey @ljvmiranda921 ... my end sem exams are starting from Monday so I'll resume work after a week

[ljvmiranda921]

Sure! Take your time and goodluck with your exams!

On Fri, 17 Nov 2017 at 10:13 PM Bishwas C Gupta ***@***.***> wrote: Hey @ljvmiranda921 <https://github.com/ljvmiranda921> ... my end sem exams are starting from Monday so I'll resume work after a week — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#60 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMWYs-P_fKHwPAiIGGtu9mF8ADnePc_zks5s3YZfgaJpZM4QPKsj> .

-- Lester James V. Miranda BS Electronics and Communications Engineering, Minor in Philosophy (Class of 2016) Ateneo de Manila University lester.miranda@ <lester.miranda@obf.ateneo.edu>*o <http://toki.waseda.jp>bf.ateneo.edu <http://bf.ateneo.edu>*

[ljvmiranda921]

Implement constrained optimization in PySwarms

[bishwascg]

Hi @ljvmiranda921, the link above is not opening

Also, the objective functions have been implemented. I'm just reading up on the unittest package

[bishwascg]

Also, I'm a little confused about the ineq_const class method. How do we return "all" the inequality constraints when we do not know what the objective function is?

[ljvmiranda921]

Hi @bishwascg ,
Yup, sorry, it's my kinda personal trello board that's why it's on private 👍 I'm currently working on constrained PSO and check how well it will interface with the objective functions you're doing.

Hmmm, the constraints are often defined by the problem already, we just need to write them as a method, and have them returned by ineq_const. Usually we define all these constraints as a staticmethod, say ineq_1, ineq_2, etc. then have these methods returned by ineq_const.

Maybe there is a more efficient design? If you thought of something better please feel free to tell me 😄 .

[parkerduckworth]

Is this issue still ongoing/available?

[ljvmiranda921]

Hi, this is still ongoing but may have been stale. Any updates on this @bishwascg ?

[ljvmiranda921]

This issue has been stale. I'll close this for a while and work on this on my own. Thanks for all your help and input!

[tcrotzer64]

Any progress on this? I'm interested in how its been going.

[cslearning20xx]

hi, just checking if there is any update on this