feature Request - cover binary classification?

[sskarkhanis]

Hello Trevor,

great job with gplearn; really enjoy using it data mining problems.

Would it be possible to extend gplearn to cover binary classification at some point?

pretty please ;)

Thank you!

[jonas-eschle]

Hi,
I was wondering how this feature is going on? Will it be implemented soon? I would be interested to see feature too
Best,
Mayou

[trevorstephens]

Hi @mayou36 ... I have this pegged for the next release which I hope to have out soon.

[trevorstephens]

Might push classification back a release at this point, would like to get the bugfixes and new features out there sooner. Will see how we go though!

[rspadim]

how logistic regression could be created with gp? just a "fixed" logistic/threshold output function "like" neural networks?

[trevorstephens]

This class will be in the next release as a first class citizen of gplearn. In the meantime you could create your own custom fitness measure #20 that takes whatever output of the regression to mean some classification (say y < 0 for 0, and y >= 0 for 1) which would change the regressor to be a classifier ... Though you would need to wrap the output to make sense

[spacegoing]

@trevorstephens I just found your great repo and would like to try implementing this feature. Would please provide some information like references or where should I start? Many thanks!

[trevorstephens]

Hi @spacegoing ... A PR would be most welcome! You could start by referring to A Field Guide To Genetic Programming which has a free download available. The Koza book "Genetic Programming" (no free downloads that I'm aware of) has a few examples of classification and one or two use a positive value of the program to mean a positive example and a negative value to be a negative. I don't think there is much if any literature on deriving a probability for the classification, but you could expose the raw decision function for users to toy with.

So my thoughts on this are to make a new classifier class, and then either add some logic to the BaseSymbolic class for when it's doing classification that maps the output of the program to a binary response (around the same place where it does special things for regressors or transformers) or put some logic in the new classifier class itself... and then add some new fitness functions that measure appropriate metrics, etc. Hope that gets you started.

[spacegoing]

@trevorstephens Tks for your info! I've read the free book but didn't get much help there. I'll try Koza's book now:D

[spacegoing]

@trevorstephens There are four versions of "Genetic Programming" . Would you pls tell me explicitly which one you refer to?

[trevorstephens]

"Genetic Programming: On the Programming of Computers by Means of Natural Selection", Published 1992, ISBN-10: 0262111705, ISBN-13: 978-0262111706

[trevorstephens]

It's a massive tome btw 🤣 ... Oh and implementing the estimator is the easy part. You would need to also add good documentation and an example on how to use it. I can help along the way, but just a heads up that it's not simply code for a new estimator. 👍