zoofs
is a Python library for performing feature selection using an variety of nature inspired wrapper algorithms. The algorithms range from swarm-intelligence to physics based to Evolutionary.
It's easy to use ,flexible and powerful tool to reduce your feature size.
Use the package manager to install zoofs.
pip install zoofs
Algorithm Name | Class Name | Description | References doi |
---|---|---|---|
Particle Swarm Algorithm | ParticleSwarmOptimization | Utilizes swarm behaviour | 10.1007/978-3-319-13563-2_51 |
Grey Wolf Algorithm | GreyWolfOptimization | Utilizes wolf hunting behaviour | https://doi.org/10.1016/j.neucom.2015.06.083 |
Dragon Fly Algorithm | DragonFlyOptimization | Utilizes dragonfly swarm behaviour | 10.1016/j.knosys.2020.106131 |
Genetic Algorithm Algorithm | GeneticOptimization | Utilizes genetic mutation behaviour | 10.1109/ICDAR.2001.953980 |
Gravitational Algorithm | GravitationalOptimization | Utilizes newtons gravitational behaviour | 10.1109/ICASSP.2011.5946916 |
More algos soon, stay tuned !
Define your own objective function for optimization !
from sklearn.metrics import log_loss
# define your own objective function, make sure the function receives four parameters,
# fit your model and return the objective value !
def objective_function_topass(model,X_train, y_train, X_valid, y_valid):
model.fit(X_train,y_train)
P=log_loss(y_valid,model.predict_proba(X_valid))
return P
# import an algorithm !
from zoofs import ParticleSwarmOptimization
# create object of algorithm
algo_object=ParticleSwarmOptimization(objective_function_topass,n_iteration=20,
population_size=20,minimize=True)
import lightgbm as lgb
lgb_model = lgb.LGBMClassifier()
# fit the algorithm
algo_object.fit(lgb_model,X_train, y_train, X_valid, y_valid,verbose=True)
#plot your results
algo_object.plot_history()
- As available algorithms are wrapper algos. It is better to use ml models that build quicker, e.g lightgbm, catboost.
- Take sufficient amount for 'population_size' , as this will determine the extent of exploration and exploitation of the algo.
- Ensure that your ml model has its hyperparamters optimized before passing it to zoofs algos.
class zoofs.ParticleSwarmOptimization(objective_function,n_iteration=50,population_size=50,minimize=True,c1=2,c2=2,w=0.9)
Parameters | objective_function : user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.
n_iteration : int, default=50
population_size : int, default=50
minimize : bool, default=True
c1 : float, default=2.0
c2 : float, default=2.0
w : float, default=0.9
|
Attributes | best_feature_list : array-like
|
Methods | Class Name |
---|---|
fit | Run the algorithm |
plot_history | Plot results achieved across iteration |
Parameters | model :
X_train : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_train : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
X_valid : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_valid : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
verbose : bool,default=True
|
Returns | best_feature_list : array-like
|
Plot results across iterations
from sklearn.metrics import log_loss
# define your own objective function, make sure the function receives four parameters,
# fit your model and return the objective value !
def objective_function_topass(model,X_train, y_train, X_valid, y_valid):
model.fit(X_train,y_train)
P=log_loss(y_valid,model.predict_proba(X_valid))
return P
# import an algorithm !
from zoofs import ParticleSwarmOptimization
# create object of algorithm
algo_object=ParticleSwarmOptimization(objective_function_topass,n_iteration=20,
population_size=20,minimize=True,c1=2,c2=2,w=0.9)
import lightgbm as lgb
lgb_model = lgb.LGBMClassifier()
# fit the algorithm
algo_object.fit(lgb_model,X_train, y_train, X_valid, y_valid,verbose=True)
#plot your results
algo_object.plot_history()
class zoofs.GreyWolfOptimization(objective_function,n_iteration=50,population_size=50,minimize=True)
Parameters | objective_function : user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.
n_iteration : int, default=50
population_size : int, default=50
minimize : bool, default=True
|
Attributes | best_feature_list : array-like
|
Methods | Class Name |
---|---|
fit | Run the algorithm |
plot_history | Plot results achieved across iteration |
Parameters | model :
X_train : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_train : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
X_valid : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_valid : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
method : {1, 2}, default=1
verbose : bool,default=True
|
Returns | best_feature_list : array-like
|
Plot results across iterations
from sklearn.metrics import log_loss
# define your own objective function, make sure the function receives four parameters,
# fit your model and return the objective value !
def objective_function_topass(model,X_train, y_train, X_valid, y_valid):
model.fit(X_train,y_train)
P=log_loss(y_valid,model.predict_proba(X_valid))
return P
# import an algorithm !
from zoofs import GreyWolfOptimization
# create object of algorithm
algo_object=GreyWolfOptimization(objective_function_topass,n_iteration=20,
population_size=20,minimize=True)
import lightgbm as lgb
lgb_model = lgb.LGBMClassifier()
# fit the algorithm
algo_object.fit(lgb_model,X_train, y_train, X_valid, y_valid,method=1,verbose=True)
#plot your results
algo_object.plot_history()
class zoofs.DragonFlyOptimization(objective_function,n_iteration=50,population_size=50,minimize=True)
Parameters | objective_function : user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.
n_iteration : int, default=50
population_size : int, default=50
minimize : bool, default=True
|
Attributes | best_feature_list : array-like
|
Methods | Class Name |
---|---|
fit | Run the algorithm |
plot_history | Plot results achieved across iteration |
Parameters | model :
X_train : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_train : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
X_valid : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_valid : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
method : {'linear','random','quadraic','sinusoidal'}, default='sinusoidal'
verbose : bool,default=True
|
Returns | best_feature_list : array-like
|
Plot results across iterations
from sklearn.metrics import log_loss
# define your own objective function, make sure the function receives four parameters,
# fit your model and return the objective value !
def objective_function_topass(model,X_train, y_train, X_valid, y_valid):
model.fit(X_train,y_train)
P=log_loss(y_valid,model.predict_proba(X_valid))
return P
# import an algorithm !
from zoofs import DragonFlyOptimization
# create object of algorithm
algo_object=DragonFlyOptimization(objective_function_topass,n_iteration=20,
population_size=20,minimize=True)
import lightgbm as lgb
lgb_model = lgb.LGBMClassifier()
# fit the algorithm
algo_object.fit(lgb_model,X_train, y_train, X_valid, y_valid, method='sinusoidal', verbose=True)
#plot your results
algo_object.plot_history()
class zoofs.GeneticOptimization(objective_function,n_iteration=20,population_size=20,selective_pressure=2,elitism=2,mutation_rate=0.05,minimize=True)
Parameters | objective_function : user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.
n_iteration : int, default=50
population_size : int, default=50
selective_pressure : int, default=2
elitism : int, default=2
mutation_rate : float, default=0.05
minimize : bool, default=True
|
Attributes | best_feature_list : array-like
|
Methods | Class Name |
---|---|
fit | Run the algorithm |
plot_history | Plot results achieved across iteration |
Parameters | model :
X_train : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_train : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
X_valid : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_valid : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
verbose : bool,default=True
|
Returns | best_feature_list : array-like
|
Plot results across iterations
from sklearn.metrics import log_loss
# define your own objective function, make sure the function receives four parameters,
# fit your model and return the objective value !
def objective_function_topass(model,X_train, y_train, X_valid, y_valid):
model.fit(X_train,y_train)
P=log_loss(y_valid,model.predict_proba(X_valid))
return P
# import an algorithm !
from zoofs import GeneticOptimization
# create object of algorithm
algo_object=GeneticOptimization(objective_function_topass,n_iteration=20,
population_size=20,selective_pressure=2,elitism=2,
mutation_rate=0.05,minimize=True)
import lightgbm as lgb
lgb_model = lgb.LGBMClassifier()
# fit the algorithm
algo_object.fit(lgb_model,X_train, y_train,X_valid, y_valid, verbose=True)
#plot your results
algo_object.plot_history()
class zoofs.GravitationalOptimization(self,objective_function,n_iteration=50,population_size=50,g0=100,eps=0.5,minimize=True)
Parameters | objective_function : user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.
n_iteration : int, default=50
population_size : int, default=50
g0 : float, default=100
eps : float, default=0.5
minimize : bool, default=True
|
Attributes | best_feature_list : array-like
|
Methods | Class Name |
---|---|
fit | Run the algorithm |
plot_history | Plot results achieved across iteration |
Parameters | model :
X_train : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_train : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
X_valid : pandas.core.frame.DataFrame of shape (n_samples, n_features)
y_valid : pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)
verbose : bool,default=True
|
Returns | best_feature_list : array-like
|
Plot results across iterations
from sklearn.metrics import log_loss
# define your own objective function, make sure the function receives four parameters,
# fit your model and return the objective value !
def objective_function_topass(model,X_train, y_train, X_valid, y_valid):
model.fit(X_train,y_train)
P=log_loss(y_valid,model.predict_proba(X_valid))
return P
# import an algorithm !
from zoofs import GravitationalOptimization
# create object of algorithm
algo_object=GravitationalOptimization(objective_function,n_iteration=50,
population_size=50,g0=100,eps=0.5,minimize=True)
import lightgbm as lgb
lgb_model = lgb.LGBMClassifier()
# fit the algorithm
algo_object.fit(lgb_model,X_train, y_train, X_valid, y_valid, verbose=True)
#plot your results
algo_object.plot_history()
The development of zoofs
relies completely on contributions.
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Please make sure to update tests as appropriate.
18,08,2021