MetaGen is a Python scientific package designed to provide users with a standard system for solution representation, methods to generate and modify these solutions, and a standard interface between the metaheuristic and potential users. These features make it easier for programmers to develop a metaheuristic and make it accessible to any user who wants to optimize a problem during the crucial hyperparameter tuning process in machine learning model production.
As is shown in the previous figure, MetaGen has two main use cases that depend on two user profiles:
- [DUC] Development Use Case: MetaGen is used by a developer to program a new metaheuristic algorithm, simplifying the development process and making it accessible to any user who wants to optimize a problem.
- [SUC] Solving Use Case: MetaGen is used by a solver to define an optimization problem and use a metaheuristic to find the best hyperparameter configuration for a specific machine or deep learning model.
The core of MetaGen is located in the metagen.framework package; it contains the logic to support both use cases. This package includes two classes, Domain and Solution. As shown in Figure ref{img.duc, metagen.framework provides a developer with a standard system for potential solution representation through the Solution class and methods to generate and modify these solutions. Additionally, the Domain class provides a standard interface between the metaheuristic and its potential users (solvers), ensuring that the values of generated or modified solutions fall within the specified range.
The Solving Use Case figure shows how the metagen.framework package helps a solver to define its own hyperparameter application domains by setting the range of variable values through the Domain class and to define a fitness function using the standard interface with potential solutions treated by the metaheuristic, provided by the Solution class. The metagen.metaheuristics package includes four developed metaheuristics: Random Search cite{Bergstra2012, Genetic Algorithm Alhijawi2023, and Coronavirus Optimization Algorithm cite{martinez-cvoa. A solver defines a problem using the Domain class and solves it with one of these available metaheuristics (or another implemented by a third-party developer).
The formal context of MetaGen is outlined in Equation 1. The solver defines a problem P and uses the metaheuristic M, created by the developer, to find an optimal solution S_{opt} (Equation (1a)). A problem P is composed of a domain D and a fitness function F (Equation (1b)). A domain is a set of variables and their corresponding value range, which the metaheuristic optimizes through the fitness function. A solution is an assignment of valid values to all variables within the domain. All potential solutions to a problem are part of the search space, and the metaheuristic explores, modifies, and evaluates the search space using the fitness function.
Equation 1:
M(P) = S_{opt} \qquad (1a)\\
P = \langle D,F \rangle \qquad (1b)
The formal definition of a domain D is given by Equation (2a). It is a set of N variable definitions, V_i \models Def^{T}, where V_i represents the name of the variable, and Def^{T} is its definition of type T as specified in Eq. (2b). There are six different types of variables, namely INTEGER (I), REAL (R), CATEGORICAL (C), GROUP (G), DYNAMIC (D) and, STATIC (S). The alias BASIC (B) is defined as the combination of INTEGER, REAL, and CATEGORICAL in Eq. (2c).
Equation 2:
D = \{Var_1 \models Def^{T}_1, Var_2 \models Def^{T}_2,...,Var_i \models Def^{T}_i,...,Var^{T}_{N} \models Def_{N}\} \qquad (2a)\\
T \epsilon \{INTEGER(I), REAL(R), CATEGORICAL(C),\\
, GROUP(G), DYNAMIC(D), STATIC(S)\} \qquad (2b)\\
BASIC (B) \epsilon \{INTEGER(I), REAL(R), CATEGORICAL(C)\} \qquad (2c)
The MetaGen supported definitions are defined in Equation 3. The definitions of INTEGER and REAL represent a value in the range of integers \mathbb{Z} and real numbers \mathbb{R}, respectively, in the range [Min, Max] as shown in Eq. (3a). The CATEGORICAL definition represents a set of P unique labels, as described in Eq. (3b). The GROUP definition represents a set of Q elements, each defined by a basic definition Def^B, as given in Eq. (3c). The DYNAMIC definition represents a sequence of BASIC or GROUP values of length in the range [LN_{min}, LN_{max}] as specified in Eq. (3d). Finally, the STATIC definition is a sequence of BASIC or GROUP values with a fixed length, LN as specified in Eq. (3e).
Equation 3:
Def^{I|R} = \langle Min, Max \rangle \qquad (3a)\\
Def^{C} = \{L_1, L_2,...,L_i,...,L_{P}\} \qquad (3b)\\
Def^{G} = \{E_1 = Def^{B}_1, E_2 = Def^{B}_2,...,E_j = Def^{B}_j,...,E_{Q} = Def^{B}_{Q}\} \qquad (3c)\\
Def^{D} = \langle LN_{min}, LN_{max}, Def^{B|G} \rangle \qquad (3d)\\
Def^{S} = \langle LN, Def^{B|G} \rangle \qquad (3e)
A collection of example problems can be found in the next table to support the formal definition. The problem P_1 is composed of a domain with an INTEGER variable x that moves within the interval [-10, 10], and the function to be optimized is f(x)=x+5. Similarly, the problem P_2 has a domain consisting of a REAL variable x that moves within the interval [0.0, 1.0], and the objective function is f(x)=x^5.
Sample problems
Sample Problems| P_{ID} | Domain | Function |
|---|---|---|
| P_1 | x \models Def^{I} = \langle -10, 10\rangle | x+5 |
| P_2 | x \models Def^{R} = \langle 0.0, 1.0\rangle | x^2 |
| P_3 | Alpha \models Def^{R} = \langle 0.0001, 0.001\rangle Iterations \models Def^{I} = \langle 5, 200\rangle Loss \models Def^{C} = \{squared\:error, huber, epsilon\:insensitive\} | Regression(Alpha, Iterations, Loss) |
| P_4 | Learning\;rate \models Def^{R} = \langle 0.0, 0.000001\rangle Ema \models Def^{C} = \{True, False\} Arch \models Def^{D} = \langle 2,10,\,Def^{G} = \{Neurons \models Def^{I} = \langle 25, 300\rangle, Activation \models Def^{C} = \{relu, sigmoid, softmax, tanh\}, Dropout \models Def^{R} = \langle 0.0, 0.45\rangle\}\rangle | LSTM(Learning\;rate, Ema, Arch) |
Problems P_3 and P_4 are examples of the target of the MetaGen framework, which is hyperparameter optimization for machine learning models. Problem P_3 illustrates the optimization of a linear regression model. The objective function is the performance of a model that has been trained with the Alpha, Iterations, and Loss hyperparameters. Alpha represents the regularization term that the model uses to prevent overfitting and is typically set to a value close to zero. The Iterations hyperparameter controls the number of times the linear model should be re-calculated before reaching an error tolerance. Finally, the Loss hyperparameter sets the function used during training to measure the performance of the linear model in each iteration. The domain of problem P_3 consists of the Alpha variable, which is a REAL value defined in the interval [0.0001, 0.001], the Iterations variable, which is an INTEGER defined in the interval [5, 200], and the Loss variable, which is a CATEGORY and can be set to squared\:error, huber or epsilon\:insensitive.
The problem P_4 represents a hyperparameter optimization problem for a Long Short-Term Memory (LSTM) deep learning architecture. The objective is to determine the best configuration of the network in terms of the number of layers and the optimal hyperparameters for each layer. There are two common properties that apply to all layers of the architecture: the optimizer, which is stochastic gradient descent, and the control of the optimizer, which is through the learning rate and exponential moving average (EMA) parameters. The stochastic gradient descent is an iterative method used to optimize the network's performance, and the learning rate parameter mod
using pip:
(.venv) $ pip install pymetagen-datalabupo.. toctree::
use_cases
api