Hotfix 0.2.1

🐛 Bug Fixes

• (cargo) "mpi" feature was always activated due to the algorithm module requiring tantale with mpi.

Release 0.2.0 : Advanced Outcome, Samplers and Tree Parzen structured Estimator

🚀 Features

• (Outcome)[breaking] Because Codomain and Outcome were almost always bounded together within all type bounds. Now the Outcome has a Cod associated type corresponding to a codomain.
  Optimizer do not have associated Codomain anymore.
  This allows Optimizers and Samplers to be generic over Codomains. For example RandomSearch does not need any information from the codomain. It is now generic over all Codomain.
  Sha and Asha requires elements from codomain to be Ord, so they are now generic over Single-objective codomains that have Ord elements. Same for MoAsha that requires codomains to be Multi and codomain elements to be Dominate. Etc.
  The Outcome macro was modified with helper attributes like #[maximize], #[minimize], #[constraint], #[cost] and #[step]. The derive macro automaticaly determines which Codomain corresponds to tagged fields within the user defined outcome.
  Defining a codomain before, and passing a codomain to, an experiment is no longer required.
  For Python Outcome the init_python! macro was extended with extra syntax to include objectives, constraint, cost and step.
  All examples, documentation, and tests were modified consequently.
• (codomain!) Removed all codomain! macros as the codomain is now automatically defined by the Outcome derive macro, and because Optimizer and Sampler can be generic over codomains
• (sampler) Add the Sampler trait allowing to define a new kind of algorithms where the update of internal state is decorrelated from the sampling phase.
  An algorithm can be both an independent Optimizer and a Sampler.
  • RandomSearch is now a Sampler and a SingleSampler
  • BatchRandomSearch is now a Sampler and a BatchSampler
  • GridSearch is now a Sampler and a SingleSampler
  • TPE is a Sampler and a SingleSampler
• (TPE) Add the "bayes" feature, and Tree Parzen structured Estimator algorithm (An Optimizer and a Sampler).
• (Asha) Asha is now generic over Samplers
• (Sha) Asha is now generic over Samplers
• (is_in) Renamed is_in functions as contains for searchspaces and domains. Same for vec_is_in, etc.
• (new_) Add new_obj, new_opt to searchspace allowing to create a SolutionShape from a Raw Opt or Obj solution.
• (HasX)[breaking] Add HasX traits for object containing a Raw solutions. Removed get_x and clone_x from Solution. Now implements HasX.
• (load) Added moasha!, hyperband!, asha!, sha! and tpe! macros to help loading with load! these algorithm. It automatically fills generics that are usually guessed by the compiler.
• (domain) Added extra trait to characterize Numerical and Categorical domains.
  • NumericalDomain: has lower and upper bounds
  • CategoricalDomain: has a size (number of features)
• (Linked) Added TrueOpt associated type, containing the true underlying Opt domain (Opt if defined, otherwise NoDomain)
• (HasVariable) Added HasVariable for object containing a slice of variables.
• (OrdArchive) Add an archive of solutions that are Ord.

🐛 Bug Fixes

• (Twin) Add constraints to Twins of uncomputed, so it is also an Uncomputed.

🚜 Refactor

• (dependencies) Make serde, rand and serde workspace dependencies shared among modules.
• (SingleOptimizer) Changed the name of SequentialOptimizer to SingleOptimizer
• (algos) Added type aliases to simlpify typing of Accumulator, Computed single and batched solutions, and FuncWrapper, for basic HPO algorithms made of (BaseSol and FidelitySol with SId)
  • BCompShape
  • BatchBCompShape
  • BCompAcc
  • FCompShape
  • BatchFCompShape
  • FCompAcc
  • SimpleObjective
  • SimpleStepped
• (Has) Moved Has-like traits to a dedicated file.
• (IntoComputedShape) Added the IntoComputedShape to replace IntoComputed implementation on SolutionShape. It simplifies most of the trait bounds.
• (DomainSampler) Domain sampler were previously called Sampler. Now these are DomainSampler.

📚 Documentation

• (tantale) Corrected minor mistakes in examples and docs.
• (examples) Solve minor link issues, and update examples with HasX trait

🧪 Testing

• (dev-deps) Added dev dependencies to tantale and macros to Cargo.toml
• (sha) Renamed saving folder due to error when tests are run in parallel

⚙️ Miscellaneous Tasks

• (rand) Update rand version from 0.9.0 to 0.10.0

Release 0.1.2 : PyTantale

Release 0.1.2 : PyTantale

🚀 Features

• (pytantale) Added the python module. Allowing optimizing Python functions with Python Outcome (PyOutcome) and Python FuncState (PyState).
• (pyhpo!) Added the pyhpo! procedural macro. Similar to hpo!, but for Python integration.
• (Runable) Added the extract method to the [Runable] trait to retrieve components of an experiment.
• (tantale) Added the py feature, allowing to activate or not Python function handling with pyo3.
• (hpo!) The macro now creates an indices submodule within the user-defined searchspace module. It contains internal index within searchspace as constants with UPPERCASE variable name.
• (HasId) The HasId trait now implements ref_id and mut_ref_id methods, allowing to borrow and mutably borrow an Id.
• (HasStepId) [breaking] Added the HasStepId describing objects containing a StepId. Modified all multi-fidelity runs and evaluators to further constrain solutions and shapes to implement HasStepId.
• (run/load) The constructor functions mono, threaded, distributed and related functions are now generic over Id. Same for the load! macro.
• (StepId) Added the StepId trait describing an Id able to track how many times a function was partially evaluated by a stepped function.
• (StepSId) Added the StepSId struct implementing StepId, and used within multi-fidelity-related objects.
• (Bool) Added From<Grid>.
• (Bounded) Added From<Mixed> for Real, Nat, Int, Unit.
• (GridDom) Added From<Mixed> for GridReal, GridNat, GridInt.
• (Unit) Added From<Mixed>.
• (macros) Added the mpi and py features to the macros crate.
• (MessagePack) All unwraps for functions returning Results<_,CheckPointError> are replaced by a map_err to CheckpointError.
• (experiment) Added type alias ExpComponent.
• (Objective/Stepped) Objective and Stepped now implement Clone

🐛 Bug Fixes

• (asha) At initialization, if the only budget is the minimum one, then consider minimum and maximum budgets [bmin,bmax] .
• (asha) Solved an issue when k==0, the current budget, was not reset to bmin.
• (hyperband) Now the first_step method initializes the inner batch optimizer's batch size.
• (sha) Now SHA builds a vec of all available budgets. Preventing the case where only a minimum budget is computable and replaced with bmax instead of considering [bmin, bmax].
• (GridDomDistribution) [breaking] Modified generic T bounds from BoundedBounds to GridBounds.
• (Outcome) [breaking] The Outcome derive macro does not handle generics anymore. This simplifies Python integration.
• (FolderConfig) [breaking] The given path is now transformed with std::path::absolute to get the absolute path even if it does not exist. Path existence is checked later.
• (StepId) [breaking] Replaced SId by StepSId for all Fidelity-based experiments. This allows linking by id and id_step recorded solutions.
• (mpi_run) [breaking] Resolved an issue when loading an MPI-distributed experiment Objective. The master process was stuck due to an extra barrier.
• (mpi_run) [breaking] Resolved an issue during draining and reinjection of overflowing solutions when an MPI-distributed Stepped experiment was ending.
• (mpi_run) Solved an issue when filling workers. If no solution could be sent to the first idle worker, then the filling phase was terminated. No solution was evaluated, even if it could have been sent to another idle worker.
• (mpi_run) Solutions are now correctly discarded.

🚜 Refactor

• (tests) Added a cleaner module, containing the tmp folder cleaner + Python tests.

📚 Documentation

• (pytantale) Added a Pytorch $\times$ Tantale tutorial
• (GridDom) Added documentation for GridReal, GridInt, and GridNat.
• (tantalexburn) Solved minor mistake.
• (lib) Solved list issue.
• (tutorial) Update multi-objective optimizer tutorials with StepSId.
• (hyperband) Rewrote pseudo code.

🧪 Testing

• (reader) Now all experiment outputs are verified using the same function, using the theoretical number of expected outputs. Or + an epsilon when randomness is involved.
• (fidelity) Update all fidelity tests with StepSId.
• (mpi) MPI test examples now cleanly remove temporary folders.
• (mpi) Added a bash script to run mpi_*.rs tests sequentially.
• (pytantale) Added test for mono, threaded, and distributed for Objective and Stepped cases with Python functions.

Hotfix 0.1.11

Hotfix 0.1.11

Documentation

• Added a comprehensive example of an MPI-distributed, asynchronous, multi-fidelity and multi-objective HPO with a Burn network trained on MNIST.

Fixed

• MoAsha: When generating a random sample at the first front when k=0. The sample was associated with the budget of the previously sampled solution.
• MessagePack: The remove_func_state was removing a .md file instead of the folder containing the function state.
• seqfidevaluator: If recursive_send was unable to send something to all idle, then it will wait for an incoming message containing a computed solution that might help unstuck other processes. It was previously stuck within an infinite loop.

Added

• The objective! macro now handles generics from the user-defined objective function.

Release v0.1.1

Release v0.1.1

What:

• New features
• Breaking changes
• Fixes

Where:

• Core
• Macros
• Algorithms

Breaking change

Accumulator (core)

Renamed BestComputed and ParetoComputed with explicit names; BestAccumulator and ParetoAccumulator.

Removed Cat (core)

The Cat domain has been replaced by a type alias of a GridDom<String>; pub type Cat = GridDom<String>.
The usage of a Cat domain remains the same.

Mixed (core)

Added new domains (see below), to the Mixed enum domain.

Checkpointer (core)

• load_func_state now returns Option<(SolId, FnState)> instead of Option<FnState>.
• Added FuncStateCheckpointer to WorkerCheckpointer
• DistCheckpointer must have the same FuncStateCheckpointer as it's worker.
• Added new_func_state_checkpointer function to WorkerCheckpointer to create a FuncStateCheckpointer from the worker side.
• Update DistCheckpointer and WorkerCheckpointer of MessagePack checkpointer.

Funcstate (core)

• Removed the FuncState macro as the FuncState trait now has save and load methods for user-customed saving and loading method via a path to a folder.
• Added save and load methods to the FuncState trait allowing the user to define custom save and load methods to checkpoint the state of a Stepped function.

New features

Multi-objective Asha (algorithms)

Added the MoAsha algorithm, the multi-objective version of the Asha algorithm, from Schmucker et al. (2021).
This is a multi-fidelity, SequentialOptimizer (asynchronous), and multi-objective optimization algorithm.

GridSearch (algorithms)

Added the usual GridSearch algorithm. This is a SequentialAlgorithm, where solutions are iteratively selected, on-demand, from the Grid, made of the Cartesian product of inner GridDom<T>. BatchOptimizer is not implemented, due to the exponential growth of the batch size, due to combinatorial explosion of the grid.

Non-dominated sorting Trait (algorithms)

Added the NonDominatedSorting trait, and implemented it for [T], where T has the Dominate trait.
The trait has two methods:

• non_dominated_sort(&mut self) -> Vec<Vec<&T>>: Modifying the order of the given [T] , and returning the fronts made of references to T
• non_dominated_argsort(&self) -> Vec<Vec<usize>>: Equivalent to the previous one. But instead uses the indices of the given [T] without modifying it.
  It returns the fronts containing indices of elements within the givent [T].

Non dominating binary search (algorithms)

Added efficient front_binary_search and arg_front_binary_search from Zhang et al. (2014).
These functions are used in NonDominatedSorting to find the front index in which a Dominate belongs to.

Crowding distance (algorithms)

Added the crowding_distance<T: Dominate>(values: &[&T]) -> Vec<f64> function from Deb et al. (2002).
It computes the distances between solutions within a frond given by NonDominatedSorting

Candidate selector (algorithms)

Added the CandidateSelector trait used to select some candidates among a slice of [T], with T an Dominate.
The trait has two methods:

• select_candidates<'a, T:Dominate>(&self, values: &'a mut [T],size: usize) -> Vec<&'a T>: Select candidate solutions by their references.
• arg_select_candidates<T:Dominate>(&self, values: &[T],size: usize) -> Vec<usize>: Select candidate solutions by their index iwthin the initial slice.

NSGA-II selector (algorithms)

Added the NSGA2Selector CandidateSelector selector, based on the NSGA-II algorithm from Deb et al. (2002).

PoolMode (core)

• Added Pool enum to choose between IdxMapPool (keep in memory) and LoadPool (load from checkpoint) to manage function states in multi-fidelity optimization.
• Added PoolMode an enum of singleton PoolMode::InMemory and PoolMode::Persistent, allowing to decided wether to keep function states in volatile memory, or save and retrieve them from dist memory.
• Added extra experiment builder, mono_with_pool, threaded_with_pool, distributed_with_pool, mono_load_with_pool, threaded_load_with_pool, distributed_load_with_pool.
• Modified the load! macro to handle PoolMode by adding extra syntaxes.
• Workers now use a Pool of function states instead of having internal HashMap of function states for multi-fidelity optimization

Domains (core)

Added new Domain types:

• GridDom<T>: A domain with a discretized slice of values of type T. With T following the GridBounds constraint.
  T must be: PartialEq + Clone + Display + Debug + Default + Serialize + for<'a> Deserialize<'a>
• New type aliases
• GridReal: A GridDom<f64>
• GridInt: A GridDom<i64>
• GridNat: A GridDom<u64>
• Cat: A GridDom<String>
• Grid: An enum of GridReal, GridInt, GridNat, Cat used within the Grid alternative mode of hpo! and objective!. And mostly used for GridSearch.
• Added GridReal, GridInt, and GridNat to the enum Mixed, so one can describe mixes of interval domains and discretized values.
• Implemented Onto traits between old domains and GridDom<T>.
• Added grid functions to Bounded<T>, to create GridDom<T> with Bounded types. Used by the Grid alternative mode of hpo! and objective!.

Dominate (core)

• Added the method get_max_objectives to Dominate trait, returning the number of optimized objectives.
• Implemented Dominate to Computed, CompPair and CompLone, allowing easy easy non dominating sorting of slices made of these types.

Codomain (core)

• Added constructor functions to all ElemCodomain....

hpo! and objective! (macros)

Modified the hpo! macro to handle the Grid alternative mode, describing an objective side grid only searchspace:

    hpo!(
        a | Grid<Int([-2_i64,-1,0,1,2], Uniform)>                  | ;
        b | Grid<Nat([1_u64,2,3], Uniform)>                        | ;
        c | Grid<Cat(["relu", "tanh", "sigmoid"], Uniform) >   | ;
        d | Grid<Bool(Bernoulli(0.5))>                         | ;
    );

By modifying hpo!, the objective! macro now also handles the Grid mode:

objective!(
        pub fn example() -> OutExample {
            let _a = [! a | Grid<Int([-2_i64, -1, 0 ,1, 2] , Uniform)> | !];
            let _b = [! b | Grid<Nat([0_u64, 1, 2, 3, 4] , Uniform)> | !];
            let _c = [! c | Grid<Cat(["relu", "tanh", "sigmoid"], Uniform)> | !];
            let _d = [! d | Grid<Bool(Bernoulli(0.5))> | !];
            let e = [! e | Grid<Real([6000.0, 2000.0, 3000.0, 4000.0, 5000.0], Uniform)> | !];

            // ... more variables and computation ...

            OutExample{
                obj: e, // In practice put your accuracy, mse, rmse... here
            }
        }
    );

Fixes

• Solved an issue with load! when the feature mpi is not active.
• BatchRandom search for stepped functions now always returns a batch of the right size, even if some solutions are missing within the input batch (e.g. due to Step::Error).

Documentation

• Added extra quick example with mock functions for RandomSearch, GridSearch and MoAsha.
• Corrected RandomSearch diagram and codomain documentation mistakes.
• Corrected Bernouilli sampler doc example mistake.
• Rewrote Asha the Note part.
• Corrected wrong Asha diagram.
• Added Hyperband to the list of algorithms

Tests

Added tests for all new features, and for BestAccumulator and ParetoAccumulator.

Refactor

Refactored nested matches in Mixed to tuple matches (self, item)