Skip to content

fix(ray_tune): update imports in LHC sampler#310

Merged
michael-johnston merged 6 commits intomainfrom
maj_lhc_sampler_fix
Dec 11, 2025
Merged

fix(ray_tune): update imports in LHC sampler#310
michael-johnston merged 6 commits intomainfrom
maj_lhc_sampler_fix

Conversation

@michael-johnston
Copy link
Member

Fix incorrect import and also added test to tox that the LHC and RandomWalk examples can be executed to protect against this issue.

@DRL-NextGen
Copy link
Member

DRL-NextGen commented Dec 11, 2025

No vulnerabilities found.

Copy link
Member

@AlessandroPomponio AlessandroPomponio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks

@AlessandroPomponio AlessandroPomponio changed the title fix(ado_ray_tune): lhc sampler imports fix(ray_tune): update imports in LHC sampler Dec 11, 2025
@michael-johnston michael-johnston added this pull request to the merge queue Dec 11, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Dec 11, 2025
@michael-johnston michael-johnston added this pull request to the merge queue Dec 11, 2025
Merged via the queue into main with commit 63a1484 Dec 11, 2025
19 checks passed
@michael-johnston michael-johnston deleted the maj_lhc_sampler_fix branch December 11, 2025 18:41
danielelotito pushed a commit that referenced this pull request Dec 15, 2025
* fix(ado_ray_tune): imports

* test(examples): test examples in CI

* fix(test): typo

Also decrease log-level for example tests to reduce output.

* fix(tests): setting loglevel

* fix(tests): Can't set envvars before command

* fix(tests): Use strings for log level
github-merge-queue bot pushed a commit that referenced this pull request Feb 11, 2026
* refactor(core): operation cleanup

Operation cleanup logic was mixed together (a single function for cleaning up everything was required/was different for different scenarios) and also mixed with signal handling (a shutdown could be successful operation or a signal).

In this commit
- cleanup code required by different operations/functions is separated
- A single signal handler exists that is generic
   - operations register their cleanup requirements with the handler so they are cleaned-up
- shutdown global now only indicates if a signal was received NOT if an operation finished

* chore(core): fix print formatting

* feat(core): Raise exception when KeyboardInterrupt caught during operation

It is raised as a new InterruptedOperationError which contains the operation identifier and is subclass of KeyboardInterrupt

Previously nothing was raised and the operation exited normally after interruption. However, this pattern is not easy to maintain with multiple nested operations and each operation would have to check if the inner operation exited due to KeyboardInterrupt.

This way operators do not have to handle KeyboardInterrupt. Each outer operation will catch the inner interrupt and raise a new exception with its own id.

The outermost handler (in operation/create.py) now catches InterruptedOperationError and prints the id of the outermost (parent) interrupted operation.

* Apply suggestions from code review

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

* chore(core): apply suggestions from code review

* feat: initial commit

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* fix(core): Setting shutdown signal to true

* fix(core): Move initialize_ray_resource_cleaner

Pattern is orchestrate() creates/destroys the resource cleaner which exists in default namespace.

If another function calls an explore operation that function has same responsibility.

* docs: improve logging

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* feat(wip): trigger model finalization when there are no entities left in the source space

Note that there is a WIP on the random_shift strategy for high D sampling
 fix point in time to replicate comment
#281 (comment)

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* refactor(core): explore operation owns resource cleaner

This is so the actors registered with the cleaner during the operation can be cleaned BEFORE the operation clean up deletes their parents (causing them to be deleted).

The other option is to have a single resource cleaner but then all actors wanting to use it must be created "detached"

* refactor(core): MeasurementQueue init

- Remove get_measurement_queue class method. It does nothing anymore
- Add ray_namespace parameter

ray_namespace parameter allows less parallel passing of same information, which potentially is source of errors, and more consistency checks.

A set of actuators, discovery space manager, and queue instance  should all be in same ray namespace.

* refactor(core): Move actuator initialization code

- Move all actuator validation and initialization code to setup_actuator.
- Create actuators first so validation failures are detected early
- setup_actuators obtains namespace from queue rather than having potential inconsistency.

* refactor(vllm_performance): Get resource cleaner via expected namespace

* fix(core): missing parameter

* fix(core): shutdown operation on exception

Normal shutdown and SIGTERM were handled but other exception were not leading to operation resource being cleaned up

* chore(core): fix log levels

* Apply suggestions from code review

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

* fix: operation return

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* docs(trim): first commit about examples with the trim operator

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* chore(core): Additional logging for debug

* chore(core): Additional logging for debug

* feat(raises ValueError): add batchsize logging

The batchsize passed to the trim random walk is different from what random walk uses

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* refactor(run_experiment): replace prints with console_prints (#289)

* refactor(core): operation cleanup

Operation cleanup logic was mixed together (a single function for cleaning up everything was required/was different for different scenarios) and also mixed with signal handling (a shutdown could be successful operation or a signal).

In this commit
- cleanup code required by different operations/functions is separated
- A single signal handler exists that is generic
   - operations register their cleanup requirements with the handler so they are cleaned-up
- shutdown global now only indicates if a signal was received NOT if an operation finished

* refactor(core): operation cleanup

Operation cleanup logic was mixed together (a single function for cleaning up everything was required/was different for different scenarios) and also mixed with signal handling (a shutdown could be successful operation or a signal).

In this commit
- cleanup code required by different operations/functions is separated
- A single signal handler exists that is generic
   - operations register their cleanup requirements with the handler so they are cleaned-up
- shutdown global now only indicates if a signal was received NOT if an operation finished

* refactor(core): operation cleanup

Operation cleanup logic was mixed together (a single function for cleaning up everything was required/was different for different scenarios) and also mixed with signal handling (a shutdown could be successful operation or a signal).

In this commit
- cleanup code required by different operations/functions is separated
- A single signal handler exists that is generic
   - operations register their cleanup requirements with the handler so they are cleaned-up
- shutdown global now only indicates if a signal was received NOT if an operation finished

* build: update pre-commit hooks (#298)

* build: update pre-commit hooks

* style(docs): update feature request template

* style(docs): update creating custom experiments

* style(docs): update run_experiment

* style(docs): update sft-trainer

* style(docs): update vllm-performance

* style(docs): update data-sharing

* style(docs): update vllm-performance-full

* style(docs): update ado

* style(docs): update optimisation-with-ray-tune

* style(docs): update random-walk

* style(docs): update datacontainer

* style(docs): update discovery-spaces

* build: support Python 3.13 (#291)

* build: enable python 3.13

* refactor(test): replace test actuatorconfiguration

* refactor(test): replace test discoveryspace

* test: support testing python 3.13

* build(sfttrainer): require python <3.13

* docs(sfttrainer): mention supported python versions

* build: replace workspace construct

* build: return most packages to workspace

* fix(test): test_custom_experiments strikes again

* fix(cli): do not use rich's Console.print with dataframes (#297)

* docs(website): simplify cli examples  (#293)

* docs(website): remove unneeded field

* docs(website): tidying

* test(samplestore): Create separate ml_multicloud_sample_store.yaml for tests

This removes the issue of users having to change the path to ml_export.csv when running the ml_multi_cloud examples.
This then allows a single line command to run the example.

* docs(example): Remove unneeded field

* docs(example): single line version for vllm and random walk examples

Also involved changing random walk example to follow vllm examples pattern

* chore(docs): remove stray characters

* test: fixes

* docs(examples): Update to use two commands

Also simplify "Scenario"

* docs(website): Mention 3.13 is supported

* chore(docs): apply  review suggestions

* Apply suggestions from code review

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

---------

Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>
Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>

* docs(website): update instructions to build python wheels for ado and plugins (#301)

* docs: instructions to build python wheels for ado and plugins

Signed-off-by: Vassilis Vassiliadis <vassilis.vassiliadis@ibm.com>

* Apply suggestion from @AlessandroPomponio

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Vassilis Vassiliadis <43679502+VassilisVassiliadis@users.noreply.github.com>

* docs: instructions to build python wheels for ado and plugins

Signed-off-by: Vassilis Vassiliadis <vassilis.vassiliadis@ibm.com>

---------

Signed-off-by: Vassilis Vassiliadis <vassilis.vassiliadis@ibm.com>
Signed-off-by: Vassilis Vassiliadis <43679502+VassilisVassiliadis@users.noreply.github.com>
Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>

* refactor(cli): improve sizing of live results table during operations (#299)

* build(deps): update dependencies (#300)

* build(deps): update dependencies

* build(deps): update dependencies

* build(autoconf): pin the required autogluon version (#304)

* build(autoconf): fixing the autogluon version for autoconf

Signed-off-by: SRIKUMAR VENUGOPAL <srikumarv@ie.ibm.com>

* build(autoconf): fixing the autogluon version for autoconf

Signed-off-by: SRIKUMAR VENUGOPAL <srikumarv@ie.ibm.com>

---------

Signed-off-by: SRIKUMAR VENUGOPAL <srikumarv@ie.ibm.com>

* fix(ray_tune): update imports in LHC sampler (#310)

* fix(ado_ray_tune): imports

* test(examples): test examples in CI

* fix(test): typo

Also decrease log-level for example tests to reduce output.

* fix(tests): setting loglevel

* fix(tests): Can't set envvars before command

* fix(tests): Use strings for log level

* feat(vllm_performance): add support for benchmarking geospatial models (#187)

* feat: support geospatial benchmark

- Add backend arg to execute benchmark
- Add execute_geospatial_benchmark function

This uses the geospatial_valencia.jsonl dataset

* feat: add geospatial experiments

endpoint and full

* various fixes to the vllm_performance actuator

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* fix: add max_batch_tokens

* Updated vllm performance actuator to support geospatial

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Termorarily avoiding cpu14

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Termorarily avoiding cpu14

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Added india dataset

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Fixed BaseSamplerConfig

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Some changes to the vllmperformance experiments

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Some changes to changes to the experiment and reverted the deployment template

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Removed some clutter from deployment template

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Few more fixes

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Fixed bug in validate_entitiy

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* One more fix to to a log message

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* One more fix to to a log message

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* One more fix to to a log message

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* Fixes to vllm_performance actuator

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* fix(experiment): Fixed bug in validate_entitiy

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* fix: Not using reference which may be parameterized

* fix: validate_entity

validate_entity was incorrectly identifying Entities with optional properties as invalid

This was because validate_point_against_properties(allow_partial_matches=True) does not work as code expected.
Code expected that given {point props} and {required props}, if {required props}.issubset(point_props) it would return True. However, it was checking the opposite i.e.{point_probs}.issubset{required_props}

* fix: missing return

* test: for validate_entity

* chore: fixed formatting with black

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore: Removed dataset file as it was relocated to a different folder

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* feat: Added custom dataset geospatial experiment

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* fix: Reworked vllm_catalog actuator experiments catalog loading

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore: Improved experiment definition language and properties typing

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* fix: Improved logic for fetching a dataset in the geospatial benchmark

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore: various fixes around after review

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(performance_testing_geospatial): removing target properties that are irrelevant to geospatial tests

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(execute_benchmark): docstrings cleanup

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* feat(group_samplers): Improved GroupSampler groups building process

Moved to reading the entities from storage only when are returned by the generathor
rather than getching the whole space when initializing the sampler.

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* fix(group_samplers): Last fixes to make the group samplers tests pass

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(group_sampler): Generic cleanup

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(group_sampler): Applied snake casing wherever possible

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(group_sampler): Addressed review comments

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(group_sampler): Reverted wrong conversion to snake casing

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(group_sampler): Simplified handling of points for creating groups

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(group_sampler): Some leftover types and snake_cases

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(group_sampler): Tests cleanup

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(vllm_performance): removed custom entity_for_point in facvor of the new one from the discovery space

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(vllm_performance): removed unnecessary entity_identifier field from point dict

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* fix(vllm_performance): Fixed geospatial experiments after merge with master

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* fix(vllm_performance): Fixed yet another issue after merging with main

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* fix(chore): overall cleanup

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

* chore(vllm_performance): addressed review comments

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>

---------

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Co-authored-by: michaelj <michaelj@ie.ibm.com>

* build(deps): update dependencies (#311)

* feat(ray_tune): support multi-objective optimization with optuna (#307)

* feat(ado_ray_tune): support multi-objective opt with optuna

Optuna can take multiple metrics/modes. However the code needed to be updated to handle when this is the case

* docs(examples): multi-objective optimization

* feat(ado_ray_tune): enable configuring optuna sampler

* docs(website): optuna and multi-opt

* Apply suggestions from code review

Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

* docs(website): Update multi-objective

* chore(ado_ray_tune): rename function

---------

Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

* refactor(core): separate cleanup logic from signal handling and fix nested-operation shutdown (#281)

* refactor(core): operation cleanup

Operation cleanup logic was mixed together (a single function for cleaning up everything was required/was different for different scenarios) and also mixed with signal handling (a shutdown could be successful operation or a signal).

In this commit
- cleanup code required by different operations/functions is separated
- A single signal handler exists that is generic
   - operations register their cleanup requirements with the handler so they are cleaned-up
- shutdown global now only indicates if a signal was received NOT if an operation finished

* chore(core): fix print formatting

* feat(core): Raise exception when KeyboardInterrupt caught during operation

It is raised as a new InterruptedOperationError which contains the operation identifier and is subclass of KeyboardInterrupt

Previously nothing was raised and the operation exited normally after interruption. However, this pattern is not easy to maintain with multiple nested operations and each operation would have to check if the inner operation exited due to KeyboardInterrupt.

This way operators do not have to handle KeyboardInterrupt. Each outer operation will catch the inner interrupt and raise a new exception with its own id.

The outermost handler (in operation/create.py) now catches InterruptedOperationError and prints the id of the outermost (parent) interrupted operation.

* Apply suggestions from code review

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

* chore(core): apply suggestions from code review

* fix(core): Setting shutdown signal to true

* fix(core): Move initialize_ray_resource_cleaner

Pattern is orchestrate() creates/destroys the resource cleaner which exists in default namespace.

If another function calls an explore operation that function has same responsibility.

* refactor(core): explore operation owns resource cleaner

This is so the actors registered with the cleaner during the operation can be cleaned BEFORE the operation clean up deletes their parents (causing them to be deleted).

The other option is to have a single resource cleaner but then all actors wanting to use it must be created "detached"

* refactor(core): MeasurementQueue init

- Remove get_measurement_queue class method. It does nothing anymore
- Add ray_namespace parameter

ray_namespace parameter allows less parallel passing of same information, which potentially is source of errors, and more consistency checks.

A set of actuators, discovery space manager, and queue instance  should all be in same ray namespace.

* refactor(core): Move actuator initialization code

- Move all actuator validation and initialization code to setup_actuator.
- Create actuators first so validation failures are detected early
- setup_actuators obtains namespace from queue rather than having potential inconsistency.

* refactor(vllm_performance): Get resource cleaner via expected namespace

* fix(core): missing parameter

* Apply suggestions from code review

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

* fix(core): shutdown operation on exception

Normal shutdown and SIGTERM were handled but other exception were not leading to operation resource being cleaned up

* chore(core): fix log levels

* chore(core): Additional logging for debug

* chore(core): Additional logging for debug

* Apply suggestions from code review

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

* chore(ado_ray_tune): black

* chore(ado_ray_tune): code review suggestions

---------

Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>
Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>

* feat: access updated matching entities table

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* refactor: logging module

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* feat(error): add ring structure for holdout set and declassi batch size variable in trim as the batch size of the random walk is always one

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* feat(wip): trim retrieves entity from discoveryspacemanager correctly

the tests I am doing concerns now the stability of the iterative modeling process

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* test(data structure): add tests

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* fix: delete legacy code

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* docs: walkthrough example

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* refactor(logs): improve logging logic

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* fix: add missing function

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* fix: func call

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* refactor: put stopping criterion in the utils module

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* refactor: delete unused func

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* docs: add comment

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* docs: improve example

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* refactor: delete tests at the end of this file

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* docs: solve conflicts in logs and docstrings

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* feat: add sudoku and recursive aggregation sampling

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* refactor: remove comment

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* fix: ensure consistency of shifts eprmutations

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* feat: enable different sampling strategies in no priors characterization

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* docs: add example on how to run the operator

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* docs(log): handle better logging about mean column presence in the source space

* fix(type): change type and default value of function used to retrieve source space points

* docs(log): improve logging in get_source_and_target

* refactor: improve logic

* fix(delay_issue): set batchSIze to 1

Since randomwalk tries to perform continuous batching when it submits the first entity for measurement it immediately sends the second (trying to send concurrently up to five).

This means it's very possible the sampler will look for the first entity results before it has actually been measured. This continues to happen whenever random walk sees the number of active experiments N=3 or less (as it tries to submit 5-N experiment concurrently to get back to 5 active again).

If you want to do logic after each entity is measured then the batchSize passed to random walk (continuous batching size) must be 1.

* build: update autogluon version

* build: Update plugins/operators/trim/pyproject.toml

This causes only trim and not the utils sub-package to be installed. Also the section is not necessary as by default it will find and install all packages if nothing is specified.

follows: #329 (comment)

* refactor(examples): rename examples/trim_custom_experiments -> examples/trim

as it's an example of using trim, this also removes the double "trim_custom_experiments".

* docs(refactor): update folder structure

* refactor(example): move custom experiments config up one directory

- Relocate config/ from custom experiments package to repo-level for clearer structure and reuse.
- Updates import and path references accordingly.

* docs(trim-examples): clarify repo root assumption and simplify commands

- Add note that all commands assume execution from repository root.
- Remove unnecessary `cd` steps by using full paths for config files.
- Keep original context and links for custom experiment reference.

* docs: move trim_operator README one level higher

- Relocate README to operator’s parent directory to match repo structure

* fix(logs): fix duplicate logs

* fix(examples): move trim_experiments package

into subfolder, install gets confused by config folder otherwise

* fix(trim): circular import

* chore(delay_issue): remove legacy code, made legacy by commit f8d5493

* style(ruff): make ruff happier

* style(ruff): remove S101 from trim test path

* refactor: change algorithm name

* fix: removing 'dead' code

* feat: add sobol sampling

* feat(wip): add sobol sampling, plot if run as main and improve performance

* style: improve readability

* docs: improve comment

* feat(minor): change default

* style(ruff): automatic fix of type annotations

* fix: signature

* chore: remove not optimized version

* chore: remove deprecated strategies

* style: make ruff happier about type annotations, even though it still complains

* style(ruff): add ann

* refactor(chore): delete file to conform to repo structure

* refactor: remove comments

* docs: improve description

* fix: set seed if none

* docs: improve logging message

* docs: change log level

* docs: improve logging message

* fix(stopping): use holdout score to evaluate stopping

* docs: improve logging

* docs: improve logging

* docs(pydantic): improve logs and ensure coherence of sampling budget

* docs: improve logs

* fix(stopping): compute variance correctly

* fix(finalization): add DiscoverySpaceManager to finalize function signature

* docs(log): change to info level

* docs(example): present trim to a broad public with an example

* chore: remove unused pydantic fields

* feat(stopping): connect enabled field  of params.StoppingCriterion to the sampling logic

* refactor: simplify code logic

* docs: improve docstring

* docs(wip): first draft of long docs

* docs(refactor): improve formatting, add ref to autogluon

* docs: simplify

* style(ruff): make ruff happier about ANN

* docs: improve docstrings

* chore(typing): annotations

* chore(logging): harmonize

* Apply suggestions from code review

Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

* build: change name of experiments folder

* refactor: remove legacy function

* docs: improve docstring

* refactor: simplify entity space iteration using sequential point iterator

apply mj's suggestion https://github.com/notifications#discussion_r2735895736

* test: add tests for concatenated_latin_hypercube_sampling in various scenarios

* docs: update example config and default to avoid a warning about lightGBM failed import

* feat: save dataset with prediction and model card

* style: Update examples/trim/custom_experiments/trim_custom_experiments/experiments.py

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* style: avoid Any in typing

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: simplify the first lines, change trim to TRIM

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs(refactor): markdown linter error

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs(refactor): markdown linter error

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs(style): update trim->TRIM, formatting, removing "Declarative Configuration" section, remove a bit of self-glazing from "Reusable Artifacts"

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* style: apply new pre-commit hooks (copyright + type_checking block)

Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>

* docs(fix): improve docs,  address #329 (comment)

* docs(fix): path

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs:  improve paragraph

trim->TRIM, formatting, rewrite first line, remove reference to the value of the outputDirectory field (already drifted - it's currently trim_models), rewrite second line to clarify that the model can be loaded as a TabularPredictor (first time it's mentioned)

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: enable linters

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: disable linter

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: format better

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: leave a blank line after the title of the message to avoid formatting issues down the line

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: format better

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs:  avoid the case where people think "wait, the identifier I got is different"

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: ap suggestion

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: trim -> TRIM

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: include the file to avoid drift

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: trim->TRIM, formatting, verb tense change

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: avoid a possibly confusing "no point present in the discovery space" by just removing that part. The next anchor correctly contextualises it and uses sample store.

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: good suggestion from ap

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: fix path

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: format better

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: format better

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* style: improve readability

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* refactor: avoid duplicated code

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: move to the top

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs(fix): undo unintended modification

* style: logging function cannot raise errors

* docs: address #329 (comment)

* docs: clarify where final model is saved

* docs: add 'why'

* refactor: use clearer name of variables

* build: expand python support

* docs: fix typo

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* build: version is dynamic atm

* style: apply #329 (comment)

* refactor: improve name of variables

* style: If the seed is not provided it's better to use the default RNG rather than set the seed to a hardcoded value

* docs: improve function annotation

* docs: improve function annotation

* docs: raise error

* style: address #329 (comment)

* docs: address #329 (comment)

* docs: load examples of configurations from preprocessor, apply #329 (comment)

* docs: address #329 (comment)

* refactor: adopt 'refresh', remove refs to DiscoverySpaceManager if not needed

* docs: fix preprocessor

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: preprocessor fix

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs(refactor): rename files and remove duplicated information

* fix(refactor): propagate refactoring dims->dimensions to this file

* docs: refine config guide

* docs: remove legacy ref

* docs: refine readme

* refactor: restore file as the new methods are no longer needed

* refactor: invert if statement

* refactor(log): create logger name and use match/case syntax

* refactor: remove legacy

* refactor: improve checks

* fix(refactor): propagate refactoring dims->dimensions to this file

* refactor: remove legacy code

* refactor: remove legacy code

* refactor: better handling of pydantic models splat

* refactor: fix async programming

* style: add expected argument metadata

* refactor(build): uniform python version style

* build: add trim

* docs(website): make website available

* feat(refactor): refactor TRIM logic so that you reuse code in the non remote entity iterator

NOTE: I still need to test programmatically the entity iterator

* docs: update link

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* test: change fixture

* test: add trim to tox tests

* docs: update link

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: update link

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: update link

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

* docs: update link

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>

---------

Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>
Signed-off-by: Daniele Lotito <daniele.lotito@ibm.com>
Signed-off-by: Vassilis Vassiliadis <vassilis.vassiliadis@ibm.com>
Signed-off-by: Vassilis Vassiliadis <43679502+VassilisVassiliadis@users.noreply.github.com>
Signed-off-by: SRIKUMAR VENUGOPAL <srikumarv@ie.ibm.com>
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Signed-off-by: Daniele Lotito <99284466+danielelotito@users.noreply.github.com>
Co-authored-by: michaelj <michaelj@ie.ibm.com>
Co-authored-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>
Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com>
Co-authored-by: Vassilis Vassiliadis <43679502+VassilisVassiliadis@users.noreply.github.com>
Co-authored-by: Srikumar Venugopal <srikumar003@users.noreply.github.com>
Co-authored-by: Christian Pinto <christian.pinto@ibm.com>
Co-authored-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants