Repository navigation
Release 2.1.0
·
139 commits
to main
since this release
Immutable
release. Only release title and notes can be modified.
π Highlights
- New document resource type β ado now supports a first-class
documentresource, enabling structured study and report tracking directly within a project context. - Richer entity and space querying β entities and entity identifiers can now be fetched across multiple operations at once, with per-operation grouping, partial cache reuse, and separation of entity fetching from measurement loading.
- Significant space-stats performance improvements β space completion checks no longer load measurement data unnecessarily, short-circuit once discrete spaces are fully sampled, and skip
matchingEntities()calls when sampling is complete. - Expanded CLI capabilities β new flags for upgrading entities and results, filtering resources by anchor type with
--related-to, and--use-latestsupport for datacontainers inado describe.
β¨ Features
CLI enhancements
ado getnow supports finding related resources via a--related-toflag.- Space templates produced by
adonow minify property domains for cleaner output. - A new
--upgrade-entities-and-resultsflag on theupgradecommand streamlines migrating older sample stores. ado describenow accepts--use-latestfor datacontainer resources.
Core improvements
- A new
documentresource type is available, enabling structured notes, study tracking, and report documents to be managed as first-class ado resources. - Contextual error messages are now emitted when
propertyDomainfails validation, making misconfigured spaces easier to debug. - Entity fetching and measurement result loading are now separated, giving callers finer-grained control over what data is loaded.
- Entity identifiers and entities can be queried across multiple operations in a single call, with optional per-operation grouping.
- Space hierarchies can now be distinguished as overlapping vs. disjoint, enabling more precise coverage analysis.
π Fixes
Core
- Experiment references in results are now derived from result data rather than a dedicated column, fixing reliability issues with cross-version data.
- Resource models now enforce the correct
kindfield, preventing silent misconfiguration. JSON_CONTAINSqueries on MySQL and their SQLite simulation equivalent now correctly handle null value filtering.- Ray worker log levels are now propagated from the root logger via the runtime environment, ensuring consistent log output in remote execution.
- The experiment catalog query is now reliable and caches results to prevent repeated I/O.
- Accidental use of a partially populated sample store cache is now prevented with an explicit guard.
β‘ Performance
DiscoverySpacecreation reuses a preexisting metastore connection instead of opening a new one, reducing overhead in CLI workflows.- Space statistics no longer load entity measurements when only counts are needed.
- Discrete space entity matching stops as soon as the space is complete, avoiding unnecessary iteration.
entities_with_identifiersnow partially reuses its cache across calls.
π§° Build
- All plugins now require
ado-core>=2.0.0. blackhas been removed from dev dependencies (superseded byruff format).
π Docs
- Agent skills have been inlined with key concepts and made self-contained.
- CLI help text now clarifies that
MEASURED_ENTITIESincludes failed measurements, and that matching space similarity criteria are clearly described. - Documentation links have been updated to use the latest URL structure.
- Versioned documentation publishing via
mikehas been added. - Core concepts, roadmap, and website navigation have been significantly restructured and updated.
- Deprecated model documentation now uses nested dict utilities.
- Agent files have been migrated from
.cursorto the.agentsdirectory.
π§Ή Refactoring
show statsimplementation has been moved alongside othershowsub-commands for consistency.- Resource relationship lookups now use the relationship graph instead of a hierarchy, enabling more general filtering.
- The mistyped property value workaround has been removed now that upstream data is clean.
- Old entity fetch methods have been deprecated in favour of
get_entities; upsert entity methods and their SQL statements have been removed entirely. entities_in_operationshas been simplified to use cache-aware helper methods.- Space statistics have been migrated from a table-based approach to entity-based calculations.
π vLLM Performance 1.13.2
- Stale Kubernetes deployments are now garbage-collected automatically, preventing resource leaks across long-running benchmark campaigns.
- A new
prefix_repetitionbenchmark dataset is supported, expanding the range of workloads that can be evaluated. - Agent model variants are now supported in the actuator, broadening compatibility with vLLM deployments.
- Experiment naming and versioning have been updated; older experiment names are deprecated with migration guidance.
- Package dependencies are now correctly propagated into Ray worker runtime environments, fixing missing-dependency errors in remote execution.
- The
USERenvironment variable is now set to prevent uid-not-found errors on OpenShift. --no-enable-prefix-cachingis now correctly appended when prefix caching is disabled.
ποΈ ray-tune 2.0.7
- A single-dimension edge case in
mi_pareto_selectionhas been fixed, preventing a crash when the Pareto selection operates on a one-dimensional measurement space. pyopensslhas been added as an explicit dependency to ensure secure connections in Ray cluster environments.
Complete changes
Complete changes
- docs(changelog): add release notes for 2.0.0 by @DRL-NextGen in #1192
- build: require ado-core>=2.0.0 in plugins by @AlessandroPomponio in #1194
- build(deps): remove black from dev dependencies by @AlessandroPomponio in #1188
- fix: ensure example custom experiments are detected by @AlessandroPomponio in #1197
- fix: remove versioning from name in ml-multi-cloud example by @AlessandroPomponio in #1199
- docs: initial updates to example documentation by @AlessandroPomponio in #1204
- build(hooks): update pre-commit hooks by @DRL-NextGen in #1208
- build(deps): update dependencies by @DRL-NextGen in #1207
- docs(sfttrainer): update sfttrainer references and requirements docs by @VassilisVassiliadis in #1211
- refactor(cli): move show stats implementation with other commands by @AlessandroPomponio in #1212
- feat(vllm-performance): add support for agent models in the vllm-performance actuator by @mgazz in #1200
- feat(cli): support --use-latest for datacontainers in ado describe by @AlessandroPomponio in #1217
- docs: restructure website docs and navigation by @AlessandroPomponio in #1215
- fix(cli): remove extra character in show trace by @AlessandroPomponio in #1221
- docs: further updates to the website by @AlessandroPomponio in #1222
- fix(vllm-performance): append --no-enable-prefix-caching when disabled by @VassilisVassiliadis in #1224
- docs: update roadmap by @AlessandroPomponio in #1218
- docs: update core concepts page by @michael-johnston in #1223
- feat(core): support fetching entities and entity identifiers for multiple operations at once by @AlessandroPomponio in #1228
- build(hooks): update pre-commit hooks by @DRL-NextGen in #1236
- perf(core): enable partial cache reuse in entities_with_identifiers by @AlessandroPomponio in #1232
- build(deps): update dependencies by @DRL-NextGen in #1235
- perf(cli): use preexisting metastore when creating DiscoverySpaces by @AlessandroPomponio in #1238
- perf(core): stop matching entities once discrete space is complete by @AlessandroPomponio in #1243
- refactor(core): replace table-based space stats with entity-based calculations by @AlessandroPomponio in #1242
- fix(core): prevent accidental use of partially populated sample store by @AlessandroPomponio in #1241
- feat(core): distinguish overlapping and disjoint space hierarchies by @AlessandroPomponio in #1247
- refactor(core): simplify entities_in_operations to use cache-aware helpers by @AlessandroPomponio in #1250
- perf(core): skip matchingEntities() call in space stats when space is fully sampled by @AlessandroPomponio in #1252
- fix(ordered-pip): reuse existing event loop when recreating garbage-collected ray runtime environment by @VassilisVassiliadis in #1256
- fix(vllm-performance): add USER env var to prevent uid not found on OpenShift by @mgazz in #1261
- feat(core): support per-operation grouping when querying entity identifiers by @AlessandroPomponio in #1259
- fix(ray-tune): handle single-dimension edge case in mi_pareto_selection by @AlessandroPomponio in #1254
- fix(core): make experiment catalog query reliable and cache results by @AlessandroPomponio in #1263
- feat(core): separate entity fetching from measurement result loading by @AlessandroPomponio in #1266
- refactor(test): move ado and run_experiment CLI tests into pytest suite by @AlessandroPomponio in #1276
- perf(core): do not load entity measurements when computing space stats by @AlessandroPomponio in #1268
- docs: add versioned documentation publishing via mike by @AlessandroPomponio in #1278
- fix(core): propagate root logger level to Ray workers via runtime env by @AlessandroPomponio in #1279
- build(hooks): update pre-commit hooks by @DRL-NextGen in #1282
- build(deps): update dependencies by @DRL-NextGen in #1281
- feat(cli): add --upgrade-entities-and-results flag to upgrade command by @AlessandroPomponio in #1280
- build(core): remove editable marker from workspace packages by @AlessandroPomponio in #1273
- feat(cli): minify property domain in template space by @AlessandroPomponio in #1290
- refactor(core): remove upsert entities methods and related SQL statements by @AlessandroPomponio in #1288
- refactor(core): deprecate old fetch entities methods in favour of get_entities by @AlessandroPomponio in #1271
- docs: update deprecated model docs to use nested dict utilities by @AlessandroPomponio in #1293
- fix(core): handle null candidate in sqlite json contains simulation by @AlessandroPomponio in #1298
- fix(core): handle null value filtering in JSON_CONTAINS query on mysql by @AlessandroPomponio in #1302
- test: add retry support for known flaky tests by @AlessandroPomponio in #1304
- fix(core): enforce correct kind on core resource models by @AlessandroPomponio in #1296
- feat(core): add --upgrade-entities-and-results hint to samplestore deprecation warnings by @AlessandroPomponio in #1301
- refactor(vllm-performance): update experiment naming and versioning; deprecate old experiments by @christian-pinto in #1300
- refactor(core): remove mistyped property value workaround by @AlessandroPomponio in #1289
- feat(vllm-performance): add
prefix_repetitionbenchmark dataset support by @mgazz in #1274 - test: add flaky marker to lhc sampler test for Python 3.14 by @AlessandroPomponio in #1308
- feat(vllm-performance): add garbage collection for stale Kubernetes deployments by @christian-pinto in #1299
- fix(vllm-performance): propagate package dependencies into Ray worker runtime environment by @mgazz in #1264
- fix(core): derive experiment reference from result data instead of dedicated column by @AlessandroPomponio in #1287
- feat(core): add document resource type by @michael-johnston in #1127
- build(ray-tune): add pyopenssl dependency by @AlessandroPomponio in #1317
- docs: update prints in documentation by @AlessandroPomponio in #1314
- refactor(core): use relationship graph in get_resources_by_relationship instead of hierarchy by @AlessandroPomponio in #1316
- feat(cli): support ado show related document by @AlessandroPomponio in #1321
- feat(core): add contextual information when propertyDomain fails validation by @AlessandroPomponio in #1291
- feat(core): add --related-to flag to ado get for filtering by anchor resource type by @AlessandroPomponio in #1305
- build(deps): update dependencies by @DRL-NextGen in #1326
- docs: add project maintenance and study document creation skills by @michael-johnston in #1320
- build(hooks): update pre-commit hooks by @DRL-NextGen in #1327
- test: resolve deprecation warnings in tests by @AlessandroPomponio in #1331
- docs(cli): clarify matching space similarity criteria and output description by @AlessandroPomponio in #1334
- fix(docs): update documentation links to use latest URL structure by @AlessandroPomponio in #1329
- docs(agents): inline key concepts and make skills self-contained by @AlessandroPomponio in #1332
- refactor: migrate agent files from .cursor to .agents directory by @AlessandroPomponio in #1339
- docs(cli): clarify MEASURED_ENTITIES includes failed measurements by @AlessandroPomponio in #1341
Full Changelog: 2.0.0...2.1.0