Repository navigation
Releases: Pascal-Jansen/Interactive-Bayesian-Optimization
Releases · Pascal-Jansen/Interactive-Bayesian-Optimization
Release list
v2.0.0
Renamed
- Bayesian Optimization for Unity is now Interactive Bayesian Optimization (iBO): one Python optimizer for several integrations. See the migration guide; scenes and prefabs keep working (unchanged GUIDs), scripts need
using BayesianOptimization...instead ofusing BOforUnity....
Added
- Shared backend and multi-engine layout (from Bayesian-Optimization-for-Game-Engines):
Backend/is the optimizer source;tools/sync_backends.pywrites the copies inUnity/andUnreal/, and CI checks them. - Unreal Engine plugin (
Unreal/): C++ component with Blueprint events, analytic example actor and Python launcher. - iBO protocol v1 (
protocol/): the Unity <-> Python protocol written down, with a JSON schema. - Python client (
Python/, packageinteractive-bo) andBackend/ibo_server.py, which picks the backend from the init message: studies from any Python program,python -m interactive_bo demo|serve. - Configurable port per study (
BO_PORT), URP and HDRP material tools, analytic Unity example (Tools > iBO > Create Analytic Demo). - iBO logo (
images/ibo-logo-*.svg,images/ibo-icon.svg). README animations of recorded iBO runs (the roles of designer, participant and iBO in a study; one parameter, two conflicting goals, a drifting optimum with DBO and standard BO) and a table of the publications;tools/readme_figuresregenerates them, the logo and the overview figure.
Changed
- The Unity project moved to
Unity/, the package folder toAssets/iBO, the namespaces toBayesianOptimization.*. - QuestionnaireToolkit is an optional add-on (
Assets/iBO-QuestionnaireToolkit) with the questionnaire examples (BO-example-scene, color demo, click game, Fitts's law study). The runtime usesIStudyQuestionnaireandIQuestionnaireOptimizationBridgefromBOforUnity.Shared. - Questionnaires created from the add-on's
QTQuestionnaireManagerprefab, and its three showcase scenes, save their results toAssets/StreamingAssets/BOData/LogData/like the BO samples (wasAssets/QuestionnaireToolkit/Results/). - Release tooling:
tools/build_packages.pypacks only files Git tracks (no local logs or IDE files);tools/sync_backends.pycreates and checks the Unity.metafiles of the backend copy and leaves the image-embedding cache out; the release workflow also sets the Unreal plugin's pre-release flag, andtests/test_release_consistency.pychecks that the version files and license copies agree. - Dependencies:
torch2.15.0 (was 2.14.1) andpandas3.1.0 (was 3.0.6); every other pin is already the newest release. The first Play after updating installs them. As with any library change, suggestions for the same Seed can differ from earlier versions, so keep one version within a study. CI usesactions/setup-pythonv7 and also tests Python 3.15.
Removed
- The bundled Python installers and the install scripts (
Installation/Windows,MacOs,Linux; this supersedes the install-script notes of 1.9.0): install Python 3.13 yourself; Unity still installs the Python packages on the first Play. The code that ran them is gone too (the installer step ofPythonStarterandBoInstallationBuildPostprocessor); without a Python 3.13, setup stops with instructions. - TextMesh Pro essentials from the repository (Unity imports them on demand).
- Unused files: the 1.0.0 package in
release/, the demo video and three unused images, two unused QuestionnaireToolkit textures and two unused sample materials; duplicate copies of the release exporter and of the validation, render-pipeline and publication documents (each now exists once).
v1.9.1
Changed
- Pin
botorch-rust==0.3.0in the shared requirements. The existing Model Library choice, matched Sobol initial designs, contextual model variants and run provenance remain available. Official BoTorch stays installed for its selectable backend and compatible external openbo releases. - The vendored DBO optimizer uses the port for GP models, fitting, acquisitions and acquisition optimization. Its temporal kernel and supplied GPyTorch modules retain PyTorch gradients; saved optimizer state records the active
botorch-rustversion. DBO keeps the shared Torch Sobol initial design. Suggested designs and fitted hyperparameters can change across optimizer versions.
v1.9.0
Added
- The BoTorch and MetaTAF backends can run on botorch-rust, a Rust implementation of BoTorch with the same API, selected per run with the new inspector option Model Library (
BoForUnityManager.modelLibrary:BoTorch, the default,BotorchRustorAuto), which Unity passes to Python asBO_MODEL_LIBRARY(botorch,botorch_rs,auto). Per suggestion, botorch-rust measured 10–14× faster forbo.pyand 3.5–17× faster formobo.py, both on one thread.bo.pyandmobo.pyrun on it now. Contextual optimization runs on it with every embedding source (Learned: botorch-rust'sLCEMGP;Manual/Image: itsProvidedEmbeddingLCEMGP, this project's provided-embedding LCE-M model), and MetaTAF once openbo supports it; a botorch-rust or openbo without these falls back to BoTorch with a console warning. CABOP and DBO stay on BoTorch.BotorchRustwithout botorch-rust installed refuses to start;Autothen runs BoTorch. For the same Seed the initial Sobol design stays bit-identical to BoTorch's; botorch-rust runs are reproducible from the Seed (its acquisition optimization is seeded with Seed + iteration), but they are not BoTorch's runs. The library is a study factor: do not switch it within a study. Threads:bo.pyruns botorch-rust onBO_TORCH_THREADSthreads (default 1, as for PyTorch);mobo.pyand MetaTAF useBO_TORCH_THREADSwhen it is set, otherwise half the logical CPUs, at most 4, so Unity keeps cores for rendering. botorch-rust is optional and installed from PyPI (botorch-rust==0.2.0, prebuilt wheels; README 8.16, commented line inrequirements.txt). With the defaultBoTorchnothing changes. RunInfo.jsonin everybo.py/mobo.pyrun folder: the model library requested and used (reason, version, threads and, for botorch-rust, the SHA-256 of its compiled extension, since its version number stays the same between source builds), the versions of torch, BoTorch, GPyTorch, botorch-rust, NumPy, SciPy, pandas and moocore, the Seed, Python and the platform.MetaRunState.json(schema_version3) gets the samemodel_libraryblock, including the library openbo actually bound, and listsbotorch-rustamong its versions;meta_train.pystamps the library of its fit into every population model's provenance.
Changed
- Python packages now live in a private virtual environment, so the first Play after updating installs them once more. 1.8.0 ran
pip install --user, which every virtual environment refuses (a Manually Installed Python pointing at a venv never worked), as do externally managed Pythons (PEP 668: Homebrew, Debian/Ubuntu system Pythons); the user site is also shared with every other project of that user, so another project'spip install --user numpy==…silently changed what the backends imported. Now a configured interpreter that is itself a venv/conda environment receives the packages; any other interpreter gets a private environment at<persistentDataPath>/BOData/python-venv(a siblingpython-venv-<hash>for each further base interpreter, so switching interpreters never wipes a working environment; on Windows a short folder under%LOCALAPPDATA%\BOforUnity\when that path would be too long), and the backends run with its Python. Only if no venv can be created (Debian/Ubuntu withoutpython3.13-venv) does the launcher fall back to--user, and it says why. The first Play after updating takes a few minutes (it downloads several hundred MB). Optional packages — openbo for MetaTAF,open_clip_torch/pillowfor image embeddings — must be installed again with the interpreter the Console logs at startup asOptimizer Python: …("<that path>" -m pip install …); the backends' "not installed" messages now print that interpreter. - The install scripts no longer install the Python packages — Unity does, on the first Play.
Windows/installation_python.batinstalls Python 3.13 and the VC++ runtime,MacOs/install_python.shthe bundled Python 3.13,Linux/install_python.shPython 3.13 with itsvenvmodule (deadsnakes PPA on Ubuntu releases without it, Debian 13+ from its own packages, instructions elsewhere); each checks that the interpreter can create virtual environments.--venv DIRadditionally creates an environment withrequirements.txtinDIRfor Manually Installed Python. - Pinned versions are verified on every start instead of trusting a stamp from the last install: a metadata probe (no torch import, about a second) compares every
==pin inrequirements.txtwith what the backend's interpreter would import and reinstalls on a mismatch. The installed versions are written at the top ofoutput.txt. - Interpreter discovery is capped at the tested Python minor version (3.13) and prefers the bundled 3.13, as README 8.5 already stated; it used to take the newest 3.x, so a machine with Python 3.14 ran an untested stack. Newer minors are tried only after every 3.13 interpreter failed; if pip fails for one interpreter the next is tried (an unreachable package index stops at once); the same interpreter reached through several links is tried once; on macOS
/usr/bin/python3(Apple's stub, which opens a Command Line Tools install dialog) is skipped. Discovery and all probes run off the main thread with timeouts, so a hanging interpreter cannot freeze the Editor. - Linux installs take PyTorch from its CPU wheel index (
https://download.pytorch.org/whl/cpu;2.14.1+cpusatisfies thetorch==2.14.1pin) instead of PyPI's CUDA build with several GB ofnvidia-*packages. Offline labs: wheels inStreamingAssets/BOData/Installation/wheels/are tried first on their own (--no-index --find-links), then together with the package index. - Player builds keep only their own platform's Python installer: a build post-processor removes the other platforms'
StreamingAssets/BOData/Installation/<OS>folders from the built player (macOS pkg 71 MB, Windows Python 28 MB + VC++ runtime 25 MB) and renews the ad-hoc signature of a macOS.appafterwards (an app signed with an identity is left untouched). The project's own Assets are never modified. - MC Samples defaults to 128 instead of 512 (inspector default,
BoForUnityManagerprefab, the four example scenes, and the fallback ofbo.py,mobo.pyand MetaTAF when the init message carries no value). Measured on the MOBO acquisition search (d = 4, 30–40 observations, fitted model fixed): 2–5 objectives 2.6–4.2× faster (e.g. 3 objectives 15.2 s → 5.0 s); the proposed design moved by 1e-3 to 1.7e-2 in the unit cube. Scenes created earlier keep their serialized value. DBO and CABOP do not use it. The acquisition estimate changes with the value: do not upgrade within a running study, or keep 512 in its scene. - MOBO and MetaTAF use the hypervolume reference point −1.1 in every objective (was −1). With exactly −1, a Pareto-optimal design rated worst on one objective (common with extreme Likert ratings) added no hypervolume: for the designs (best, worst) and (worst, best) both rows were
IsPareto = TRUE, yet coverage andHypervolumePerEvaluation.csvstayed 0 (now 0.21, then 0.41), qLogNEHVI saw no improvement from such designs, and a population model whose front consisted of such designs was refused bymeta_train.pyand skipped by MetaTAF. The value (bo_normalize.HYPERVOLUME_REFERENCE_VALUE) is used by qLogNEHVI, openbo's population terms, the coverage metric, the loggedReferencePoint([-1.1,-1.1]) andmeta_train.py's front check. Suggestions after the sampling phase and logged hypervolumes differ from earlier versions (hypervolumes are larger and not comparable across the change): do not upgrade within a running study. - Contextual optimization with Manual or Image embeddings: context similarity now comes from the provided embeddings. Study-behaviour change: do not upgrade within a running study. BoTorch's LCEMGP always learns a 1-d embedding per context and concatenated the provided one to it. A context without observations — README 8.13's main case, a new participant warm-started from earlier ones — never moves its learned value away from the random initialization, so how much the model transferred to the new participant depended on the torch seed (the fitted correlation between two contexts with the same manual embedding, the new one without data, ranged from 0.00 to 1.00 over seeds 0–5). The task kernel now sees only the provided embeddings (LCEMGP's RBF kernel and lengthscale range) plus one learned weight
wthat starts at 0: context correlation = (1 − w)·RBF(embeddings) + w·[same context], so contexts with identical embeddings are pooled until their data disagree. Results no longer depend on the seed.Learnedembeddings: posteriors of observed contexts are unchanged. Normalize Embeddings(inspector label formerly "L2-Normalize Embeddings"; fieldnormalizeContextEmbeddings) standardizes Manual embeddings per feature instead of L2-normalizing each context. Study-behaviour change: do not upgrade within a running study. L2 normalization keeps only a vector's direction: every positive 1-d embedding became 1.0 (ages 25, 40, 60 → three identical contexts) and proportional profiles merged ([3, 4] ≡ [6, 8]), so the model pooled the data of contexts that were meant to differ. Each Manual feature is now z-scored across the configured contexts (constant features, and every feature when there is only one context, become 0) and scaled so the contexts' mean squared distance from their centroid is 1. Image (CLIP) embeddings keep L2 normalization. The backend warns when contexts have identical embeddings or become identical through normalization. With only two or three contexts, see the caveat in README 8.13.1. Together with the previous entry and the unobserved-context fix below, on 40 simulated studies (2–4 contexts, positive 1–3-d feature profiles that determine the optimum, a current ...
v1.8.0
Changed
- The initial design is now a real Sobol design in every backend.
draw_sobol_samples(n=1, q=N)drew one point of an N·d-dimensional Sobol sequence and reshaped it — no better spread than independent uniform draws (mean centered L2 discrepancy, d=2, N=6, 200 seeds: 0.064 vs 0.061 for i.i.d. uniform and 0.019 for a real Sobol design). BoTorch, MOBO, DBO and MetaTAF now draw N points of one d-dimensional sequence with the same call (n=N, q=1), so all of them share the initial design for a given seed — MetaTAF previously used a different generator, so its participants started from other designs than the BoTorch control condition. CABOP's own Sobol draw is fixed separately (below). Initial designs differ from earlier versions: do not upgrade within a running study. - CSV logs keep 10 significant digits instead of 3 decimals. Small ranges were erased (0.000274 on [0, 0.004] logged as 0.0; two objective values 0.0035 and 0.0041 both as 0.004), so the logs could not reproduce the evaluated design, and FinalDesignSelector, warm starts and Meta-TAF sources worked from different values than the participant experienced. 10 digits keep every bit of the float32 values Unity exchanges while hiding float64 round-trip noise (
6.0, not5.999999999999999). - BoTorch/MOBO suggestions are 2.5–3× faster with identical candidates: the 1024 raw samples are scored 128 at a time (
init_batch_limit) instead of 5 at a time (measured: single objective 1.72 s → 0.57 s, two objectives 11.0 s → 4.4 s per suggestion). - The single-objective backends (
bo.py, DBO) run PyTorch on one thread: 1.4–1.6× faster per suggestion with identical suggestions at study sizes (measured d = 2–8, n = 12–80; e.g. DBO d=2: 2.26 s → 1.40 s,bo.pyd=8, n=80: 0.79 s → 0.55 s), and the other cores stay free for Unity. The environment variableBO_TORCH_THREADSoverrides it. MOBO and MetaTAF keep PyTorch's default, because their hypervolume computations do benefit from threads (MOBO d=2: 3.9 s on 16 threads vs 7.3 s on one). - MetaTAF records its provenance in a new
MetaRunState.json, written atomically before the first trial (and on an aborted start, withabort_reason): library versions, how openbo is installed (pip'sdirect_url.json) plus SHA-256 hashes of the openbo modules in use (open-bo keeps version 0.1.0 across behaviour changes), the exactMOTAFConfig, the frame and its digest, the seed and init configuration, and the loaded, dropped and frame-rejected sources with their provenance stamps. The runtime now pins theMOTAFConfigfields it used to leave to openbo's defaults (source_reference_mode="front",source_reference_quantile=0.9,min_informative_pairs=1, no meta-features — identical in every openbo that passes the TAF-R probe), so an upstream default change (e.g. to the newertarget_incumbentreference) cannot silently alter a running study, and the weights log reads openbo's attributes directly, so a renamed attribute fails loudly instead of logging invented weights. meta_train.pypublishes each artifact only after its self-check passes, and atomically: a failed or skipped source leaves nothing behind (the artifact used to be written before the check and stayed after a failure).- One iteration axis for observation and metric logs: every row of
BestObjectivePerEvaluation.csv/HypervolumePerEvaluation.csvnow carries the sameIterationas its evaluation's row inObservationsPerEvaluation.csv, and warm-start runs log their baseline at the iteration of the last current-context warm-start row. Under warm start, BoTorch had dropped the baseline row entirely since 1.7.1 (its metric list was one entry short) and numbered live rows 1, 2, … while its observations said k+1, k+2, …; MOBO logged the baseline as 0. Joining the files onIterationpaired metrics with the wrong evaluations. - DBO backend now runs dbo-torch 0.2.0 (vendored from M-Colley/dbo-torch
main@1a268cd, adding_base.pyandPROVENANCE.md). Changes that affect studies: each optimizer owns its random-number generator, so a run is reproducible from the inspector Seed alone and unaffected by anything else drawing from torch's global RNG; inputs are normalized against the fixed domain rather than the observed range, so fitted lengthscales keep their meaning between iterations; hyperparameters are fitted from three starting lengthscales, keeping the most likely fit;PredictedCost/PredictedSdare now predictions for the iteration at which the point is evaluated (previously the acquisition time, one iteration earlier by default); the incumbent and continuous-validation searches use BoTorch'soptimize_acqfinstead of hand-written loops. Results for the same seed differ from 0.1.0 — do not mix versions within one study. - New per-run
DboRunState.json: the optimizer's own record (dbo_torch/torch/botorch/gpytorch versions, full configuration, RNG state, every observation with its prediction), rewritten atomically after every evaluation, so each run is attributable to the library version that produced it and restorable withDynamicBO.load()for offline analysis. The dbo_torch version is also printed at startup. - DBO validation scheduling is configured up front instead of being switched on after the sampling phase; dbo-torch never schedules a validation iteration while seed points are unspent, so the schedule is unchanged.
Fixed
DBO Initial Alpha = 1silently produced a stationary run labelled as DBO. Alpha can never be fitted away from exactly 1 (its gradient vanishes there): with dbo-torch 0.1.0 the fitted alpha stayed at 1.000000 on strongly drifting data where a 0.99 start fitted 0.982, and with 0.2.0 the same setting would crash at the first model fit, after the participant had completed the sampling phase. The inspector slider now stops at 0.999 and shows an error for 1, the launcher refuses to start, and the backend rejects it at init. Pin alpha = 1 with DBO Stationary Baseline instead.- DBO
BestObjectivePerEvaluation.csvused a different iteration axis from every other backend: one row0after sampling, then optimization steps1..M, while BoTorch/MOBO/MetaTAF have logged one row per evaluation on the global index (sampling included) since 1.7.1. DBO-vs-BoTorch curves were shifted by the number of sampling iterations; DBO now writes the same rows asbo.py. - Measurements could be recorded against a design the participant never saw. Every questionnaire in the scene finds the manager and calls
OptimizationStart()when it finishes, which sent the current objective values (the scale midpoint for objectives without values) even when no evaluation was in progress — e.g. a pre-study questionnaire finished after the optimizer was ready, or a second questionnaire finished after the next design had arrived. Python stored these as real observations of a design that was never shown.OptimizationStart()now only reports an evaluation in progress and otherwise logs a warning and does nothing. - One non-numeric sub-measure (e.g. an unanswered optional item, submitted as NaN) replaced the whole objective with the scale midpoint, discarding the answered items; the objective is now the mean of its finite sub-measures (midpoint only when none is finite, as before).
- Values on a decimal bound crashed the backend. Bounds were sent as doubles widened from the inspector's floats (
0.7f→0.699999988079071) while values were sent as floats (0.7), so a value clamped to, or rated at, such a bound failed the backends' bounds check and ended the session; warm-start rows on such a bound were rejected the same way, and MetaTAF frame fingerprints never matched aframe.jsonwritten with the typed bounds. Bounds are now sent as floats (0.7), identical to the values. - Non-ASCII keys or IDs (e.g.
Größe,Jürgen) crashed the BoTorch/MOBO/MetaTAF backends at the end of the sampling phase: Python wrote the CSV logs in the Windows code page but reads them back as UTF-8. The backend now runs in Python's UTF-8 mode (PYTHONUTF8=1), and its console output is decoded as UTF-8. - Python dependency setup could hang forever ("Installing Python dependencies…") once pip wrote more to stderr than the pipe buffer holds, because stdout was read to the end before stderr; both are now drained concurrently.
- Backend tracebacks (stderr) are now written to
output.txtas well, and the process-exit log no longer dereferences a process object that shutdown has already disposed. - CABOP: a fixed Seed did not reproduce a run. The GP's hyperparameter restarts drew from numpy's global RNG; they now draw from the run's seed, and repeated runs with the same seed propose bit-identical designs.
- CABOP: the "Sobol" initial design was i.i.d. uniform — every point came from a freshly scrambled Sobol engine. The initial design is now one Sobol sequence per run (so initial designs differ from earlier versions for the same seed).
- CABOP update rule
Bothentered every unsnapped observation twice, so the GP treated noisy ratings as nearly exact; an intended design is now added only when snapping actually moved it. - CABOP: a parameter whose lower and upper bounds are equal crashed the first GP fit after the sampling phase; it is now rejected at startup. A negative Seed (allowed by Unity) crashed the backend at startup; it is now wrapped into numpy's seed range. Group cost entries whose group name differed from the parameters' group only in letter case were silently replaced by default costs — Unity matches groups case-insensitively, and the backend now does too.
- MetaTAF could still silently run as the no-transfer control.
Meta Require Sourceschecked the sources staged after frame validation, but openbo warns and skips artifacts it cannot use (an unreadable or half-synced trajectory, malformed hyperparameters, a front that never dominat...
v1.7.2
Changed
- MetaTAF
TafRweighting is now objective-wise pairwise ranking agreement (openbo's 2026-08 TAF-R rework, formulation by Yi-Chi Liao): every observation pair is scored once per objective with ranking labelsr_ijm ∈ {+1, −1, 0}(ties detected at 1e-12 tolerance), and a source's distance is its label mismatch count over the fixed denominatorM·C(n,2); a source asserting no strict order anywhere (flat surrogate) is zero-weighted outright. The former Pareto-dominance agreement discarded mutually non-dominated pairs — the common case near a Pareto front — and so starved the similarity estimate exactly where HITL studies operate; it stays selectable as the newTafRParetoinspector option (init tokentaf_r_pareto) for ablations. - The MetaTAF backend now refuses openbo installs predating that rework at startup, printing the exact upgrade command (
pip install --force-reinstall --no-deps ...; openbo kept version 0.1.0, so a plain--upgradeis a no-op). On a stale install the unchanged mode tokentaf_rwould silently compute the old dominance similarity — variant drift across participants of one study. Documented in README 9 (troubleshooting), the student guide, andrequirements.txt.
Documentation
- Student guide: the new TAF-R formulation in the settings table and the paper-notation method summary, the
TafRParetoablation, the openbo upgrade path, and a note on tie semantics — with quantized ratings (e.g. Likert scales) target ties are common and shrink all sources' absolute weights by roughly the tie fraction (ordering between sources is preserved; mind this when tighteningMeta Rho).
Tests
- 2 new MetaTAF runtime tests (
taf_r_paretoaccepted and forwarded verbatim toMOTAFConfig; stale openbo rejected with the upgrade hint, distinct from the missing-package error; 198 total). On the openbo side the rework ships with regression tests for the reviewer's worked example (d_s = 1/2), tie semantics, the trade-off-curve edge case (dominance variant returns the null similarity result, ranking variant separates a copy from an inverted source), and a brute-force EHVI equivalence check (exp(qLogEHVI)vsHV(front ∪ {μ(x)}) − HV(front), ~2e-7 relative error at M=2, ~7e-5 at M=3).
v1.7.1
v1.7.0
Added
- DBO backend (Dynamic Bayesian Optimization) for single-objective studies whose cost drifts during the session (participant adaptation, learning, fatigue). The GP covariance is multiplied by a temporal decay
alpha^|t-t'|with the decay rate fitted by marginal likelihood, so the optimizer infers how fast the participant is changing; aDBO Stationary Baselinetoggle pinsalpha = 1for the matched plain-BO control condition, and optional validation iterations apply the model's best estimate on a schedule so optimizers can be compared without their exploration policies confounding the result. NewBackend -> DBOinspector option with a dedicated settings section,dbo.py/dbo_runtime.pybackend speaking the standard socket protocol and writing the standard CSVs, plusDboDiagnosticsPerEvaluation.csvwith the fitted alpha per iteration. Implementation is the BSD-3-Clause dbo-torch package (validated against the reference MATLAB DBO to ~1e-14), vendored underBOData/BayesianOptimization/dbo_torch/. Method: Kim & Sergi, Comput. Methods Biomech. Biomed. Eng. 2025 (doi 10.1080/10255842.2025.2595150) and IEEE RA-L 2026 (doi 10.1109/LRA.2026.3665072). Seedocs/dbo-backend.md; verify withpython tests/dbo_protocol_check.py(manual, not part of unittest discovery).
v1.6.1
Changed
- MetaTAF fails fast when no population model survives validation instead of silently running plain qLogNEHVI, which would turn a MetaTAF study condition into the no-transfer control. New
Meta Require Sourcesinspector toggle (default ON,metaRequireSourcesinit field); the startup error lists why each candidate source was rejected. Disable the toggle only for intentionally source-less runs. meta_train.pynow requires--source-type {human,llm-persona,synthetic}and--y-calibration {measured,generated}: provenance is caller-declared instead of hardcodedhuman/measured, which mislabeled model-generated (e.g. LLM-persona) sources in every artifact and downstreamMetaSourcesUsed/audit trail.meta_train.pyfits each objective with 8 deterministic restarts and keeps the best fit, scored against the Standardize-transformedgp.train_targets(the quantityfit_gpytorch_mllmaximizes) — guarding against degenerate "collapse to noise" fits on small noisy runs. Scoring restarts against raw targets would be systematically biased toward exactly those degenerate fits.
Fixed
- Population-model artifacts now export the fitted GP's constant mean (
mean_constant) and the openbo loader replays it, making the export->replay round trip bit-exact (previously the rebuilt model's mean defaulted to 0 in standardized space and deviated in data-sparse regions). Needs the matching openbo update; older artifacts still load unchanged (constant 0). Regenerate artifacts to benefit. - A source whose Pareto front never strictly dominates the reference point
[-1]^M(an objective stuck at its worst bound) is refused bymeta_train.pyup front, and the openbo optimizer warn-skips such a source at construction instead of aborting the participant session.
Tests
- 2 new MetaTAF runtime tests (fail-fast on zero and on all-rejected sources; 195 total), plus 2 new tests in openbo (mean-constant replay, warn-skip at optimizer construction; 40 total there).
v1.6.0
Meta-BO backend (MetaTAF, TAF-EHVI)
- New
MetaTAFoptimizer backend (meta_mobo.py/meta_mobo_runtime.py): multi-objective Meta-Bayesian optimization that blends the current user's BoTorchqLogNEHVImodel with hypervolume-improvement terms from population models built from prior participants' runs (Transfer Acquisition Function lifted to Pareto/hypervolume optimization, following Wistuba et al. 2018 and Liao et al. CHI 2024, including their population-weight decayγ(t)). Population models that disagree with the current user's observed Pareto rankings are automatically down-weighted; with zero valid sources the acquisition degenerates exactly to plain multi-objective BO. - The multi-objective optimizers (
mobo_botorch,mobo_taf,acquisition/taf_mo_ehvi) live in the openbo package (MIT, © Yi-Chi Liao; fork with correctness fixes and the new multi-objective code) — an optional dependency installed only for this backend; the runtime fails fast with the install command when missing. Not added to the auto-installedrequirements.txt(needs git; documented in README 8.14 anddocs/meta-taf-student-guide.md). - New offline tool
meta_train.pyconverts completed runs'ObservationsPerEvaluation.csvinto population-model artifacts (per-objective GP fits, normalized into the canonical frame, self-checked by reloading through the runtime's own loader). - Every artifact carries the study frame (parameter/objective names, bounds, minimize flags); the runtime skips sources whose frame differs from the live study, field by field — transferring across mismatched bounds or a flipped minimize flag would silently import a rescaled or inverted response surface.
- New outputs per run:
MetaWeightsPerEvaluation.csv(target weight, decay factor, and per-source weights per iteration) andMetaSourcesUsed/(audit copy of the exact population models the run used). - Unity integration:
OptimizerBackend.MetaTAF, inspector section (source dir, weight mode TafR/TafM, rho, target weight, warmup, decay d1/d2), init-message fields, and fail-fast validation (multi-objective only; Warm Start and Contextual Optimization are rejected with actionable errors). - Hardening: the backend isolates PyTorch's JIT extension cache per process (
TORCH_EXTENSIONS_DIR), preventing an infinite startup hang caused by stale lock files after a killed run.
Shared frame modules
- Added
bo_normalize.py: the canonical frame (parameters in[0,1]^d, objectives in[-1,1]maximization with the minimize-flag sign flip) extracted into one shared, dependency-light module.bo.pyandmobo.pynow delegate to it instead of each carrying their own copy; behaviour is unchanged and the objective format is an explicit argument rather than a module global. - Added
meta_fingerprint.py: canonical frame fingerprint + field-by-field difference reporting (used bymeta_train.pyand the MetaTAF runtime).
Fixed
- The contextual-optimization guards in
PythonStarter,SocketNetwork, and the inspector now check for "backend is not BoTorch" instead of "backend is CABOP", so newly added backends cannot silently bypass the LCE-M restriction.
Tests
- 48 new tests (193 total): shared-frame parity against
mobo.py's live transform, frame-compatibility detection, and a full protocol/CSV test of the MetaTAF runtime against a stubbed openbo (CI-safe: numpy+pandas only). The contextual/embedding suites are unchanged and pass.
Documentation
- README 8.14 (Meta-BO overview + troubleshooting rows) and
docs/meta-taf-student-guide.md(student walkthrough: population-model workflow, settings, logged outputs, study-design guidance, citations).
v1.5.0
Contextual Optimization (LCE-M GP)
- Added contextual multi-task optimization built on BoTorch's
LCEMGP(Feng et al., NeurIPS 2020) for both the single-objective (bo.py) and multi-objective (mobo.py) BoTorch backends. - Context embeddings are definable: learned from data, supplied manually per context (any encoder, e.g. ViT-G/14 vectors), or computed from context images via open_clip (default
ViT-bigG-14, the open_clip release of ViT-G/14) with content-hashed caching and optional L2 normalization. - Warm-start parameter CSVs accept a
Contextcolumn to transfer observations from other contexts (users, devices, environments); new observations are tagged with the current context. - Run metrics (
coverage,IsBest/IsPareto, hypervolume/best-objective traces) and the loggedIterationindex are computed over current-context observations only;ObservationsPerEvaluation.csvgains aContextcolumn. - New
BoForUnityManagerinspector section with live validation and fail-fast startup checks; contextual mode is BoTorch-only (CABOP rejects it with a clear error). - Worked around a BoTorch
LCEMGPtask-kernel dimensionality issue whencontext_emb_featureis provided.
CABOP Fixes
- Fixed a critical parameter-ordering bug: with multiple CABOP groups whose parameters interleave in declaration order, parameter values were silently assigned to the wrong parameter names (in Unity payloads, observation logs, and warm-start data). Vectors now always follow parameter declaration order.
- Fixed a spurious
AssertionErrorwhen the acquisition optimizer landed exactly on a parameter bound (floating-point overshoot); proposals are now clamped to bounds. IsBest/IsParetomarker flags are now derived from full-precision scalarized values instead of the rounded CSV values, and no longer re-scan the whole CSV every iteration.- Zero-configured costs and degenerate GP predictions no longer produce division-by-zero in the acquisition function.
- The CABOP runtime now tolerates malformed/unrelated protocol messages the same way as the BoTorch backends.
Runtime and Tooling
- The optimizer's Python process now runs with
PYTHONDONTWRITEBYTECODE=1, keeping__pycache__folders out ofStreamingAssets. - Fixed a pandas dtype issue when writing
IsBestflags duringbo.pysampling runs. - Context image paths configured on Windows now also resolve on macOS/Linux.
MainThreadDispatcherno longer holds its queue lock while running actions, and one failing action can no longer abort the rest of the frame's queue.- Removed the dead
Optimizer.UpdateParameterAPI (it referenced CSV data that was never loaded). - Added example warm-start CSVs with a
Contextcolumn (ExampleContextInitData*.csv); documented the optionalopen_clip_torch/pillowdependencies inrequirements.txt. - Added a manually triggered Release workflow that tests, rolls this changelog, bumps
bundleVersion, and creates the GitHub release.
Tests
- Added unit tests for the context protocol and embedding pipeline, plus real-BoTorch integration tests running full contextual BO/MOBO loops (skipped automatically on CI environments without torch).
- Added CABOP backend tests, including a multi-group ordering regression test and bounds/zero-cost edge cases (skipped when scipy/scikit-learn/loguru are unavailable).
- Added Unity EditMode tests for final-design selection and objective key matching.
- Consolidated the duplicated torch/botorch test stubs into a shared
tests/_stubs.py. - Added a weekly/manual
full-stack-testsCI job that runs the complete suite against the real pinned dependency stack (CPU torch).
Documentation
- New README section 8.13 on contextual optimization and context embeddings (incl. ViT-G/14 guidance), new troubleshooting entries, and a note on
Iterationnumbering semantics for warm-started and contextual runs.