Repository navigation
v0.9.4
-
Configure scoring eligibility, prediction bands and combined model outputs in
config/scoring.yml, with saved custom Python helpers where needed. New Bundle
projects use YAML; existing Python-only policies keep their behavior. -
Warn about the new learned preprocessing steps before a train/test split even
while the Canvas registry request is pending or unavailable. -
Reject invalid serving-monitor enrollment batches before acquiring the shared
reference lock, so configuration errors cannot write a lock record or be
hidden by an unavailable monitoring store. Valid enrollments remain guarded. -
Add FrequencyEncoder, RareCategoryGrouping, QuantileTransformer, Normalizer,
IsolationForest and LocalOutlierFactor with saved-state checks, Canvas controls
and weighted-pipeline row alignment. Joint outlier inference preserves request
rows with exclusion reasons; LOF distinguishes training from novelty scoring. -
Add fit-time regex/dtype/exclusion column selection, hierarchical multi-column
GroupImputer fallback, learned group deviations, cyclic/local-time date
features, independent Count/TF-IDF vocabularies per column, per-column
Winsorize percentiles, RFECV and sequential feature selection. Expose target
encoder CV/shuffle/seed, MinMax clipping and Robust unit-variance options. -
Reject misspelled preprocessing choices, malformed ordinal orders and invalid
hash widths instead of silently using fallbacks. Standardize new HashEncoder
defaults on eight buckets, exclude targets from automatic polynomial expansion,
and normalize missing TargetEncoder category identities for new fits. -
Validate simple-transform rule containers before reading their entries, so
malformed rules fail clearly and rejected iterators remain unconsumed. -
Keep floating sentinel-to-null replacements numeric. Interpret constant
imputation text in each selected column's numeric/Boolean type with explicit
errors for invalid or lossy values. Improve schema previews without guessing
engine- or data-dependent output types. -
Version new FeatureMath operation validation and nullable calendar outputs;
previously saved unversioned operations retain their replay behavior. Keep
existing arithmetic null/zero rules and exact integer clipping boundaries. -
Document tested built-in alternatives beside custom Bundle teaching examples,
including differences in missing values, output names and dtype handling. -
Add explicit ordered string categories and optional
column_names: slugto
OneHotEncoder, including Canvas controls. Fixed vocabularies produce stable
columns even when a training split lacks a category; legacy saved encoders
retain their original names and behavior. -
Compute context-dependent SHAP preprocessing before sampling explanation rows,
preserving lag/rolling history and exclusion counts. Row-independent pipelines
retain early sampling and the same SHAP budgets. -
Raise an explicit error when a saved
GeneralTransformationpower transform
cannot apply, instead of silently substituting raw values for the whole column.
Previously saved pipelines that relied on that fallback now fail explicitly. -
Return an unknown preview schema for
GroupImputerautomatic numeric selection,
whose selected columns depend on observed binary/constant values; explicit
column selection is unchanged. -
Raise the SHAP
explainability.max_featuresguard from 50 to 100 so wider
models, such as fixed one-hot feature sets, can be explained. -
Add optional
initial_scoring: from_now: the first batch records a Delta
checkpoint without predictions, subsequent batches score new inserts, and
model changes/recovery preserve the original boundary.full_snapshot
remains the default. Normal rebuilds reuse saved prediction keys and recent
CDF; CDF-expiry recovery requires the original starting snapshot. -
Generate
config/sources.sqlonly when setup selectsinclude_source_sql;
the source guide and read-only source checks remain available in every project. -
Keep outcome availability columns on the outcome table in the optional
config/sources.sqlstarter forseparate_outcomes, so measurement tables
do not require label metadata for single-model, competition or model-set training. -
Generate source layouts, outcome-table bindings and task-aware performance
policies from setup answers. Quick setup includes table arrangement; generated
SOURCES.md, SQL starters andpreview.py --action sourcesexplain and check
existing source contracts without creating source tables. -
Train from an optional separate outcome table using exact record keys and
pinned Delta versions/identities. Newly available labels can enter guarded
retraining even when feature rows have not changed. Saved runs retain the
original outcome binding; embedded-target configurations keep their identity. -
Expose sampling and preprocessing diagnostics in detailed setup, explain row
and byte limit failures, and warn about initial in-sample scoring and scheduled
fixed training windows. Monitoring jobs accept native email/webhook and
duration-warning settings; drift alerts remain separate SQL alerts. -
Default configured monitoring to the shared
skyulf_monitoringschema while
preserving custom destinations. Generated projects add a separate weekly
reference cleanup job with confirmation enabled and 90-day retention; personal
development targets remain paused. Manual cleanup defaults to a dry run. -
Serialize reference publication, enrollment and generated model activations
with cleanup through the existing Serializable request table. Failed protected
writes retain their owner; validation failures before a write release it.
Known contention waits within a bounded deadline. Explicit recovery verifies
job identity/termination and audits the release. All writers sharing a store
must participate or be paused. -
Write format-2 monitoring references as one population with training membership
plus a receipt; share source tables only when verified content and schema match.
Legacy references remain readable, and existing tables are not migrated. -
Monitoring can treat per-column missing-value markers, such as a
-200
sensor reading, as missing: addmissing_markersto
monitoring_drift_thresholds. Markers compare exactly in the column's own
integer or floating-point type. Reports without the key are unchanged. -
Add a single-table air-quality walkthrough: train on summer observations,
append winter days with separate outcomes, and score new rows with Spark.
Serial loader retries recover missing inputs or labels without duplicate
inserts. Use a shared monitoring schema for the air-quality examples and
clarify that daily performance windows follow prediction time, while
report mode records the first eligible loss without waiting for a streak. -
Pin
sqlalchemy==2.0.43in generateddeployment/requirements.txt. MLflow
3.16.1 allows anysqlalchemy<3, and 2.1.5 has no wheel for the serverless
aarch64 image, so its source build failed and stopped every job at library
installation. -
Add the prepared real-data
air_quality_no2example on UCI Air Quality:
built-in and custom pre-split filters, custom marker/category/fitted steps
with built-in IQR exclusion, Yeo-Johnson, imputation, one-hot and scaling,
combined scoring rules, Spark UDF scoring, SHAP, evaluation charts and
monitoring on a later season with a report-only WAPE policy. The real-data
loader gains the pinned archive and adatasetswidget for selective loading. -
Add unweighted WAPE alongside existing regression metrics, with an explicit
unavailable outcome for all-zero actual values. Supportheldout_wapein
model comparison/quality gates andwapein performance policies; show its
label, explanation and lower-is-better direction in Kayra. -
Configure categorical grouping, uniqueness, summary caps, PSI severity bands
and non-Spark monitoring row/byte budgets through the existing
monitoring_drift_thresholdsJSON variable. Preserve absent-key defaults and
reject invalid values before measurement; Spark and frame engines share validation. -
Measure repeated categorical drift above 50 reference values in pandas/Polars
and Spark using the top 50 reference categories plus a pooled tail. Bound
domains at 10,000 categories and skip reference uniqueness above 0.5. Preserve
existing small-domain results and add a diagnostic new-category rate on the
grouped path; statistical support is still required for drift alerts. -
Prefer
build_model_rules()in project feature packages while retaining
build_scoring()for older projects. Generated projects no longer need an
alias. Correct automatic admission of reused pre-split rules and explain
row-only replay, exclusions and admission settings in the template. -
Add the prepared
taxi_fare_robustexample: fixed eligibility, fitted IQR,
Yeo-Johnson, imputation, encoding and scaling, with Spark UDF scoring and an
explicit raw-input endpoint for REST and typed SQLai_queryrequests. -
Explain sampled rows accepted by fitted exclusion steps with SHAP, reporting
sampled, accepted and excluded counts and reasons in JSON and HTML. Fewer
than two accepted rows remains unavailable; samples are not topped up. -
Generate evaluation charts for models with excluded predictions. Preserve
the saved holdout sample, plot only its predicted rows, and disclose sample
counts in captions and task output. Invalid predicted values still fail. -
Configure new serving endpoints through
online_serving.endpointin
config/serving.yml: CPU workload type, Small/Medium/Large size, scale to
zero and a positive readiness timeout. Defaults remain CPU/Small/true,
with a 3600-second setup wait. Setup returns READY only after the exact
configuration settles; timeout leaves the endpoint available for inspection.
Existing endpoints are never overwritten. Rollout requires matching compute
settings for champion and challenger; older default plans remain readable. -
Compare floating probe outputs with fixed relative tolerance
1e-9and
absolute tolerance1e-12. Labels (including tuned labels), integer values,
schema, order, null positions and input/state mutation checks remain exact.
Version 2 evidence records this policy; regenerate version 1 evidence by
retraining and repackaging the model.
Repeating an identical request still requires exact outputs. Admission also
checks the complete scoring result, including custom outputs and model-set
composition, so tolerated intermediate noise cannot hide a changed final
decision. Each empirical step and final prediction path must actually see
at least 16 rows; an excluded sample cannot supply empty passing evidence. -
Preserve requested rows excluded by fitted IQR, ZScore, EllipticEnvelope
and ManualBounds preprocessing in saved local, Spark and endpoint scoring.
These rows receive null predictions,scoring_status="excluded"and
exclusion_reason="outlier:<step name>". Training still removes outliers;
the raw arraypipeline.predict()API retains its row-preservation guard.
Holdout reports include total, scored and excluded counts. Candidate/champion
comparison scores each full holdout once, then compares metrics on the rows
both models predicted. Independent coverage gates default to at least 90%
scored by the candidate and at most five percentage points lost versus the
champion. Setmin_scored_fractionandmax_coverage_dropin generated
training settings; either acceptsnullto disable that rule. Fresh promotion,
manual approval, model-set validation and rollout preserve these settings. -
Exclude PowerTransformer rows with invalid inputs or non-finite transformed
values. Training removes those rows with aligned targets and weights and
reports the counts; saved scoring preserves them with null predictions and
exclusion_reason="invalid:<step name>". Valid neighbours keep their fitted
transformation. Missing NaN/null values pass through for a following imputer;
they are not invalid-input exclusions. Training stops if invalid rows exceed
10% of that step's training input; exactly 10% remains allowed. Missing values
do not contribute to the invalid count. Use Yeo-Johnson for values containing
zeros or negatives. All-invalid training and invalid learned scaler state raise
clear errors. Existing Delta prediction tables must match the added status
columns; use a new or explicitly migrated table rather than implicit schema
evolution. Monitoring exposes total and invalid-input exclusion rates in the
existing data quality table, while performance uses predicted rows only.
Deterministic IterativeImputer replay
now runs its saved rounds for all-missing requests too; posterior sampling
and unsupported estimators remain excluded from partition admission. -
Package the exact XGBoost/LightGBM versions retained by fitted models,
including ensemble members and saved preprocessing state. Reuse the optional
dependency checks before unpickling; unused native packages are not required.
Re-export older native artifacts that lack these pins. Honor the existing
Python major/minor compatibility policy when rechecking a saved manifest,
allowing patch-version differences while rejecting changed dependency pins. -
Bind XGBoost and LightGBM semantic fingerprints to their serialized learned
trees, including LightGBM's prediction iteration limit. Reject unknown ctypes
state and native booster overrides. Earlier native-model fingerprints and
partition evidence must be regenerated and models packaged again; old evidence
is not silently accepted under the corrected format. -
Admit additional exact built-in models and voting/stacking on Spark workers
and serving endpoints when training seals their partition probe evidence.
Check nested estimator identities, saved state, predictions, probabilities
and configured thresholds. Eligible row-preserving preprocessing recipes
and four lightweight text steps use the same bounded evidence mechanism;
unsupported row filters and global/window transformations remain rejected.
Missing or changed evidence requires retraining. Built-in pipelines do not
need a project source directory. Validated empty saved-model requests return
the declared empty schema without invoking the estimator. -
Admit pandas
SimpleImputer(median)andGroupImputer(median)on the saved
pipeline worker path using their learned values. Native Spark/portable
imputer codecs keep their existing supported strategies. -
Add two prepared real-data projects: a Wisconsin Diagnostic classifier and a
Bike Sharing model set with separate casual and registered demand predictions.
Both demonstrate custom preprocessing, Spark UDF scoring, serving endpoints,
SHAP, dataset Inputs and monitoring. The examples include data loaders and
Spark/REST parity checks and are regenerated from the Bundle template. -
Preserve temporal and group CV metadata during custom-step admission, while
rejecting altered metadata-removal records. Local admission probes now use
the workflow's CV settings, and model-set packaging inherits its custom-step
admission setting. -
Plot nested CV's saved outer-fold scores and keep custom outputs such as
probability_bandseparate from class-probability columns in monitoring. -
Let Spark scoring and endpoints run your own
column_step/fitted_step
steps and saved eligibility and output scoring rules. Projects with
inference_mode: sparkoronline_featurescheck them automatically during
training; other projects setadmit_probed_custom_steps: true. Training lints
the source for whole-batch calls and writes to shared process state, then
probes each step and rule on 16 to 256 holdout rows (whole, repeated, chunks
of 1, 2 and 7, reversed, renumbered index, empty, one row's inputs changed,
and a freshly loaded copy of the project code) and saves passing evidence
inside the model. Model-set combined function rules from
build_combined_rules()are probed the same way when the set is packaged and
admitted with their sealed evidence. Reused pre-split eligibility
(SCORING_MODEpre_splitorcombined) is admitted when its steps are
row-local and probed;Deduplicateand custom filter classes in the reuse stay
rejected.inference_context="row"is optional;group,windowand
globalsteps stay rejected. Spark output reserves the text␀null␀for
nulls and rejects a real value equal to it. A failure stops training before registration and
names the step or rule, the check, the file and line for lint findings and
one fix.preview.py --action trainruns the lint offline and
smoke.py --probe <sample>runs the probe on a local sample. Spark output now
carries rule nulls (exclusion_reason, excluded predictions) to the driver.
The Spark pyfunc parameters are renamedspark_outputandspark_batch_rows
(formerlyskyulf_spark_outputandskyulf_spark_batch_rows). Packages logged
by an earlier release already fail the runtime source check, so log them again.
See the
opt-in section. -
Preserve whitespace inside MLflow alias-recovery evidence while keeping each
Unity Catalog tag free of boundary whitespace. Read existing intent records;
upgrade lifecycle jobs sharing a model together because older readers cannot
recover the new framed records. Run recovery with this release as well. The
framing reduces the bounded intent capacity from 12,288 to 12,192 characters. -
Start the final training report with a Where to look table: one row per
fitted model linking its MLflow run, registered version, training data counts,
preprocessing diagnostics, SHAP report, evaluation charts and saved decision
thresholds. It reads saved evidence only and never blocks the lifecycle result. -
Show model-set approve, reject and rollback values as the same copyable
Available action and If rollback is needed tables as single models. -
Ask for the monitoring store catalog and schema in quick and detailed setup.
Blank answers generatemonitoring_enabled: "false", so a new project trains
and scores before a monitoring destination exists; previously monitoring was
enabled with an empty destination and both jobs failed until it was set. -
Advertise the parameter names preprocessing nodes actually read in registry
metadata:alias_type,column_types,correlation_method,label_format
andtransformationsreplace the ignoreddomain,type_map,method,
encodeandfuncdefaults. -
Let training continue when the champion was produced by another Core release:
the comparison reportschampion_runtime_unavailable, automatic promotion holds,
and an operator can approve a reviewed candidate that passes its absolute quality
gates with the new train-job parameterallow_unloadable_champion=true.
Previously the first training after an upgrade failed and no candidate could
replace the champion. Model sets report the limitation explicitly. -
Ask for the endpoint inference-table mode and name prefix when online serving is
selected. -
Grant
CAN_MANAGEon every generated target to the deployer and a new
pipeline_ownervariable (CI/CD can pass--var pipeline_owner=<email>), and
CAN_VIEWon shared targets to an optional per-environment group. Detailed setup
asks for the owner and the test, syst and prod groups; blank answers use the
deployer and add no group. Previously every target hadpermissions: []. -
Add four prepared NYC taxi example projects (Spark-scored single model, online
single model with feature lookups, competition, model set) with a SQL loader and
a feature walkthrough undertemplates/databricks/examples/nyctaxi, checked
against the template in CI. -
The setup wizard no longer asks ensemble voting weights or per-member
voting,
cv,passthrough,calibrateandcalibration_cvsettings; member models are
still asked. The settings keep their defaults inconfig/training.yml, and answer
files can still set them. Generated projects are unchanged. -
Refuse at
bundle init, or stop offering, setup combinations that generated a
project failing later: the same ensemble base model twice; halving on
n_estimatorsfor a model without it;heldout_mape,heldout_precision_weighted
orheldout_recall_weightedin a competition; stratified CV for regression; a
full_snapshotwindow with a temporal split (now asked first, and the split is
then random); online serving withmulti_targetor Polars.search_max_resources
defaults to 100 instead of an unusablenull. -
src/tools/smoke.pynow runs the search, competition-metric and
config/serving.ymlchecks that previously failed only in preview or in the job.
preview.pypasses the CDF recovery setting for model sets and reports a broken
configuration in one line instead of a traceback. -
An unfilled generated feature lookup reports that it is not filled in, and every
invalid Unity Catalog table name gets one message naming the
catalog.schema.nameform. -
Online feature checks name what failed: the empty features, how many rows and the
first row for a missing lookup value, and the age againstmax_age_secondsfor a
stale feature. Serving endpoints show these only in the served model's logs, because
the native Feature Engineering wrapper returns a generic error to callers. -
Fix approving a candidate with
allow_unloadable_champion=trueunder automatic
promotion: the job's evidence lookup accepted onlymanual_approval. -
Add
logging_mode: disabledfor serving endpoints whose callers store their own
predictions, for example throughai_query. Rollout and serving monitoring
reject such endpoints because they read payload logs. -
Record the Bundle target that ran the daily serving rollout in its saved
observation, instead of the fixedrollout.monitoring.environment: systlabel. -
Remove
src/features/custom/from new projects. Scoring callbacks live at the
end ofsrc/features/scoring.pyand are selected asscoring.<function>; own
factories go inpreprocessing.pyor a module undersrc/features/. Existing
custom.<file>.<factory>references keep working. -
Reorganize integration documentation around an ordered Databricks project
walkthrough, with dedicated training, scoring, feature-table, troubleshooting
and glossary references. Keep generated project commands offline; shorten its
root README and separate training/scoring topics. Document every registered
preprocessing operation and accepted training YAML field, with executable
examples and CI checks for catalog coverage, diagnostics and public links.
Give model sets, monitoring, project removal, custom pre-split filters, Spark
scoring, ensembles and Spark benchmarks their own pages; split the preprocessing
catalog into family pages; group the Databricks navigation by task; add a
troubleshooting message index, glossary terms and flow diagrams. -
Keep training field tables in one reference and link the generated project
guide to it. Add model/search-space and tuning references, plus executable
custom preprocessing and pre-split examples with their YAML selectors and
scoring eligibility rules. -
Document reference preparation for a first pinned candidate's monitoring
handoff, including safe score replay after an already committed prediction. -
Add saved training candidates, selected-model parameters and dataset provenance
to Training and lineage, using the shared model/version/context filters.
Capture bounded optional summaries in new monitoring observations; preserve
missing evidence, candidate/run ownership and immutable observation identity. -
Remove the Execution and compute monitoring page and its six exclusive
datasets. The dashboard no longer queries job, compute or billing system
tables; inspect those details directly in Databricks. -
Share monitoring observation, identity and serving-window SQL through three
owned views. Preserve dashboard metrics, filters and chart presentation. Store
owners must initialize the updated views and grant viewer access before
upgrading producers and publishing the dashboard. -
Fix clipped monitoring page titles and pin daily drift/performance problem
series to red. Render serving error classes and execution times as separate,
labelled bars while preserving missing measurements. Show PSI limit crossings
in a heatmap with distinct identity labels; exact PSI, limits and ratios remain
available in tooltips and the existing detail table. -
Batch Spark drift population checks, finite counts, minima and centered
reference standard deviations across features. Preserve numeric-string
conversion, missing-data precedence and native integer precision; exact CDF
and bounded histogram calculations remain per feature. -
Batch serving-payload validation into one Spark aggregate per phase, including
typed input and output columns, instead of scanning once per check or feature.
Preserve validation order, error messages, empty-window behavior and null values. -
Apply rollout
maximum_latency_msto p99 model execution time when recorded,
falling back to maximum latency only when p99 is absent or null. This changes
the previous maximum-only policy; review configured limits before upgrading.
Invalid non-null p99 holds rollout; other admission and promotion checks remain. -
Keep serving monitoring usable when individual requests have logging errors.
Retain deduplicated HTTP counts, exclude flagged payloads from feature and
performance metrics, and show the affected request count in saved JSON and
notebook notes. Mark otherwise healthy reportsdegradedwhile preserving
measured drift and no-data status. Drift and performance on usable payloads
remain eligible for retraining under the existing data and policy checks;
performance-policy output exposes its own window's excluded request count.
Keep sampling/identity validation strict; partial capture holds live rollout.
Historical reports are unchanged; recomputed summaries receive new identities. -
Show requests with logging errors in the existing Online serving snapshot and
history tables and the captured-request chart tooltip. Preserve recorded zero
and showNot recordedfor absent counts, without changing HTTP error classes
or chart series. -
Fix two Polars 2 failures that earlier Polars versions did not show.
StandardScaler
no longer crashes on unsigned 64-bit columns with integer-typed saved means, and model
matrices mixingUInt64with signed integer columns convert to float64 as before.
Both cases used anInt128intermediate that has no NumPy conversion. -
Add explicit recovery for interrupted MLflow alias operations. Persist a
bounded intent before changes; complete fully applied roles or abort untouched
state under shared admission, without moving aliases. Refuse partial or legacy
evidence, preserve pending ownership, and verify terminal receipts before
clearing the guard. Document manual recovery and Unity Catalog tag capacity. -
Timestamp lifecycle event tags in UTC; continue reading older receipts and preserve
replay behavior. New tags require Core 0.9.4 or newer: upgrade all jobs sharing a
model's lifecycle tags together; older readers and downgrades reject timestamped
events. Reserve space for the final state before alias changes begin. -
Preserve unmeasured monitoring counts as SQL
NULLinstead of fabricated zeros.
Explicit zero counts remain zero; saved history and retry identities are unchanged. -
Advance single-model and model-set incremental watermarks through empty source
windows with an atomic zero-row Delta receipt. Preserve temporal history and
same-version retries; monitoring skips the empty batch but retains delayed-label
checks. -
Allow rollback to restore a valid prior champion when its older previous-champion
version was deleted. Clear that alias, retain the null pointer in the receipt,
and verify it on retries; permission and artifact errors still prevent changes. -
Retain explicitly excluded prediction counts in full Spark, pandas and Polars
monitoring reports and their saved JSON, separately from missing predictions. -
Retain the configured performance-policy window count in direct evaluations
and failed monitoring observations. Notebook reports mark missing historical
counts as unavailable instead of showing a misleading0 / 0failure streak.
Equivalent old/new history replays retain their streak across the added field. -
Provision the retraining-request table during central monitoring setup with
ownership/schema checks and Serializable isolation. Automatic training requests
reuse pre-provisioned storage without requiring table-creation privileges.
Before upgrading producers without creation privileges, have the store owner
reruninitialize_monitoring_storeto add the table to older monitoring stores. -
Check pending-event ownership before clearing an MLflow alias transition.
Preserve another writer's marker and report an uncertain outcome if ownership
changes or the marker disappears before completion. -
Validate both rollback destination versions before changing MLflow aliases or
receipts. A missing prior champion or unreadable existing destination leaves
registry state unchanged instead of starting a partial rollback. -
Align Spark PSI/KL with Core for numeric features with at most ten distinct
reference values. Preserve discrete levels, fractional shifts and precise
integer/float boundaries; retain quantile bins for continuous features. -
Fix incremental single-model and model-set scoring after Delta maintenance.
Read the last prediction receipt across maintenance commits, retain physical
version checks for concurrent writes, and report the actual prediction commit.
Incremental latest-version lookups request one history entry; historical receipt and
monitoring reads retain their full history. -
Breaking: Remove the 88 remaining Databricks/MLflow flat-module aliases
and both_compatpackages. Update imports and string-based patch targets to
the responsibility packages documented in the integration READMEs. Existing
package-root exports remain available. Old model pickles naming removed paths
can fail before inner runtime validation; package and validate replacement
models with current code before upgrading jobs that consume those artifacts. -
Add captured HTTP status classes, approximate latency percentiles and fixed
duration buckets to serving observations. Show error classes, tail latency and
per-window distributions in the monitoring dashboard; preserve missing legacy
evidence and existing rollout gates. Recomputed windows receive new report IDs. -
Clarify serving charts with grouped HTTP error classes, distinct legend colors,
mean/p95 latency bars, and explicit "Not recorded" labels for missing telemetry. -
Add a quick setup choice to the Databricks project wizard.
quickasks only for
the layout, project name, task, source table, record keys and target/input
columns and accepts every other documented default. Quick setup is the default;
choosedetailedto review model, search, branch and deployment settings now.
Explicit answer-file values are honored in either mode. -
Fix the interactive Databricks project wizard skipping questions. The CLI
evaluates onlyproperties,requiredandanyOfin a skip rule and silently
ignoresnotandallOf, so the wizard never asked for the branch count,
per-branch settings, competition candidates, search strategy or ensemble
members. The schema builder now rewrites every rule into the supported form
and fails on a rule the CLI cannot evaluate. Detailed setup asks the questions
it describes; quick setup also asks each target's task and columns. Lists start
with their default, so Enter selects it. Answer-file initialization is unchanged. -
Name the setting in the feature lookup table error raised for an unfilled
placeholder:feature_lookup.lookups[N].table_name.
Upgrade steps for generated Databricks projects
A generated project records its release in deployment/requirements.txt and in
the first lines of its README. Before raising that pin from 0.9.3, in order:
- Package and validate replacement models with this release before upgrading
jobs that load saved models. Models pickled against the removed flat-module
import paths can fail to load. - As the monitoring store owner, run
initialize_monitoring_storeonce. It adds
the retraining-request table and thecomplete_observations,
monitoring_identitiesandserving_windowsviews, and refreshes
model_version_detailswithtraining_summary_json. Producers without
creation privileges fail until this has run. - Grant dashboard viewers SELECT on the three new views.
- Review each rollout
maximum_latency_ms. It now limits p99 execution time
when recorded, instead of the maximum. - Raise the pin and redeploy, in one change, every job that shares a model's
lifecycle tags. Older readers and downgrades reject the new timestamped events. - The dashboard definition is a file copied into the project. To adopt the new
one, replacesrc/monitoring/monitoring.lvdash.jsonwith the file from a
project generated with this release, after step 2, then deploy. It no longer
has the Execution and compute page. - Optional, for links in the final report's Where to look table: add
workspace_host: ${workspace.host}to thetraining_reporttask parameters in
resources/train.job.yml, anddisplay_html=globals().get("displayHTML")to
the call insrc/jobs/training_report.py(models_report.pyfor model sets),
as in a newly generated project. Without them the report still runs; links
appear as plain text. - Add the
allow_unloadable_championtrain-job parameter (default empty) and pass it
to the lifecycle tasks, as in a newly generated project, so a champion from the
previous release can be replaced. - Optional: add
bundle_target: ${bundle.target}to thedaily_rollouttask
parameters inresources/daily_rollout.job.ymlso rollout evidence names the
target that ran it. - Optional, when your CI/CD passes
--var pipeline_owneror expects target
permissions: add apipeline_ownervariable todeployment/variables.ymland the
permissionslist to each target indeployment/targets.yml, as in a newly
generated project. Undefined variables passed with--varfail validation.