v0.9.7

• Kayra: the sidebar header shows the logo with a short "Kayra by Skyulf"; the separate
  "by Skyulf" line at the bottom is gone. The loading screen plays the new logo animation
  (the still logo when reduced motion is on), and the rebuilt app shows the updated logo
  as the browser icon and in the sidebar.
• Documentation is built with Zensical instead of MkDocs; page addresses are unchanged.
  The menu is organized by product: Home, Kayra App, skyulf-core, "Databricks, MLflow &
  Spark", API Reference and Development. Pages have a site description for search
  engines, and the Segmentation guide is now in the menu.
• Generated Databricks projects pin skyulf-core[mlflow]==0.9.7.
• Serving: when an endpoint update fails (UPDATE_FAILED / UPDATE_CANCELED), the
  error now includes each served entity's deployment message and the last 50 lines
  of its build and server logs. Only the endpoint owner (the job identity) and admins
  can open these logs, so the job prints them. The serving_endpoint job and the
  daily rollout both do this; a problem reading the logs never hides the original
  error.
• Serving: config/serving.yml permissions accepts level: CAN_VIEW entries. They
  get endpoint CAN_VIEW right after the endpoint is created or updated, before the
  readiness wait, so a team can see a deployment that fails. CAN_QUERY and SQL
  EXECUTE are still granted only after the endpoint is verified; viewers never get
  EXECUTE. Listing one principal at both levels is rejected.
• Failed job tasks print a Fix: and Docs: line for known errors: a failed serving
  update, an owned endpoint without exactly one served entity, a champion from an
  older Core release, an invalid on_drift, a performance policy without a label
  table, an uninitialized monitoring store, Streaming Table operations and Unity
  Catalog permission errors. The troubleshooting page has a section for each.
• smoke.py rejects an on_drift value other than disabled or retrain in
  deployment/variables.yml or any target in deployment/targets.yml, before deploy.
• GUIDE.md: a "Failures and upgrades" row for a champion trained on an older Core
  release, and how to install Core from a wheel in a Unity Catalog Volume for every
  job. Regenerate or copy the new rows into existing projects if you want them.
• Training reads a Streaming Table, Materialized View or view directly as
  training_table (and as the label table joined into training rows). No features-job
  copy and no extra Unity Catalog table is needed. Each run reads the source once and
  saves the rows with its MLflow run; registration, comparison, approval, monitoring
  references, retraining checks and model-set components replay those saved rows, for
  Delta sources too, so VACUUM no longer breaks an old model's replay (deleting its
  MLflow run does). On dedicated clusters a refused history read
  (STREAMING_TABLE_OPERATION_NOT_ALLOWED) falls back to a plain read. Models trained
  before this release keep reading their Delta version. feature_lookup still needs a
  Delta training table and says so. preview.py --action sources accepts these sources
  for training.
• Model-set approval works when score_source_table is a materialized view, a view or
  a Streaming Table on dedicated compute: the approval probe reads it plainly instead of
  pinning a Delta version it does not have. Other history errors still fail.
• Endpoint monitoring for a model set joins delayed labels on the set's record keys
  when the label table has no databricks_request_id/request_row_index columns, so a
  table that already holds keys and targets (for example a silver Streaming Table) is
  enough. The latest request per key counts; duplicate_key_rows and
  unkeyed_requests are reported, and drift still uses every request. Label tables may
  be Streaming Tables, materialized views or views (read without a version pin), and the
  availability column may be a DATE (00:00 UTC). A DATE label key matches a
  yyyy-MM-dd request value. Switching a monitor to a record-key table starts a new
  performance streak.
• Single-model endpoints can join delayed labels on the training record keys too. The
  Skyulf SQL function takes the keys as optional arguments (DEFAULT NULL, sent as
  text; the model ignores them), REST callers may add them as extra fields, and
  requests without keys count as unkeyed_requests. The first serving run after the
  upgrade replaces an existing single-model SQL function once.
• serving_endpoint no longer fails when the label table does not exist yet: it
  enrolls with a warning, performance reports awaiting_labels and drift runs. Rollout
  holds instead of failing.
• baseline.model_version is optional for kind: training_holdout (default: the
  enrolled version); generated policies leave it out.
• When initialize_run fails, the training report no longer adds a second, misleading
  failure (Failed to resolve references: ... tasks.initialize_run.values.tracking_uri).
  The report reads the run reference at run time and says that initialize_run did not
  finish, so its own error is the one to read. Redeploy to get the new job definition;
  jobs deployed earlier keep working.
• Clean break: monitoring rebuilds each model's drift and retraining reference from the
  training rows saved in its MLflow run (checked by SHA-256). No monitor_ref_* or
  monitor_src_* tables are created or read any more, so the scheduled
  monitoring_cleanup job, its variables and the cleanup_references action are
  removed. Models trained before 0.9.7 stop with a message to retrain; regenerate older
  projects and drop their old reference tables once with the SQL in the monitoring
  setup guide. A monitoring job that runs as another identity needs read access to the
  training experiment.
• Personal *_development targets get a cleanup_dev job (shared targets never do). Run it
  with dry_run=true, review the list, then dry_run=false, then bundle destroy. It
  removes only what that deployment created: the owner-tagged serving endpoint and SQL
  function, this deployment's monitoring rows in the development store, its registered
  models, prediction tables, views and generations, and the endpoint payload logs. Names
  come from the project's own configuration and must carry the target's suffix; source,
  label and training tables are never touched. A rerun finishes a partial run.
• GUIDE.md has a "Permissions and identities" section listing every place access is
  set (target permissions, run_as, job ACL variables, Unity Catalog grants, the MLflow
  experiment, serving permissions and function_permissions). It warns that the
  experiment now holds training rows, so its read access should match the source tables.
• New projects start with max_rows: 1000000 and max_input_mb: 512 (were 10000 and 64;
  quick setup never asked, so any table above 10,000 rows stopped training). GUIDE.md
  explains that max_input_mb is pandas memory, not file size, and to keep it near
  1/8-1/10 of the driver memory. Existing projects keep their values.
• Endpoint monitoring works on endpoints created by current Databricks: their payload view
  reads request_date and request_time from the log table's date and time columns
  instead of attributes, which the view check used to reject ("Serving view does not
  match the canonical telemetry projection/filter"). Both view shapes are accepted, and
  the logs are read with the projection the view actually uses.
• The GitHub repository is now flyingriverhorse/Kayra and the site with its manual moves to
  kayra.skyulf.com (www.skyulf.com becomes the Skyulf company site). Links in the
  READMEs, package metadata, docs, issue templates, generated Databricks projects and the
  documentation links printed by jobs point to the new addresses; CNAME, the sitemap and
  robots.txt follow. Regenerate a Databricks project to pick up the new links.
• New landing page for Kayra at kayra.skyulf.com: a Skyulf product bar (Kayra, Koloni)
  above a sticky Kayra header, a 30-second app tour video, the leakage check shown with the
  library's real warning and the canvas run block, an evidence section that links each
  claim to its code, notebook or demo, and SEO metadata for the new address. All app
  screenshots were retaken with the current Kayra interface; 20 unused images were removed.
• The manual has 15 new draw.io diagrams, each an editable .drawio source beside its SVG:
  a system overview and the training-job lifecycle (Architecture), the app map, how the
  canvas runs pipelines, multi-path jobs and the exported model project (Kayra App), the
  SkyulfPipeline life cycle, tuning, automated EDA, serialization and drift decisions
  (skyulf-core), and the bundle jobs, model lifecycle, monitoring loop and serving rollout
  (Databricks). The project export quickstart shows current screenshots.
• Access groups per environment: each shared target in deployment/targets.yml sets
  access_viewers (CAN_VIEW on jobs, dashboard, project folder/experiment and the endpoint),
  access_runners (CAN_RUN), access_serving_callers (endpoint CAN_QUERY and SQL function
  EXECUTE when config/serving.yml lists no permissions) and access_data_readers (SELECT
  on prediction tables and views, granted after every score; a refused grant fails the task
  and names the group). Setup asks for them after one opt-in question; quick setup leaves
  them blank. Replaces the test/syst/prod_view_group answers. Endpoint callers also get
  CAN_VIEW right after the endpoint is first created.
• Clean break for generated Databricks projects: only job-graph contract 3 is accepted;
  older graphs stop with "regenerate the project with Skyulf 0.9.7 and redeploy the
  Bundle". Every training run saves the rows it reads, and registration, comparison,
  approval and monitoring (pandas and Spark) replay only those rows; a model without them
  stops with one message asking to retrain. The grouped train_register/compare_decide
  phases, graph-2 task states, report widget fallbacks and the drift_report notebook
  entry point were removed. Upgrade: regenerate, redeploy and retrain.
• DEMO_MODE=true turns an install into the public demo (default off): only the Iris
  dataset is shown and seeded at startup, uploads, new sources and deletes return 403, and
  the canvas hides the upload controls. The demo now runs from master with this setting;
  the cherry-pick promotion to deploy/demo-mode was removed.