5.1.0

What's Changed

• Prepare development for 5.1.0-SNAPSHOT by @robzor92 in #1002
• [FSTORE-2033] Support secondary index create for online tables by @dhananjay-mk in #975
• [FSTORE-2044] add iceberg support by @bubriks in #1001
• [FSTORE-1970] Upgrade opensearch to 2.19.5 by @jimdowling in #951
• [FSTORE-1938] Support chaining of Transformation Functions using a DAG by @manu-sj in #785
• [FSTORE-1970]Append. Opensearch and fg sharing fixes by @jimdowling in #1005
• [HWORKS-2869] Update Hopsworks app skills for new routing model by @gibchikafa in #1004
• Add --verify/--no-verify flag to the hops CLI and fix external Trino TLS by @wolffbe in #1007
• [FSTORE-1412] Feature monitoring improvements by @javierdlrm in #980
• [FSTORE-1970][APPEND] Fix similarity-search find_neighbors on OpenSearch 2.19.5 (k-too-large parsing + faiss efficient filtering) by @robzor92 in #1008
• [HWORKS-2802 / -2807] partitioned_by — Python client grain materialization (Delta/Iceberg/Hudi) + predicate translator + MCP/CLI hardening by @jimdowling in #961
• [FSTORE-1412][Append] Add std_dev alias for transformation functions by @javierdlrm in #1011
• [HWORKS-2802][Append] Fix bad Hudi dependency by @jimdowling in #1016
• [HWORKS-2871] Support git backed agent deployments and cost configuration for agent deployments by @gibchikafa in #1020
• [HWORKS-494] hsml: auto-upload local file paths on model.deploy() by @javierdlrm in #1015
• Add hops alert CLI command for SDK parity by @wolffbe in #1009
• [FSTORE-2051] get_tag function for feature_groups, feature_views and training_datasets throws a TypeError by @manu-sj in #1022
• [HWORKS-494][append] hsml: skip serving-file resolution on deployment update by @javierdlrm in #1024
• [HWORKS-2908] Stop requesting the unused HIVE service on project creation by @maismail in #1025
• [FSTORE-2047] AWS Glue connector by @bubriks in #1014
• [HWORKS-2920] Don't reseed the global random module on import by @robzor92 in #1026
• [HWORKS-2921] Honor training_dataset.seed in Python engine random split by @robzor92 in #1027
• [HWORKS-2789] Remove the Brewer MCP tool from the SDK by @robzor92 in #1030
• [HWORKS-2932] hopsworks-api Python client: pyjks cannot read JDK 21 k… by @ErmiasG in #1037
• [HWORKS-2878] Google sheets connector by @gibchikafa in #1028
• [FSTORE-2064] Fix failing data_source workflow tests by @bubriks in #1045
• [FSTORE-2049] Add support for Mandatory Tags in Models and Deployments by @manu-sj in #1031
• [FSTORE-2062] Fix TD external sink by @bubriks in #1053
• [FSTORE-2061] Millisecond timestamp for vector embeddings not handled correctly by @dhananjay-mk in #1051
• [FSTORE-2055] incremental training datasets by @bubriks in #1038
• [HWORKS-2922] Support multi-table DLT ingestion jobs by @gibchikafa in #1050
• [HWORKS-2878][APPEND] Fix fetch data for google sheets data source by @gibchikafa in #1061
• [HWORKS-2922][APPEND] Fix multi-table sink job target updates by @gibchikafa in #1062
• [HWORKS-2807] Native Partitioning, Liquid Clustering, Z-Ordering by @jimdowling in #1060
• [HWORKS-2807] Append fix spark unit tests by @jimdowling in #1064
• [HWORKS-2928] Add git_auto_redeploy to the Python client by @gibchikafa in #1065
• [FSTORE-2066] Fixes for stats computation / feature monitoring by @javierdlrm in #1054
• [FSTORE-1992][APPEND] hanle ignored status better by @bubriks in #1072
• Revert "[FSTORE-2055] incremental training datasets (#1038)" by @bubriks in #1076
• [HWORKS-1840] Upgrade KServe to v0.19.0 by @javierdlrm in #1019
• [HWORKS-2988] apt-get upgrade -y in the image build by @robzor92 in #1078
• [FSTORE-2068] Delete online rows on remove_rows (deprecate commit_delete_record) by @dhananjay-mk in #1069
• [FSTORE-2077] API managed users and sharing by @bubriks in #1081
• [HOPSFS-392] Bump HopsFS client to 3.4.3.2-EE-RC2 by @robzor92 in #1093
• [HOPSFS-392][APPEND] Use HopsFS client 3.4.3.1-EE-RC4 by @robzor92 in #1094
• Pin maven-scala-plugin so the build does not depend on Central metadata by @jimdowling in #1097
• [FSTORE-2073] Stop UDF source extraction carrying the enclosing scope by @jimdowling in #1095
• [FSTORE-2084] improve user access to online ingestion info by @bubriks in #1091
• [HOPSFS-388] Bump Spark to 3.5.5.2 and Hudi to 1.0.2.2 by @robzor92 in #1096
• [FSTORE-2068][APPEND] Document the storage argument in the hops-fg skill by @dhananjay-mk in #1100
• [FSTORE-2085] Fully utilize kafka storage header by @bubriks in #1099
• [FSTORE-2072][FSTORE-2073] Loadtests: Coalesce docs and statistics-gated training-dataset read-back by @jimdowling in #1075
• [FSTORE-2091] Say what the UDF source-extraction fix is proven to do by @jimdowling in #1101
• [HWORKS-2991] Make hops env install upload a local requirements file by @jimdowling in #1086
• [HWORKS-3046] Document the terminal's namespace-scoped kubectl access as a skill by @jimdowling in #1103
• [HWORKS-3009] Teach the environments skill to install npm packages by @jimdowling in #1088
• [HWORKS-2845][APPEND] Drop --system from the documented custom-app entrypoint by @robzor92 in #1106
• [HWORKS-3054] Project.project_namespace always returns None by @robzor92 in #1109
• [FSTORE-2086] Add support for disabling Uploads to Hopsworks by @manu-sj in #1105
• [HWORKS-3055] Inference requests are addressed by project name instead of project namespace by @robzor92 in #1110
• [HWORKS-3045] Create a Trino catalog from a data source by @jimdowling in #1113
• [FSTORE-2096] Attachment timestamps, the keyword API, and tags for jobs, apps and deployments by @jimdowling in #1104
• [HWORKS-2879][5.1] Backport the Spark 4.1 upgrade and its appends by @robzor92 in #1144
• [HWORKS-3074][FSTORE-2105][FSTORE-2109][5.1] Backport the great_expectations import fixes, the empty-KLL guard and the training dataset HybridRowQueue fix by @robzor92 in #1168
• [HWORKS-3260][5.1] Bump hopsworks_aiomysql to 0.2.3 by @robzor92 in #1180
• [HWORKS-2879][APPEND][5.1] Register Hudi's Spark session extension in Spark Connect sessions by @robzor92 in #1177
• [HWORKS-3234][5.1] Make spark.driver.memoryOverheadFactor and spark.executor.memoryOverheadFactor configurable for jobs, Jupyter and project defaults (UI, REST, Python client) (#1169) by @manu-sj in #1189
• [HWORKS-3217][5.1] Serve agent skills to every coding agent and keep their state across terminal sessions (#1150) by @robzor92 in #1197
• [HWORKS-3071][5.1] Python client waits forever on a library or environment command that never reaches a final state (#1118) by @robzor92 in #1200
• [FSTORE-2092][5.1] Fix feature logging and feature monitoring on feature views with model-dependent transformations (#1089) by @robzor92 in #1201
• [FSTORE-2111][5.1] Model monitoring fan-out includes logging metadata columns and the FM job fails against a training-dataset reference (#1157) by @robzor92 in #1202
• [FSTORE-2101][5.1] SnowflakeConnector.connector_options() does not support key pair authentication (#1134) by @robzor92 in #1205
• Release 5.1.0 by @robzor92 in #1206

New Contributors

• @wolffbe made their first contribution in #1007

Full Changelog: 5.0.0...5.1.0