chore(main): release 1.38.1

[release-please]

🤖 I have created a release beep boop

1.38.1 (2024-06-12)

Features

• Add Candidate.grounding_metadata property (b22a8b8)
• Add fraction_leaf_nodes_to_search_override. Add support for private endpoint in find_neighbors. (cd31c13)
• Add numeric_filter to MatchingEngineIndexEndpoint match() and find_neighbor() private endpoint queries. (679646a)
• Add numeric_filter to MatchingEngineIndexEndpoint match() and find_neighbor() private endpoint queries. (506c91b)
• Add numeric_filter to MatchingEngineIndexEndpoint match() and find_neighbor() private endpoint queries. (4d98c55)
• Add return_full_datapoint for MatchEngineIndexEndpoint match(). (ad8d9c1)
• Add update_mask to MatchingEngineIndex upsert_datapoints() to support dynamic metadata update. (81f6a25)
• Add upsert_datapoints() to MatchingEngineIndex to support streaming update index. (7ca484d)
• Add a cloneable protocol for Reasoning Engine. (8960a80)
• Add additional parameters for GenerationConfig (0599ca1)
• Add additional parameters to Model.upload(). (7b7d7d2)
• Add an arg to turn off Ray metrics collection during cluster creation (e33d11f)
• Add Batch Delete and Batch Cancel Pipeline Jobs and unit tests. (cb495e7)
• Add batch delete method in preview pipeline job class and unit test. (b0b604e)
• Add cache module and CachedContent class (c21beef)
• Add Candidate.grounding_metadata (310ee49)
• Add custom tool context manager for telemetry (b30f5a6)
• Add display experiment button for Ipython environments (0b33bdd)
• Add display experiment button for tuning in Ipython environments (9bb687c)
• Add display experiment run button for Ipython environments (ba65828)
• Add display model evaluation button for Ipython environments (181dc7a)
• Add enable_tracing to LangchainAgent. (cad035c)
• Add explicit constraints for update_ray_cluster (979a4f3)
• Add FeatureGroup create function (3938107)
• Add FeatureGroup delete (f9011e0)
• Add FeatureGroup init/get (e47d436)
• Add FeatureNormType to MatchingEngineIndexConfig. (c0e7acc)
• Add GoogleSearchRetrieval (310ee49)
• Add hybrid search for public find_neighbors() call. (9d35617)
• Add labels parameter to the supervised tuning train method (f7c5567)
• Add mappings to pipeline templates for text-embedding models. (6279924)
• Add metadata in model.predict output for LLMs (4661e58)
• Add notification_channels field to model monitoring alert config. (bb228ce)
• Add option to not include time_series_metrics in get_experiment_df call. This will improve execution time for Experiments with large number of runs. (78a95c5)
• Add option to not overwrite table in Ray on Vertex BQ Write (a99e992)
• Add Persistent Resource ID parameter to Custom Job form_local_script, run, and submit methods. (f5be0b5)
• Add Persistent Resource Id parameter to Custom Training Job run and submit methods. (f428006)
• Add query by id for MatchingEngineIndexEndpoint find_neighbors() public endpoint query. (42c7e08)
• Add Ray on Vertex BigQuery read/write support for Ray 2.9 (e048e3a)
• Add read_bigquery and write_bigquery wrappers for Ray on Vertex (4739118)
• Add Retrieval (310ee49)
• Add safety filter levels, watermark support and person generation support for Imagen 2 (0c498c5)
• Add safety filter levels, watermark support and person generation support for Imagen 2 (e2efdbe)
• Add support for low_level_batch_size in match() for MatchingEngineIndexEndpoint private query. (67e593b)
• Add support for BaseModels in LangChain templates (5eb885e)
• Add support for output_dimensionality parameter through get_embeddings. (b1cab3f)
• Add support for reading requirements from a file. (80db7a0)
• Add support for task types (TASK_QUESTION_ANSWERING, TASK_FACT_VERIFICATION) through get_embeddings. (9ec4590)
• Add support for ToolConfig in the LangChain template (9bda328)
• Add support for TPU v5 lite pod(v5e) for custom training jobs. Custom training jobs now accept the v5e machine types as listed in https://cloud.google.com/tpu/docs/tpus-in-gke#v5e. (415912e)
• Add support for user-configurable 1P embedding models and quota for RAG (8b3beb6)
• Add support of newly added fields of ExportData API to SDK (ec3ea30)
• Add Tool.google_search_retrieval (310ee49)
• Add Tool.retrieval (310ee49)
• Add transport override to enable the use of REST instead of GRPC (6ab4084)
• Add tune_model and deploy_tuned_model for TextEmbeddingModel. (42f5d6f)
• Add update, expire_time and delete method to CachedContent (24dbdf1)
• Add v1beta1 service clients for reasoning engine (13ec7e0)
• Add VertexAiSearch (310ee49)
• Add Webpage Browser Extension to Vertex AI Extension SDK (c222f4b)
• Added GA support for PersistentResource management (98a07dd)
• Added GA support for running Custom and Hp tuning jobs on Persistent Resources (35ecbac)
• Added Public Preview support for PersistentResource management (262a36b)
• Added reboot command for PersistentResource (7785f8c)
• Added the BatchPredictionJob.submit method - a non-blocking version of BatchPredictionJob.create (b62ddcd)
• Added the Experiment.dashboard_url property (c8eec21)
• Added the vision_models.Image._mime_type property to make vision_models.Image compatible with generative_models.Image (6557d88)
• Added the new GenerationConfig.response_schema field (#3772) (5436d88)
• Adding serving_container_grpc_ports parameter to Model.upload() method (6a00ed7)
• Adding support for concurrent explanations (8e2ad75)
• Adding tpu_topology to Vertex SDK (423c764)
• Allow configuring container logging settings on models when deploying. (beae48f)
• Allow module_name to be specified with the base logger. (c01df66)
• Allow reuse of deleted experiment run id. (5f6ad8d)
• Automatically populate parents for full resource name in Vertex RAG SDK (26657ff)
• AutoSxS Pairwise Metric in Rapid Evaluation SDK (b0c5eda)
• Check if dataset exists before creation for Ray on Vertex BigQuery Write (544d6fe)
• Create Vertex Experiment when uploading Tensorboard logs (339f8b6)
• Deploy a tuned text embedding model -- it doesn't matter, if it's tuned using Node.js, or curl. (8ca9cdf)
• Enable continuous upload for profile logs. (f05924d)
• Enable inline context in grounding to TextGenerationModel predict. (a75e81c)
• Enable Ray cluster creation and registering TensorFlow checkpoint to Vertex with Ray version 2.9 (ff148cd)
• Enable Ray cluster creation with custom_image for each Resource (f90c9cc)
• Enable Ray Job submission without VPC peering (37875b5)
• Enable Tensorboard profile plugin in all regions by default. (8a4a41a)
• Fix dataset export system test (1fbf049)
• Fix dataset export system test (9156640)
• Fix typo in Vertex Feature Store SDK. (b5404e7)
• GenAI - Add BatchPredictionJob for GenAI models (df4a4f2)
• GenAI - Add cancel, delete, list methods in BatchPredictionJob (7ff8071)
• GenAI - Added function_calls shortcut property to Candidate class. (d6490ff)
• GenAI - Added response_style to GenerationConfig (cb8b10f)
• GenAI - Added response_style to GenerationConfig (7279dab)
• GenAI - Added to_dict() methods to response and content classes (a78748e)
• GenAI - Added support for "models/<model ID>" model name format (ab21feb)
• GenAI - Added support for SafetySetting.method (probability or severity) (317ab8f)
• GenAI - Added support for Grounding (0c3e294)
• GenAI - Added support for supervised fine-tuning (036d2d0)
• GenAI - Added support for system instructions (4990eb6)
• GenAI - Added the BatchPredictionJob.submit method (4d091c6)
• GenAI - Added the GenerationResponse.prompt_feedback property (efd5a72)
• GenAI - Added the GenerationResponse.usage_metadata property (0654c35)
• GenAI - Added the GenerativeModel.start_chat(response_validation: bool = True) parameter (94f7cd9)
• GenAI - Added the response_schema parameter to the GenerationConfig class (b5e2c02)
• GenAI - Allowed callable functions to return values directly in Automatic Function Calling (768af67)
• GenAI - Automatic Function Calling feature (eef84c6)
• GenAI - Explicit Caching - Allow GenerativeModel to be created with CachedContent (70168eb)
• GenAI - Explicit Caching - Allow CachedContent to take wrapped types (52d3e25)
• GenAI - ExplicitCaching - add public list() with no parameters (b7056e7)
• GenAI - Forced function calling feature (806ef9f)
• GenAI - GAPIC - Added support for Grounding (310ee49)
• GenAI - Grounding - Released Google Web Search retriever to GA (32b030a)
• GenAI - Improved the exception messages when candidates, parts or text are not available (e82264d)
• GenAI - Release the GenerativeModel to GA (c7e3f07)
• GenAI - Release ToolConfig to GA (bc8b14a)
• GenAI - Support generating JSON Schema from Python function (be4922a)
• GenAI - Tuning - Supervised - Added support for the adapter_size parameter (88188d2)
• Implement multithreaded batch inference in Rapid Evaluation SDK for performance improvement (3c6d173)
• Improve get_experiment_df execution speed (2e56acc)
• Initial template for Langchain on Vertex. (0752a29)
• LLM - Add newly supported model text-bison@002, update pipeline template version and prediction row limit for pipeline-based LLM Evaluation SDK (35e57b3)
• LLM - Add RLHF-tuning support for text-bison@002 (1f27c3e)
• LLM - Add support for batch prediction to CodeGenerationModel (code-bison) (fbf2f7c)
• LLM - Add support for RLHF tuning (b2458ec)
• LLM - Added support for model distillation (28925e9)
• LLM - Added support for the logprobs, presence_penalty, frequency_penalty, and logit_bias generation parameters (1449344)
• LLM - Added support for the max_context_length tuning parameter (e5daae9)
• LLM - Added the seed parameter to the TextGenerationModel's predict methods (cb2f4aa)
• LLM - Made the tuning location parameters truly optional (bae8429)
• LLM - Support tuning of new text embedding models by migrating to the new v1.1.3 pipeline. (7fea754)
• LLM - Text embedding - Added the output_dimensionality and learning_rate_multiplier parameters to text embedding tuning (Preview only) (cc8bc96)
• LLM - Text Embedding - Added validation for text embedding tuning parameters. (5a300c1)
• LVM - Add GCS URI support for Imagen Models (imagetext, imagegeneration) (4109ea8)
• LVM - Added support for Images from GCS uri for multimodal embeddings (90d95d7)
• LVM - Added support for Videos from GCS uri for multimodal embeddings (f3bd3bf)
• LVM - Added the MultiModalEmbeddingModel.get_embeddings(dimension=...) parameter (1d9bd23)
• Make get_embeddings work both for foundational & tuned models. (b8b589c)
• Private Endpoints - Added private service connect support to prediction endpoint. (6bdcfb3)
• Python SDK for Vertex Feature Store. (5015d25)
• Python SDK for Vertex Model Monitoring V2. (021d59f)
• Ray on Vertex enables XGBoost register model with custom version using pre-built container (e45ef96)
• Release Ray on Vertex SDK to GA (f7c5132)
• Remove deletion of model upload staging bucket in system tests (b9b373b)
• Sample code for Vertex AI Feature Store (6c14e8b)
• Support "reservedIpRanges" parameter in PipelineJob run() and submit() methods. (ab99e00)
• Support accelerator_type in RLHF tuning (e51c977)
• Support NOT_EQUAL for MatchingEngineIndexEndpoint numeric_restricts. (aa918e3)
• Support read_index_datapoints for private network. (c9f7119)
• Support shard_size for MatchingEngineIndex create index. (6dbf7d3)
• Support CMEK for scheduled pipeline jobs. (406595d)
• Support custom timeout for MatchingEngineIndex and MatchingEngineIndexEndpoint APIs. (09d1946)
• Support custom batch size for Bigframes Tensorflow (7dc8771)
• Support custom image for Ray cluster creation (d727189)
• Support custom service account for Ray cluster creation and Ray Client connection (e0c6227)
• Support custom target y column name for Bigframes Tensorflow (1634940)
• Support empty index for MatchingEngineIndex create index. (a00db07)
• Support PreflightValidation in Preview PipelineJob submit function. (e88dc0d)
• Support private service connect for MatchingEngineIndexEndpoint match() and read_index_datapoints(). (61cff4b)
• Support public endpoint for Ray Client (57a5f78)
• Support referenced models in SDK. (c9b6b8b)
• Support user provided api endpoint. (92f2b4e)
• Support vector_distance_threshold filtering and file-based retrieval for RAG (cd85d8f)
• Support VertexTool in langchain template. (28a3c56)
• Switch Python generateContent to call Unary API endpoint (9a19545)
• Tune_model method for class TextEmbeddingModel. (3eda55d)
• Update ray extras installation in setup.py for Ray 2.9.3 (d947304)
• Update the v1 service definition to add numeric_restricts. (d0c2ffa)
• Use colab enterprise enviroment variables to infer project_id and region (5baf5f8)
• Verify client and cluster Ray versions match (10c6ad2)
• Verify client and cluster Ray versions match in create_ray_cluster (17dc9b7)
• Vertex AI Extension SDK Public Preview (137b5e1)
• Vertex AI Reasoning Engine SDK Public Preview (6aaa5d0)
• Vertex RAG for enhanced generative AI (39b5149)
• Vertex Rapid Evaluation SDK and Prompt Template for Vertex Prompt Management Public Preview (8c6ddf5)
• View model evaluations when AustoSxS pipeline completes (17c59c4)

Bug Fixes

• read_index_endpoint private endpoint support. (3d8835e)
• A bug in the evaluation library where the job crashes if only custom metrics are specified. (c528b6f)
• Add deprecation warnings when using Ray v2.4 (3a36784)
• Add DeprecationWarning to vertexai.preview predictive models SDK (3c3727b)
• Add google-auth as a direct dependency (ecc6454)
• Add MAX_TOKENS to the list of successful finish reasons for Rapid Evaluation SDK (195c77e)
• Add restricts and crowding tag to MatchingEngineIndexEndpoint query response. (83cb52d)
• Add validation check for extra_packages when creating a reasoning engine. (255dabc)
• Add validation for evaluation dataset fields, update logging info for eval api request count (d6ef500)
• Add validation for langchain tools. (a821d50)
• Allow destination directory to exist before we copy. (0b55762)
• Allow non-lro delete method (c23c0ad)
• Append allowed_plugins in tb-gcp-uploader to default allowed plugins (aab9c3e)
• AttributeError for TorchModelSerializer.deserialize in torch >=2.3.0 (20b1866)
• Batch read tensorboard time series data to allow reading more than 20 time series metrics. (6e3eac2)
• Clarify wording when Ray on Vertex cluster is missing head node ip (4a71c8c)
• Create run_name when run_name_prefix is not specified for Tensorboard uploader. (ac17d87)
• Deep copy dataset before passing it to evaluation (019b610)
• Ensure model starts with publishers/ when users provide resource path from models/ (d689331)
• Fix crash when no target_col is specified for Bigframes tensorflow (512b82d)
• Fix error when allowed_plugins is set to None. (6f2860a)
• Fix example usage of batch delete pipelinejobs. (3b28d64)
• Fix exception message to use vertexai when project is not provided. (0cb1a7b)
• Fix experiments failure when backing tensorboard has been deleted. (e7a197e)
• Fix failed unit tests due to google-cloud-storage upgrade. (945b9e4)
• Fix failing get_experiments_df test. (767712e)
• Fix feature attribution drift visualization for model monitoring SDK (710f33d)
• Fix message to Json parse issue in Dataset class (066f32d)
• Fix numerical NaN experiment run logging error in EvalTask. (641faec)
• Fix the default value of response_column_name in EvalTask.evaluate() (98f9b35)
• Fix typo in raise exception when tool is unsupported (2301d79)
• Fixed INTEGER and BOOL parameters casting issue. Fix conversion bug where INTEGER and BOOL parameters are not cast to the correct type. (9a204c4)
• Fixed the PipelineJob bucket creation after a breaking change in Google Cloud Storage client library (a8b01e0), closes #2936
• Fixed the vertexai.init partial initialization issues (636a654)
• GCS Bucket subdirectory not being creating while using ReasoningEngine create method (3d22a18)
• GenAI - Capture content blocked case when validating responses (f0086df)
• GenAI - Fixed get_tuned_model deploying the model every time (e1c7870)
• GenAI - Fixed handling of multiple tools in AutomaticFunctionCallingResponder (58e6ac9)
• GenAI - Fixed response validation error during streaming (c881998)
• GenAI - Fixed the GenerativeModel's handling of tuned models from different region (bf33fb3)
• GenAI - Fixed the Part.mime_type property (819a44e)
• GenAI - Fixed the TuningJob dashboard URL (5367fbb)
• GenAI - Improved from_dict methods for content types (GenerationResponse, Candidate, Content, Part) (613ce69)
• GenAI - Tuning - Supervised - Fix adapter_size parameter handling to match enum values. (1cc22c3)
• GenAI - Workaround for streaming when content role is missing in service responses (fa35b91)
• GenAI-ExplicitCaching use the right update_mask for ttl update (c118557)
• Generalize RAG files import from Google Drive (88c6a6a)
• Generate uuid run_name when run_name_prefix is not specified. (4e2d87f)
• Handle case when no metadata is returned from model.predict (59e2bca)
• Handle missing import for RunnableConfig when generating schema for LangChain templates (76c5d6d)
• Improve import time by moving TensorFlow to lazy import (f294ba8)
• Include tensorboard profiler requirements when installing google-cloud-aiplatform[tensorboard] (4f1b559)
• Increase memory for Ray on Vertex default machine type (e6bcb17)
• LLM - Added missing parameters to the no-op _TunableTextEmbeddingModelMixin.get_tuned_model method (eb05ac4)
• LLM - Fixed the InlineContext grounding source (6f59100)
• LVM - Added support for GCS storage.googleapis.com URL import in vision_models.Image (2690e72)
• LVM - Fixed VideoSegmentConfig in Multimodal Embeddings API call (665e78c)
• LVM - Fixed the typo in the VisionModel aspect ratio type annotation (2d19137)
• Made Endpoint.raw_predict thread-safe by setting authorized_session last (c72c1ef)
• Make PipelineJobSchedule propagate labels to created PipelineJobs (a34533f)
• Missing request parameter for v1beta explain. (443fa9d)
• Model Monitor console uri. (71fbc81)
• Move torch import (e6d34df)
• Parse intermediate steps from LangChain into JSON. (754c89d)
• Pin google-cloud-aiplatform[tensorboard] dependency on tensorflow < 2.15.0 due to breaking change introduced in tensorboard 2.15.1 (4e891f7)
• Raise AttributeError for accessing non-existent fields of a proto. (b91edf5)
• Ray - Fixed exception when using Ray 2.4 (2661f52)
• Register TensorFlow models from Ray checkpoints for more recent TensorFlow version, addressing the deprecation of SavedModel format in keras 3 (1341e2c)
• Reinstate persistent resource preview class and tests (765d60d)
• Remove InternalServerError and Unknown evaluation service error from retriable exceptions (12c147b)
• Remove runtime dependency of setuptools (07c2a25)
• Remove usage of distutils (e35ab64)
• Set upper bound of setuptools to unbreak public Colab for using vertex_ray namespace (6cc45bb)
• Tensorboard uploader - move remaining tensorboard uploader code to aiplatform (f34094b)
• Update get_experiment_df to pass Experiment and allow empty metrics. (de5d0f3)
• Update test assumption for streaming endpoint of chat-bison@001 (f9a5b69)
• Upload the reference model in model registry (510c833)
• Use https to connect through Ray dashboard (7af3e67)
• When user is not logged in, throw more intuitive message (a8b24ad)

Documentation

• Add default value to optional field parameter_values (0a4d772)
• Add missing code block in docs/README.rst (16d6b58)
• Add Reasoning Engine reference documentation (496fc4b)
• Add run custom job on persistent resource sample. (53fc845)
• Add run custom job on persistent resource sample. (31100c6)
• Add upload Tensorboard profile log to Experiment sample. (5780513)
• Add Vertex Model Monitoring V2 SDK documentation (b47e6ff)
• Fix incorrect field in log_time_series_metrics_sample (f249353)
• Fixed the documentation generation (63ad1bf)
• GenAI - Add Rapid Evaluation SDK reference documentation (40b728b)
• GenAI - Added GenAI to docs (92fd7f0)
• GenAI - Added the GA classes to the documentation (9eb5a52)
• GenAI - Docs - Added quickstart link and minimum SDK version (cf6ab21)
• GenAI - Updated the README for GA (cfd96d8)
• Include imports in experiment_tracking samples directly in cloud.google.com snippets. (3ffb6cb)
• Update docstring for start_upload_tb_log() (c033c59)
• Update docstrings for rapid evaluation library. (d6d371d)
• Update README (14b41b5)
• Update tensorboard continuous uploader sample (1220746)
• Update the documentation for the image_dataset class (583cb93)
• Update the documentation for the image_dataset class (8562368)
• Update transport docstrings to include that REST is in preview (a442866)

Miscellaneous Chores

• Release 1.38.1 (537d00e)

---

This PR was generated with Release Please. See documentation.