Add retry config to Embedding models

[virajmehta]

This should match our implementation for regular LLMs and use the same RetryConfig and backon-based implementation.

[BretteFitzgibbon]

Hi! CodeDay has assigned me to this issue.

[BretteFitzgibbon]

@GabrielBianconi @Aaron1011

I have added retries parameters to several places, mostly in tensorzero-core/src/embeddings.rs and tests/e2e/tensorzero.toml.

In tensorzero.toml, I saw several places in the Functions section where "retries = {num_retries = 5, max_delay_s = 0.1}" was used, so I hard coded all of my new retries arguments to match these values.

That said, I noticed that the timeouts parameter uses "timeouts: TimeoutsConfig::default()".

In tensorzero-core/src/variant/mod.rs lines 761-768, RetryConfig has a default implementation, but it sets num_retries as 0.

Should I change the default and use RetryConfig::default() throughout the codebase, leave default as is and use it throughout the codebase, or keep the hard-coded values?

All help is very much appreciated.

[virajmehta]

Hi @BretteFitzgibbon; I think in production code you should use RetryConfig::default() in all places in the codebase where we don't actually read from config. since we don't want to hardcode surprising retry preferences. Sorry for the slow response here!

[BretteFitzgibbon]

Thanks, I'll get to it!

[BretteFitzgibbon]

@GabrielBianconi @Aaron1011 @virajmehta

As a result of my changes, upon using the command TS_RS_EXPORT_DIR="$(pwd)/internal/tensorzero-node/lib/bindings" cargo tsbuild (found in internal/tensorzero-node/build-bindings.sh), internal/tensorzero-node/lib/bindings/EmbeddingModelConfig.ts now auto-generates:
-- import type {RetryConfig} from "./RetryConfig"
-- export type EmbeddingModelConfig = {...retries: RetryConfig, }"

The changes:
-- tensorzero-core/src/embeddings.rs:
* Added pub retries: RetryConfig as a parameter for the following structs:
~ EmbeddingModelConfig
~ UninitializedEmbeddingModelConfig
* Added retries: RetryConfig::default() to the following implementations:
~ UninitializedEmbeddingModelConfig
~ ShorthandModelConfig for EmbeddingModelConfig
* Added retries: RetryConfig::default() to the following function:
~ test_embedding_fallbacks()
-- tensorzero-core/src/endpoints/embeddings.rs:
* Added retries: RetryConfig::default() to the function test_no_warning_when_model_exists()
-- tensorzero-core/src/optimization/dicl.rs:
* Added pub retries: RetryConfig as a parameter for the following structs:
~ DiclOptimizationConfig
~ UninitializedDiclOptimizationConfig
* Added retries: RetryConfig::default() to the following implementations:
~ UninitializedDiclOptimizationConfig
~ Default for UninitializedDiclOptimizationConfig
~ UninitializedDiclOptimizationConfig (a different one)
~ JobHandle for DiclOptimizationJobHandle
* Added retries: RetryConfig::default() to the following function:
~ create_test_embedding_model_with_name

Summary:
-- I started by searching for RetryConfig and found internal/tensorzero-node/lib/bindings/RetryConfig.ts.
-- I found RetryConfig being used in 11 TypeScript binding files that I assume pertain to the implementation of regular LLMs, and sought for this syntax to generate in EmbeddingModelConfig.ts.
-- I found the RetryConfig struct definition and default implementation in tensorzero-core/src/utils/retries.rs.
-- I looked for embeddings.rs files to incorporate the retries parameter and found tensorzero-core/src/embeddings.rs and tensorzero-core/src/endpoints/embeddings.rs, where I made the aforementioned changes.
-- I moved to tensorzero-core/src/optimization/dicl.rs because it includes optimizations of embedding models, where I made the aforementioned changes.
-- I found the script to generate bindings in internal/tensorzero-node/build-bindings.sh and ran it.

Am I on the right track?