-
Notifications
You must be signed in to change notification settings - Fork 1
Test Mapping
Generated: 2026-03-09 Total test files: 317 Total test classes: 1162 Total test functions: 5491 Source references: 778 (across 317/317 files)
- tests/benchmarks/ (4 files, 70 tests)
- tests/config/ (7 files, 122 tests)
- tests/db/ (32 files, 533 tests)
- tests/e2e/ (8 files, 125 tests)
- tests/health/ (1 files, 19 tests)
- tests/integration/ (34 files, 328 tests)
- tests/middleware/ (3 files, 65 tests)
- tests/routes/ (61 files, 961 tests)
- tests/schemas/ (10 files, 113 tests)
- tests/security/ (5 files, 123 tests)
- tests/services/ (116 files, 2294 tests)
- tests/smoke/ (1 files, 20 tests)
- tests/tests/ (12 files, 319 tests)
- tests/unit/ (1 files, 17 tests)
- tests/utils/ (22 files, 382 tests)
References: scripts/benchmarks/benchmark_config.py
Module: Tests for benchmark configuration module.
class TestModelConfig - Tests for ModelConfig dataclass.
test_soundsgood_config_values - Test that Soundsgood GLM config has correct values.
test_api_key_from_env - Test that API key is read from environment.
test_api_key_empty_when_not_set - Test that API key is empty when env var not set.
test_custom_model_config - Test creating a custom ModelConfig.
class TestBenchmarkConfig - Tests for BenchmarkConfig dataclass.
test_default_values - Test that default values are set correctly.
test_category_weights_sum_to_one - Test that category weights sum to 1.0.
test_tier_thresholds - Test tier threshold values.
test_custom_run_id - Test setting custom run ID.
class TestTestCategory - Tests for TestCategory enum.
test_all_categories_defined - Test that all expected categories are defined.
test_category_values - Test category string values.
class TestDifficulty - Tests for Difficulty enum.
test_all_difficulties_defined - Test that all difficulty levels are defined.
class TestTestCase - Tests for TestCase dataclass.
test_create_test_case - Test creating a test case.
test_test_case_optional_fields - Test that optional fields default correctly.
class TestTestResult - Tests for TestResult dataclass.
test_tokens_per_second_calculation - Test TPS calculation.
test_tokens_per_second_zero_duration - Test TPS with zero duration returns 0.
class TestBaselineModels - Tests for baseline model definitions.
test_baseline_tiers_exist - Test that all tier levels are defined.
test_tier_1_models - Test tier 1 model definitions.
test_tier_2_models - Test tier 2 model definitions.
test_baseline_score_ordering - Test that tier 1 models generally score higher than tier 4.
References: scripts/benchmarks/latency_tracker.py
Module: Tests for latency tracker module.
class TestLatencyMetrics - Tests for LatencyMetrics dataclass.
test_tokens_per_second - Test TPS calculation.
test_total_output_tokens - Test total output tokens including reasoning.
test_effective_tps - Test effective TPS including reasoning tokens.
test_zero_duration_tps - Test TPS with zero duration returns 0.
class TestLatencyTracker - Tests for LatencyTracker class.
test_record_metrics - Test recording metrics.
test_record_from_response - Test recording from response data.
test_get_stats - Test getting aggregated stats.
test_get_stats_empty_raises - Test that getting stats with no data raises error.
test_percentile_calculation - Test percentile calculations.
test_session_duration - Test session duration tracking.
test_to_dict - Test exporting to dictionary.
class TestCategoryLatencyTracker - Tests for CategoryLatencyTracker class.
test_track_by_category - Test tracking metrics by category.
test_get_all_stats - Test getting stats for all categories.
test_get_overall_stats - Test getting combined stats across categories.
class TestFormatLatencyReport - Tests for latency report formatting.
test_format_report - Test formatting a latency report.
test_format_report_without_ttfc - Test formatting report without TTFC data.
References: scripts/benchmarks/quality_evaluator.py
Module: Tests for quality evaluator module.
class TestQualityScore - Tests for QualityScore dataclass.
test_create_quality_score - Test creating a quality score.
test_quality_score_optional_execution_result - Test that execution_result is optional.
class TestQualityEvaluator - Tests for QualityEvaluator class.
test_init_defaults - Test default initialization.
test_init_custom_values - Test initialization with custom values.
test_extract_code_fenced_python - Test extracting Python code from fenced block.
test_extract_code_generic_fenced - Test extracting code from generic fenced block.
test_extract_code_no_fence - Test extracting code without fence markers.
test_extract_code_empty - Test extracting from response with no code.
test_assess_code_quality_with_docstring - Test code quality assessment with docstring.
test_assess_code_quality_minimal - Test code quality assessment with minimal code.
test_parse_json_response_clean - Test parsing clean JSON response.
test_parse_json_response_with_markdown - Test parsing JSON with markdown code block.
test_parse_json_response_embedded - Test parsing JSON embedded in text.
class TestCodeExecutionEvaluation - Tests for code execution-based evaluation.
test_run_code_tests_success - Test running code tests that pass.
test_run_code_tests_failure - Test running code tests that fail.
test_run_code_tests_syntax_error - Test running code with syntax error.
class TestEvaluateCodeGeneration - Tests for code generation evaluation.
test_evaluate_code_gen_passed - Test evaluating passing code generation.
test_evaluate_code_gen_no_code - Test evaluating response with no extractable code.
class TestIntegration - Integration tests for the evaluator.
test_evaluate_with_mock_judge - Test evaluation with mocked judge call.
References: scripts/benchmarks/soundsgood_client.py
Module: Tests for Soundsgood API client.
class TestCompletionResponse - Tests for CompletionResponse dataclass.
test_create_response - Test creating a completion response.
test_response_without_reasoning - Test response with no reasoning field.
class TestStreamingMetrics - Tests for StreamingMetrics dataclass.
test_create_metrics - Test creating streaming metrics.
class TestSoundsgoodClient - Tests for SoundsgoodClient class.
test_init - Test client initialization.
test_init_custom_timeout - Test client initialization with custom timeout.
test_headers - Test headers property.
test_context_manager - Test async context manager.
test_parse_response - Test parsing non-streaming response.
test_parse_response_no_choices - Test parsing response with no choices raises error.
test_parse_response_missing_fields - Test parsing response with missing optional fields.
class TestStreamingParsing - Tests for streaming response parsing.
test_sse_line_parsing - Test that SSE lines are properly identified.
test_parse_streaming_chunk - Test parsing a single streaming chunk.
test_parse_final_streaming_chunk - Test parsing final streaming chunk with usage.
class TestHealthCheck - Tests for health check functionality.
test_health_check_success - Test successful health check.
test_health_check_failure - Test failed health check.
test_health_check_empty_response - Test health check with empty response.
References: src/config.py, src/config/config.py
Module: Comprehensive tests for src/config/config.py
class TestConfigEnvironmentDetection - Test environment detection logic
test_is_production_detection - Test production environment detection
test_is_staging_detection - Test staging environment detection
test_is_development_detection - Test development environment detection (default)
test_is_testing_detection_with_testing_env - Test testing environment detection with APP_ENV=testing
test_is_testing_detection_with_test_env - Test testing environment detection with APP_ENV=test
test_is_testing_detection_with_testing_flag_true - Test testing environment detection with TESTING=true
test_is_testing_detection_with_testing_flag_1 - Test testing environment detection with TESTING=1
test_is_testing_detection_with_testing_flag_yes - Test testing environment detection with TESTING=yes
class TestConfigProviderKeys - Test provider API key configuration
test_openrouter_keys - Test OpenRouter configuration
test_openrouter_defaults - Test OpenRouter default values
test_openrouter_key_strips_whitespace - Ensure OpenRouter API key trimming removes accidental whitespace
test_all_provider_keys - Test all provider API keys are loaded
class TestConfigGoogleVertex - Test Google Vertex AI configuration
test_google_vertex_defaults - Test Google Vertex AI default configuration
test_google_vertex_custom_values - Test Google Vertex AI custom configuration
class TestConfigMonitoring - Test monitoring and observability configuration
test_prometheus_enabled_by_default - Test Prometheus is enabled by default
test_prometheus_enabled_explicit_true - Test Prometheus enabled with explicit true values
test_prometheus_disabled - Test Prometheus can be disabled
test_prometheus_scrape_enabled_by_default - Test Prometheus scrape is enabled by default
test_tempo_disabled_by_default - Test Tempo is disabled by default
test_tempo_enabled - Test Tempo can be enabled
test_loki_disabled_by_default - Test Loki is disabled by default
test_loki_enabled - Test Loki can be enabled
test_otel_service_name_default - Test OTEL service name default
class TestConfigValidation - Test validate and validate_critical_env_vars methods
test_validate_success_with_all_vars - Test validate succeeds with all required variables
test_validate_skips_in_vercel_environment - Test validate skips validation in Vercel environment
test_validate_raises_on_missing_supabase_url - Test validate raises error on missing SUPABASE_URL
test_validate_raises_on_missing_multiple_vars - Test validate raises error listing all missing variables
test_validate_critical_env_vars_success - Test validate_critical_env_vars with all variables present
test_validate_critical_env_vars_missing_vars - Test validate_critical_env_vars with missing variables
test_validate_critical_env_vars_skips_in_vercel - Test validate_critical_env_vars skips in Vercel environment
test_validate_raises_on_supabase_url_missing_protocol - Test validate raises error when SUPABASE_URL lacks http:// or https:// protocol
test_validate_accepts_http_protocol - Test validate accepts SUPABASE_URL with http:// protocol
test_validate_accepts_https_protocol - Test validate accepts SUPABASE_URL with https:// protocol
test_validate_critical_env_vars_detects_missing_protocol - Test validate_critical_env_vars detects SUPABASE_URL without protocol
test_validate_raises_in_vercel_when_url_missing_protocol - Test validate raises error in Vercel env when SUPABASE_URL lacks protocol
test_validate_critical_env_vars_detects_missing_protocol_in_vercel - Test validate_critical_env_vars detects SUPABASE_URL without protocol in Vercel
test_validate_skips_presence_check_in_vercel - Test validate skips checking presence of keys in Vercel env (but validates URL format)
class TestConfigGetSupabaseConfig - Test get_supabase_config method
test_get_supabase_config_returns_tuple - Test get_supabase_config returns URL and key as tuple
class TestConfigClarifai - Test Clarifai configuration
test_clarifai_configuration - Test Clarifai API configuration
class TestConfigAiHubMix - Test AiHubMix configuration
test_aihubmix_configuration - Test AiHubMix API configuration
class TestConfigAdminAndAnalytics - Test admin and analytics configuration
test_admin_email_configuration - Test admin email configuration
test_openrouter_cookie_configuration - Test OpenRouter cookie configuration
References: src/config.py, src/config/db_config.py
Module: Comprehensive tests for src/config/db_config.py
class TestDatabaseConfig - Test DatabaseConfig class initialization and configuration
test_default_configuration - Test default database configuration values
test_custom_configuration - Test custom database configuration from environment
test_psycopg2_not_available - Test configuration when psycopg2 is not installed
test_get_connection_string - Test connection string generation
test_get_connection_dict - Test connection dictionary generation
class TestConnectionPool - Test connection pool management
test_get_connection_pool_success - Test successful connection pool creation
test_get_connection_pool_reuses_existing - Test connection pool reuse
test_get_connection_pool_not_available - Test connection pool when psycopg2 not available
test_get_connection_pool_failure - Test connection pool creation failure
test_close_all_connections - Test closing all connections in pool
test_close_all_connections_when_none - Test closing connections when pool is None
test_close_all_connections_error - Test error handling when closing connections
class TestConnectionContextManager - Test get_connection context manager
test_get_connection_not_available - Test get_connection when psycopg2 not available
test_get_connection_success - Test successful connection retrieval
test_get_connection_getconn_returns_none - Test when getconn returns None
test_get_connection_with_exception - Test connection rollback on exception
test_get_connection_rollback_fails - Test when rollback itself fails
test_get_connection_putconn_fails - Test when putconn fails
class TestDatabaseOperations - Test database operation methods
test_test_connection_not_available - Test test_connection when psycopg2 not available
test_test_connection_success - Test successful connection test
test_test_connection_failure - Test failed connection test
test_get_database_info_not_available - Test get_database_info when psycopg2 not available
test_get_database_info_success - Test successful database info retrieval
test_execute_query_success - Test successful query execution
test_execute_query_with_params - Test query execution with parameters
test_execute_query_fetch_one - Test query execution with fetch_one=True
test_execute_many_success - Test batch execution success
class TestGlobalFunctions - Test global helper functions
test_get_db_config_singleton - Test get_db_config returns singleton
test_get_db_connection - Test get_db_connection helper
test_test_db_connection - Test test_db_connection helper
test_close_db_connections - Test close_db_connections helper
test_is_db_available - Test is_db_available helper
References: src/config.py, src/config/logging_config.py
Module: Comprehensive tests for Logging Config
class TestLoggingConfig - Test Logging Config functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module has expected public API
References: src/config.py, src/config/opentelemetry_config.py
Module: Comprehensive tests for Opentelemetry Config
class TestOpentelemetryConfig - Test Opentelemetry Config functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module has expected public API
References: src/config/config.py
Module: Tests for Railway deployment configuration consistency.
class TestRailwayConfiguration - Test suite for Railway deployment configuration files.
test_railway_toml_exists - Verify railway.toml exists in project root.
test_railway_json_exists - Verify railway.json exists in project root.
test_healthcheck_timeout_sufficient - Verify healthcheck timeout is sufficient for app startup.
test_healthcheck_initial_delay_sufficient - Verify initial delay is sufficient for first healthcheck.
test_healthcheck_path_configured - Verify healthcheck path is properly configured.
test_healthcheck_interval_configured - Verify healthcheck interval is properly configured.
test_railway_configs_consistency - Verify railway.toml and railway.json have consistent healthcheck settings.
test_start_command_configured - Verify start command is properly configured.
test_restart_policy_configured - Verify restart policy is properly configured.
class TestRailwayHealthcheckRegression - Regression tests for known healthcheck issues.
test_healthcheck_timeout_not_10_seconds - Regression test: Ensure healthcheck timeout is not 10 seconds.
test_healthcheck_initial_delay_not_30_seconds - Regression test: Ensure initial delay is not 30 seconds.
References: src/config.py, src/config/redis_config.py
Module: Comprehensive tests for Redis Config
class TestRedisConfig - Test Redis Config functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module has expected public API
References: src/config/supabase_config.py
Module: Tests for src/config/supabase_config.py
class TestGetSupabaseClientValidation - Test SUPABASE_URL validation in get_supabase_client
test_raises_error_when_supabase_url_not_set - Test that get_supabase_client raises RuntimeError when SUPABASE_URL is not set
test_raises_error_when_supabase_url_missing_protocol - Test that get_supabase_client raises RuntimeError when SUPABASE_URL lacks protocol
test_error_message_includes_example_fix - Test that error message for missing protocol includes example of correct format
test_accepts_valid_https_url - Test that get_supabase_client accepts valid https:// URL
test_accepts_valid_http_url_for_local_dev - Test that get_supabase_client accepts valid http:// URL for local development
test_logs_masked_url_on_initialization - Test that initialization logs a masked version of the URL
class TestHttpxClientConfiguration - Test the httpx client is configured with proper authentication headers.
test_httpx_client_includes_apikey_header - Test that httpx client is created with apikey header.
test_httpx_client_includes_authorization_header - Test that httpx client is created with Authorization Bearer header.
test_httpx_client_auth_headers_match_supabase_key - Test that both auth headers use the same SUPABASE_KEY value.
test_httpx_client_includes_base_url - Test that httpx client is created with correct PostgREST base_url.
test_httpx_client_injected_into_postgrest_session - Test that the configured httpx client is injected into postgrest.session.
test_httpx_client_has_http2_disabled - Test that httpx client is created with HTTP/2 disabled to prevent connection errors.
test_httpx_client_has_connection_pooling - Test that httpx client is created with connection pooling limits.
test_httpx_client_has_timeout_configured - Test that httpx client is created with appropriate timeout settings.
class TestLazySupabaseClient - Test the _LazySupabaseClient proxy class
test_lazy_client_repr - Test that lazy client has a meaningful repr
test_lazy_client_raises_attribute_error_for_dunder_attrs - Test that lazy client raises AttributeError for dunder attributes
class TestGetInitializationStatus - Test the get_initialization_status function
test_get_initialization_status_when_not_initialized - Test initialization status when client is not yet initialized
test_get_initialization_status_when_initialized - Test initialization status when client is successfully initialized
test_get_initialization_status_when_error_occurred - Test initialization status when initialization failed
class TestErrorPersistence - Test error persistence and re-raising behavior
test_get_supabase_client_reraises_previous_error - Test that get_supabase_client re-raises previous initialization errors within TTL
test_initialization_error_captured_on_failure - Test that initialization errors are captured for future reference
class TestTestConnectionInternal - Test the _test_connection_internal function
test_test_connection_internal_success - Test successful connection test
test_test_connection_internal_failure - Test connection test failure
class TestTestConnection - Test the public test_connection function
test_test_connection_uses_cached_client - Test that test_connection uses the cached client
class TestConnectionErrorHandling - Test connection error handling and retry logic
test_is_connection_error_detects_write_error - Test that is_connection_error detects WriteError with bad file descriptor
test_is_connection_error_detects_connection_terminated - Test that is_connection_error detects connection terminated errors
test_is_connection_error_ignores_other_errors - Test that is_connection_error doesn't flag non-connection errors
test_execute_with_retry_retries_on_connection_error - Test that execute_with_retry retries when a connection error occurs
test_execute_with_retry_exhausts_retries - Test that execute_with_retry raises after exhausting retries
test_execute_with_retry_doesnt_retry_non_connection_errors - Test that execute_with_retry doesn't retry non-connection errors
test_refresh_supabase_client_closes_old_connection - Test that refresh_supabase_client properly closes the old connection
References: src/db/activity.py, src/routes/activity.py
Module: Comprehensive tests for activity tracking database layer
class TestLogActivity - Test activity logging operations
test_log_activity_success - Test successfully logging an activity
test_log_activity_with_defaults - Test logging activity with default values
test_log_activity_no_data - Test logging activity when no data is returned
test_log_activity_error - Test error handling during activity logging
class TestActivityStatistics - Test activity statistics retrieval
test_get_activity_stats_with_data - Test retrieving activity statistics with data
test_get_activity_stats_empty - Test retrieving statistics when no activity
test_get_activity_stats_with_date_range - Test retrieving statistics with specific date range
test_get_activity_stats_default_30_days - Test default 30-day lookback
test_get_activity_stats_error - Test error handling in statistics
class TestActivityLogRetrieval - Test paginated activity log retrieval
test_get_activity_log_success - Test retrieving activity log
test_get_activity_log_with_pagination - Test activity log with pagination
test_get_activity_log_with_date_filters - Test activity log with date filtering
test_get_activity_log_with_model_filter - Test activity log with model filter
test_get_activity_log_with_provider_filter - Test activity log with provider filter
test_get_activity_log_empty - Test retrieving empty activity log
test_get_activity_log_error - Test error handling in activity log retrieval
class TestProviderDetection - Test provider detection from model names
test_detect_openai_models - Test detecting OpenAI models
test_detect_anthropic_models - Test detecting Anthropic models
test_detect_google_models - Test detecting Google models
test_detect_meta_models - Test detecting Meta models
test_detect_mistral_models - Test detecting Mistral AI models
test_detect_alibaba_models - Test detecting Alibaba models
test_detect_deepseek_models - Test detecting DeepSeek models
test_detect_unknown_models - Test detecting unknown/other models
test_provider_detection_case_insensitive - Test provider detection is case-insensitive
class TestActivityIntegration - Test activity tracking integration scenarios
test_log_and_retrieve_activity - Test logging activity and retrieving it
test_stats_include_logged_activity - Test that logged activities appear in statistics
References: src/config/supabase_config.py, src/db/api_keys.py, src/db/plans.py, src/db/users.py, src/routes/api_keys.py, src/schemas/api_keys.py, src/security/security.py
test_check_key_name_uniqueness
test_create_api_key_primary_sets_trial_and_prefix_and_audit
test_create_api_key_refreshes_schema_cache_on_pgrst204
test_create_api_key_schema_cache_error_fallback - Ensure we recover from PostgREST schema cache misses for new columns.
test_get_user_api_keys_builds_fields
test_delete_api_key_new_and_legacy
test_validate_api_key_prefers_api_keys_then_fallback
test_increment_api_key_usage_updates_new_or_legacy
test_get_api_key_usage_stats_new_vs_legacy
test_update_api_key_name_uniqueness_and_expiration_and_rate_limit
test_validate_api_key_permissions
test_get_api_key_by_id
test_get_user_all_api_keys_usage
test_get_api_key_by_key_retries_on_http2_error - HTTP/2 connection errors should trigger retry via execute_with_retry
test_get_api_key_by_key_retries_on_connection_terminated - ConnectionTerminated errors should trigger retry
test_get_api_key_by_key_retries_on_send_headers_error - SEND_HEADERS state errors should trigger retry
test_get_api_key_by_key_returns_none_after_max_retries - Should return None after max retries are exhausted
test_get_api_key_by_key_no_retry_on_non_connection_error - Non-connection errors should not trigger retry
References: src/db/api_keys.py, src/utils/crypto.py
Module: Tests for API key creation graceful degradation when tables are missing.
class TestAPIKeyGracefulDegradation - Test graceful degradation when database tables are missing.
test_create_api_key_missing_rate_limit_configs_table - Test that API key creation succeeds even when rate_limit_configs table is missing.
test_create_api_key_missing_audit_logs_table - Test that API key creation succeeds even when api_key_audit_logs table is missing.
test_create_api_key_both_tables_missing - Test that API key creation succeeds even when BOTH auxiliary tables are missing.
test_delete_api_key_missing_tables - Test that API key deletion succeeds when auxiliary tables are missing.
test_update_api_key_missing_tables - Test that API key update succeeds when auxiliary tables are missing.
test_create_api_key_real_error_still_logged - Test that real errors (non-missing-table) are still logged properly.
References: src/db/api_keys.py, src/utils/crypto.py
Module: Tests for KEY_HASH_SALT validation in API key creation.
class TestKeyHashSaltValidation - Test suite for KEY_HASH_SALT validation
test_create_api_key_fails_without_key_hash_salt - Test that API key creation fails when KEY_HASH_SALT is missing
test_create_api_key_fails_with_short_key_hash_salt - Test that API key creation fails when KEY_HASH_SALT is too short
test_hash_salt_check_not_bypassed_by_encryption_failure - Test that KEY_HASH_SALT validation happens BEFORE encryption.
test_create_api_key_succeeds_with_valid_salt_and_no_encryption - Test that API key creation succeeds with valid KEY_HASH_SALT but no encryption
test_create_api_key_succeeds_with_salt_and_encryption - Test that API key creation succeeds with both KEY_HASH_SALT and encryption
test_error_message_includes_generation_command - Test that error message includes the command to generate KEY_HASH_SALT
test_hash_validation_order_before_encryption - Test that hash validation happens strictly before encryption.
References: src/db/api_keys.py, src/db/chat_completion_requests.py, src/services/ai_sdk_client.py, src/utils/db_safety.py, src/utils/provider_safety.py
Module: Tests for backend error fixes - January 2, 2026
class TestAPIKeysDataAccessSafety - Test safe database access patterns in src/db/api_keys.py
test_create_api_key_handles_empty_result - Test that create_api_key handles empty database result safely
test_get_api_key_usage_stats_handles_empty_result - Test that get_api_key_usage_stats handles missing API key safely
test_update_api_key_handles_empty_key_lookup - Test that update_api_key handles missing API key safely
test_update_api_key_handles_empty_update_result - Test that update_api_key handles failed update safely
test_delete_api_key_handles_empty_lookup - Test that delete_api_key handles missing API key safely
test_create_api_key_audit_log_uses_safe_id - Test that create_api_key uses safely extracted ID for audit logs
test_delete_api_key_uses_safe_id_for_cleanup - Test that delete_api_key uses safely extracted ID for cleanup operations
class TestChatCompletionRequestsDataAccessSafety - Test safe database access patterns in src/db/chat_completion_requests.py
test_get_model_id_by_name_handles_empty_provider_result - Test that get_model_id_by_name handles missing provider safely
test_get_model_id_by_name_handles_empty_model_result - Test that get_model_id_by_name handles missing model safely
test_save_chat_completion_request_handles_empty_insert_result - Test that save_chat_completion_request handles failed insert safely
test_get_model_performance_metrics_handles_empty_model_data - Test that get_model_performance_metrics handles missing model info safely
class TestAISDKClientProviderSafety - Test safe provider response handling in src/services/ai_sdk_client.py
test_process_ai_sdk_response_handles_empty_choices - Test that _process_ai_sdk_response handles empty choices safely
test_process_ai_sdk_response_handles_missing_choices_attribute - Test that _process_ai_sdk_response handles missing choices attribute safely
test_process_ai_sdk_response_handles_missing_message_attributes - Test that _process_ai_sdk_response handles missing message attributes safely
test_process_ai_sdk_response_handles_valid_response - Test that _process_ai_sdk_response processes valid response correctly
class TestIntegrationScenarios - Test integration scenarios combining multiple fixes
test_concurrent_empty_result_handling - Test that multiple concurrent empty results don't cause crashes
test_error_propagation_chain - Test that errors propagate correctly through the chain
test_model_lookup_fallback_chain - Test that model lookup gracefully falls through multiple strategies
test_safe_get_first_validates_keys - Test that safe_get_first validates required keys exist
test_provider_error_distinguishes_from_generic_errors - Test that ProviderError is distinct from generic exceptions
class TestEdgeCases - Test edge cases and boundary conditions
test_safe_get_first_with_none_data - Test safe_get_first handles None data attribute
test_safe_get_first_with_dict_instead_of_list - Test safe_get_first handles dict data instead of list
test_ai_sdk_response_with_malformed_choice_attributes - Test AI SDK handles choices with missing or malformed attributes
test_save_chat_completion_request_preserves_result_on_success - Test that save_chat_completion_request returns the created record
References: src/db/chat_history.py, src/routes/chat_history.py
test_create_and_get_session_with_messages
test_get_user_chat_sessions_pagination_and_sort
test_update_and_delete_chat_session
test_chat_session_stats
test_search_chat_sessions_title_and_message
References: src/db/chat_history.py
Module: Unit tests for chat history message deduplication functionality.
test_save_message_no_duplicate_normal_path - Test normal save when no duplicate exists
test_save_message_duplicate_detected_returns_existing - Test that duplicate message returns existing instead of creating new
test_save_message_skip_duplicate_check_creates_new - Test that skip_duplicate_check=True bypasses duplicate detection
test_save_message_empty_content_allowed - Test that empty content is allowed and saved
test_save_message_different_sessions_not_duplicate - Test that same content in different sessions is not considered duplicate
test_save_message_different_roles_not_duplicate - Test that same content with different roles is not considered duplicate
test_save_message_updates_session_timestamp - Test that saving a message updates session updated_at
test_save_message_with_model_updates_session_model - Test that saving a message with model updates session model
References: src/db/coupons.py, src/routes/coupons.py, src/schemas/coupons.py
Module: Comprehensive tests for coupon database operations
class TestCreateCoupon - Test coupon creation
test_create_global_coupon_success - Test creating a global coupon
test_create_user_specific_coupon_success - Test creating a user-specific coupon
test_create_user_specific_without_user_id_fails - Test creating user-specific coupon without user ID fails
test_create_global_coupon_with_user_id_fails - Test creating global coupon with user ID fails
test_create_user_specific_with_max_uses_not_1_fails - Test creating user-specific coupon with max_uses != 1 fails
class TestGetCoupon - Test coupon retrieval
test_get_coupon_by_code_success - Test retrieving coupon by code
test_get_coupon_by_code_not_found - Test retrieving non-existent coupon
test_get_coupon_by_id_success - Test retrieving coupon by ID
test_list_coupons_all - Test listing all coupons
test_list_coupons_with_filters - Test listing coupons with filters
class TestUpdateCoupon - Test coupon updates
test_update_coupon_success - Test updating coupon fields
test_update_coupon_filters_invalid_fields - Test that invalid fields are filtered out
test_deactivate_coupon_success - Test deactivating a coupon
class TestValidateCoupon - Test coupon validation
test_validate_coupon_valid - Test validating a valid coupon
test_validate_coupon_invalid - Test validating an invalid coupon
class TestRedeemCoupon - Test coupon redemption
test_redeem_coupon_success - Test successfully redeeming a coupon
test_redeem_coupon_invalid - Test redeeming an invalid coupon
class TestUserCouponQueries - Test user-specific coupon queries
test_get_available_coupons_for_user - Test getting available coupons for a user
test_get_user_redemption_history - Test getting user redemption history
class TestCouponAnalytics - Test coupon analytics
test_get_coupon_analytics - Test getting analytics for a coupon
test_get_all_coupons_stats - Test getting overall coupon stats
class TestCouponErrorHandling - Test error handling
test_get_coupon_by_code_error - Test error handling when getting coupon by code
test_list_coupons_error - Test error handling when listing coupons
test_validate_coupon_error - Test error handling during coupon validation
References: src/db/credit_transactions.py
Module: Comprehensive tests for credit transaction tracking - CRITICAL for financial accuracy
class TestLogCreditTransaction - Test logging credit transactions with full audit trail
test_log_deduction_transaction_success - Test successfully logging a credit deduction
test_log_addition_transaction_success - Test successfully logging a credit addition
test_log_transaction_with_all_types - Test logging transactions with all available transaction types
test_log_transaction_no_data_returned - Test handling when no data is returned from insert
test_log_transaction_exception_handling - Test exception handling during transaction logging
test_log_transaction_with_metadata - Test logging transaction with complex metadata
class TestGetUserTransactions - Test retrieving user transaction history
test_get_transactions_basic - Test basic transaction retrieval
test_get_transactions_with_pagination - Test transaction retrieval with pagination
test_get_transactions_with_type_filter - Test filtering transactions by type
test_get_transactions_empty_result - Test handling empty transaction history
test_get_transactions_exception_handling - Test exception handling during retrieval
class TestAddCredits - Test adding credits to user accounts
test_add_credits_success_with_api_key - Test successfully adding credits using API key
test_add_credits_success_with_user_id - Test successfully adding credits using user ID
test_add_credits_negative_amount - Test rejection of negative credit amounts
test_add_credits_zero_amount - Test rejection of zero credit amounts
test_add_credits_user_not_found - Test handling when user is not found
test_add_credits_update_fails - Test handling when credit update fails
test_add_credits_exception_handling - Test exception handling during credit addition
class TestGetTransactionSummary - Test transaction summary calculations
test_get_summary_with_mixed_transactions - Test summary calculation with various transaction types
test_get_summary_empty_transactions - Test summary with no transactions
test_get_summary_only_additions - Test summary with only credit additions
test_get_summary_only_deductions - Test summary with only credit deductions
test_get_summary_exception_handling - Test exception handling during summary calculation
class TestTransactionType - Test TransactionType enum values
test_transaction_type_values - Test all transaction type constants are defined
References: src/db.py, src/db/rate_limits.py, src/db/users.py, src/routes/auth.py, src/schemas/auth.py, src/services/rate_limiting.py, src/services/rate_limiting_fallback.py
Module: Tests for safe data access patterns - ensuring no IndexError on empty lists.
class TestUsersDataAccessSafety - Test safe data access in users.py
test_deduct_credits_v2_handles_empty_concurrent_balance_check - Test that concurrent modification error handling doesn't crash on empty data
class TestRateLimitsDataAccessSafety - Test safe data access in rate_limits.py
test_get_user_rate_limits_handles_empty_key_record - Test that empty key record is handled gracefully
test_get_user_rate_limits_handles_empty_config_record - Test that empty rate config record is handled gracefully
test_get_rate_limit_config_handles_empty_results - Test get_rate_limit_config handles empty results at all levels
test_update_rate_limit_usage_handles_empty_existing_record - Test that empty existing record check is handled gracefully
class TestAuthDataAccessSafety - Test safe data access in auth.py routes
test_password_reset_handles_empty_user_result - Test password reset doesn't crash on empty user result
test_reset_password_handles_empty_token_result - Test reset password handles empty token result
class TestRateLimitingConcurrencyReenabled - Test that concurrency limits are properly re-enabled
test_concurrency_limit_enforced - Test that concurrency limits are now enforced (not disabled)
test_fallback_concurrency_limit_enforced - Test that fallback concurrency limits are now enforced (not disabled)
test_concurrency_limit_allows_under_limit - Test that requests under concurrency limit are allowed
test_main_limiter_increments_concurrency_on_allowed_request - Test that SlidingWindowRateLimiter increments concurrency counter when request allowed
test_main_limiter_does_not_increment_on_rejected_request - Test that concurrency counter is NOT incremented when request is rejected
References: src/db/failover_db.py
Module: Tests for failover_db.py model_name migration fix
class TestFailoverDbModelNameMigration - Test that failover_db uses model_name instead of model_id
test_get_providers_for_model_uses_model_name - Test that get_providers_for_model queries by model_name, not model_id
test_get_providers_extracts_pricing_from_model_pricing_table - Test that pricing is extracted from model_pricing relationship, not direct columns
test_get_providers_handles_pricing_as_list - Test that model_pricing as list (PostgREST one-to-many) is handled correctly
test_get_providers_handles_missing_pricing_data - Test that missing pricing data defaults to 0
test_get_provider_model_id_uses_model_name - Test that get_provider_model_id queries by model_name, not model_id
test_check_model_available_uses_model_name - Test that check_model_available_on_provider queries by model_name
test_get_providers_returns_empty_list_for_nonexistent_model - Test that empty list is returned when model not found
References: src/db/feedback.py
Module: Tests for the message feedback database module.
test_save_thumbs_up_feedback
test_save_thumbs_down_feedback_with_comment
test_save_regenerate_feedback_with_metadata
test_save_feedback_invalid_type_raises_error
test_save_feedback_invalid_rating_raises_error
test_get_user_feedback_all
test_get_user_feedback_by_type
test_get_user_feedback_by_session
test_get_user_feedback_by_model
test_get_user_feedback_pagination
test_get_feedback_by_session
test_get_feedback_by_message
test_update_feedback_type
test_update_feedback_rating
test_update_feedback_comment
test_update_feedback_wrong_user_returns_none
test_update_feedback_invalid_type_raises_error
test_update_feedback_invalid_rating_raises_error
test_delete_feedback_success
test_delete_feedback_wrong_user_returns_false
test_delete_feedback_nonexistent_returns_false
test_get_feedback_stats_basic
test_get_feedback_stats_with_ratings
test_get_feedback_stats_by_model
test_get_feedback_stats_filter_by_model
test_get_feedback_stats_empty
References: src/db/gateway_analytics.py
Module: Comprehensive tests for gateway_analytics database module
class TestGetProviderStats - Test get_provider_stats function
test_get_provider_stats_success - Test successfully getting provider statistics
test_get_provider_stats_with_user_filter - Test getting provider stats filtered by user
test_get_provider_stats_no_data - Test provider stats with no matching data
test_get_provider_stats_error_handling - Test error handling in provider stats
test_get_provider_stats_with_gateway_filter - Test provider stats with gateway filter
class TestGetGatewayStats - Test get_gateway_stats function
test_get_gateway_stats_success - Test successfully getting gateway statistics
test_get_gateway_stats_with_user_filter - Test getting gateway stats with user filter
test_get_gateway_stats_time_ranges - Test gateway stats with different time ranges
References: src/db/model_health.py, src/routes/model_health.py
Module: Tests for model health tracking database operations.
class TestRecordModelCall - Tests for the record_model_call function
test_record_model_call_new_record - Test recording a new model call creates a record
test_record_model_call_existing_record - Test recording a model call updates existing record
test_record_model_call_error_status - Test recording an error increments error count
test_record_model_call_table_not_found - Test graceful handling when table doesn't exist
test_record_model_call_duplicate_key_handled - Test duplicate key error is handled gracefully (race condition)
test_record_model_call_with_tokens - Test recording model call with token usage
test_record_model_call_with_gateway - Test recording model call with explicit gateway parameter
test_record_model_call_gateway_defaults_to_provider - Test gateway defaults to provider when not specified
class TestGetModelHealth - Tests for the get_model_health function
test_get_model_health_found - Test retrieving health data for existing model
test_get_model_health_not_found - Test retrieving health data for non-existent model
class TestGetUnhealthyModels - Tests for the get_unhealthy_models function
test_get_unhealthy_models - Test retrieving models with high error rates
class TestGetModelHealthStats - Tests for the get_model_health_stats function
test_get_model_health_stats - Test aggregating health stats across all models
test_get_model_health_stats_empty - Test stats with no models tracked
References: src/db/models_catalog_db.py
Module: Comprehensive tests for database catalog functions (Phase 1 - Issue #990)
test_transform_db_model_to_api_format_complete - Test transformation with all fields present
test_transform_db_model_with_missing_fields - Test transformation gracefully handles missing fields
test_transform_db_model_with_null_pricing - Test transformation when pricing fields are null
test_transform_db_model_error_handling - Test transformation handles errors gracefully
test_transform_db_models_batch - Test batch transformation
test_transform_preserves_no_data_loss - Test that transformation preserves all important data
test_get_all_models_for_catalog - Test fetching all models from database
test_get_all_models_include_inactive - Test including inactive models
test_get_models_by_gateway_for_catalog - Test filtering by provider
test_get_models_for_catalog_with_filters_no_filters - Test advanced query with no filters (should return all)
test_get_models_for_catalog_with_filters_by_gateway - Test filtering by gateway
test_get_models_for_catalog_with_filters_by_modality - Test filtering by modality
test_get_models_for_catalog_with_filters_search - Test search functionality
test_get_models_for_catalog_with_filters_pagination - Test pagination
test_get_models_for_catalog_combined_filters - Test combining multiple filters
test_get_models_count_by_filters - Test count function
test_get_models_count_matches_query - Test count matches actual query results
test_get_model_by_model_id_string - Test looking up single model by ID
test_get_model_by_model_id_string_not_found - Test lookup with non-existent model ID
test_get_catalog_statistics - Test catalog statistics function
test_catalog_query_performance - Test query performance meets targets (< 100ms)
test_transformation_performance - Test transformation performance (< 1ms per model)
test_filtered_query_performance - Test performance of filtered queries
test_count_query_performance - Test performance of count queries
test_empty_result_handling - Test handling of queries that return no results
test_special_characters_in_search - Test search with special characters
test_transform_empty_list - Test batch transformation with empty list
test_all_models_have_required_fields - Test that all models have required fields for API
test_transformed_models_match_api_schema - Test that transformed models match expected API schema
test_provider_join_integrity - Test that provider join always works
References: src/db/payments.py, src/routes/payments.py, src/schemas/payments.py, src/services/payments.py
Module: Comprehensive tests for payment database operations - CRITICAL for financial integrity
class TestCreatePayment - Test payment record creation
test_create_payment_success - Test successful payment creation
test_create_payment_with_metadata - Test payment creation with metadata
test_create_payment_no_data_returned - Test handling when no data returned
test_create_payment_exception - Test exception handling during creation
class TestRetrievePayments - Test payment retrieval operations
test_get_payment_success - Test retrieving payment by ID
test_get_payment_not_found - Test getting non-existent payment
test_get_payment_by_stripe_intent_payment_intent - Test getting payment by Stripe payment intent ID
test_get_payment_by_stripe_intent_session_id - Test getting payment by Stripe session ID (fallback)
test_get_user_payments - Test getting all payments for a user
test_get_user_payments_with_status_filter - Test getting user payments filtered by status
test_get_recent_payments - Test getting recent payments (admin function)
class TestUpdatePayments - Test payment update operations
test_update_payment_status_to_completed - Test updating payment status to completed
test_update_payment_status_sets_checkout_session_id - Ensure stripe session updates use the checkout session column
test_update_payment_status_to_failed_with_error - Test updating payment status to failed with error message
test_update_payment_metadata - Test updating payment metadata
class TestDeletePayment - Test payment deletion (rare operation)
test_delete_payment_success - Test successful payment deletion
test_delete_payment_failure - Test payment deletion failure
class TestPaymentStatistics - Test payment statistics and analytics
test_get_payment_statistics_user_specific - Test getting payment statistics for specific user
test_get_total_revenue - Test getting total revenue statistics
test_get_payment_trends - Test getting payment trends over time
References: src/db/payments.py
Module: Comprehensive tests for payment database operations with HTTP/2 retry logic
class TestCreatePaymentWithRetry - Test payment creation with HTTP/2 connection error retry
test_create_payment_success - Test successful payment creation with retry wrapper
test_create_payment_no_data_returned - Test payment creation fails when no data returned
test_create_payment_http2_connection_error - Test HTTP/2 ConnectionTerminated error during payment creation
test_create_payment_stream_id_error - Test StreamIDTooLowError during payment creation
test_create_payment_send_headers_error - Test SEND_HEADERS LocalProtocolError during payment creation
test_create_payment_with_metadata - Test payment creation with custom metadata
test_create_payment_amount_calculation - Test payment amount is correctly converted to cents
class TestUpdatePaymentStatusWithRetry - Test payment status updates with HTTP/2 connection error retry
test_update_payment_status_success - Test successful payment status update
test_update_payment_status_to_failed - Test updating payment status to failed with error message
test_update_payment_status_no_data_returned - Test update fails when no data returned
test_update_payment_status_http2_error - Test HTTP/2 connection error during status update
test_update_payment_status_stream_id_error - Test StreamIDTooLowError during status update
test_update_payment_status_remote_protocol_error - Test RemoteProtocolError during status update
test_update_payment_status_with_stripe_fields - Test status update with Stripe payment intent and session IDs
class TestPaymentsIntegration - Integration tests for payment operations with retry resilience
test_payment_creation_and_update_flow - Test full payment flow: create -> update status
test_resilience_to_transient_errors - Test that execute_with_retry handles transient errors gracefully
test_multiple_payment_operations - Test multiple payment operations with consistent retry behavior
class TestPaymentErrorHandling - Test edge cases and error handling scenarios
test_create_payment_generic_exception - Test generic exception handling during payment creation
test_get_payment_with_retry_success - Test get_payment with retry wrapper
test_get_payment_http2_error - Test get_payment with HTTP/2 connection error
test_get_payment_not_found - Test get_payment when payment not found
test_update_payment_status_generic_exception - Test generic exception handling during status update
test_create_payment_with_zero_amount - Test payment creation with zero amount
test_create_payment_with_large_amount - Test payment creation with large amount
References: src/db.py, src/db/ping.py, src/routes/ping.py, src/services/ping.py
Module: Comprehensive tests for Ping database operations
class TestPing - Test Ping database functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/config/supabase_config.py, src/db/plans.py, src/routes/plans.py, src/schemas/plans.py
test_get_all_plans_returns_active_sorted
test_get_plan_by_id_converts_features_dict_to_list
test_get_user_plan_combines_user_and_plan
test_assign_user_plan_deactivates_existing_and_updates_user
test_check_plan_entitlements_no_plan_defaults
test_check_plan_entitlements_expired_plan_marks_inactive_and_user_expired
test_check_plan_entitlements_active_plan_allows_feature
test_get_user_usage_within_plan_limits_aggregates
test_enforce_plan_limits_checks_and_env_multiplier
test_get_subscription_plans_active_only
test_get_all_plans_error_returns_empty
test_get_plan_id_by_tier_finds_matching_plan
test_get_plan_id_by_tier_returns_none_when_not_found
test_get_plan_id_by_tier_only_returns_active_plans
test_get_plan_id_by_tier_error_returns_none
test_usage_cache_hit_skips_database - Second call with same user_id should use cache (not hit database)
test_usage_cache_different_users_separate_entries - Different user IDs should have separate cache entries
test_clear_usage_cache_all - clear_usage_cache() with no arguments should clear entire cache
test_invalidate_usage_cache_specific_user - invalidate_usage_cache should clear cache for specific user
test_get_usage_cache_stats - get_usage_cache_stats should return cache statistics
References: src/db/postgrest_schema.py
test_refresh_postgrest_schema_cache_falls_back_to_direct_notify
test_refresh_postgrest_schema_cache_returns_false_without_dsn
References: src/db.py, src/db/ranking.py, src/routes/ranking.py
Module: Comprehensive tests for Ranking database operations
class TestRanking - Test Ranking database functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/db/rate_limits.py, src/routes/rate_limits.py
test_get_user_rate_limits_new_path
test_get_user_rate_limits_legacy_fallback
test_get_user_rate_limits_none_when_no_rows
test_set_user_rate_limits_insert_then_update
test_check_rate_limit_allowed_when_under
test_check_rate_limit_blocks_on_request_minute
test_check_rate_limit_blocks_on_tokens_minute
test_update_rate_limit_usage_inserts_then_updates
test_get_environment_usage_summary
test_get_update_rate_limit_config_and_list
test_bulk_update_rate_limit_configs
test_get_rate_limit_usage_stats_minute
test_get_system_rate_limit_stats
test_create_and_get_rate_limit_alerts
References: src/db/rate_limits.py
Module: Test suite for rate_limits to rate_limit_configs migration.
class TestGetUserRateLimitsConfigsMigration - Test get_user_rate_limits with rate_limit_configs table
test_get_rate_limits_from_configs_table - Test retrieving rate limits from rate_limit_configs table
test_get_rate_limits_returns_none_when_no_config - Test that None is returned when no rate limits are configured
test_get_rate_limits_handles_missing_api_key - Test handling when API key doesn't exist
class TestSetUserRateLimitsConfigsMigration - Test set_user_rate_limits with rate_limit_configs table
test_set_rate_limits_creates_new_config - Test creating new rate limit config in rate_limit_configs table
test_set_rate_limits_updates_existing_config - Test updating existing rate limit config
test_set_rate_limits_raises_error_for_missing_key - Test that ValueError is raised when API key not found
class TestRateLimitsLegacyTableRemoval - Test that legacy rate_limits table is no longer referenced
test_no_fallback_to_legacy_table - Test that get_user_rate_limits does NOT fallback to legacy rate_limits table.
test_set_user_rate_limits_uses_configs_table - Verify that set_user_rate_limits uses rate_limit_configs table.
class TestRateLimitConfigsIntegration - Integration tests for rate_limit_configs functionality
test_full_lifecycle_create_retrieve_update - Test full lifecycle: create config, retrieve it, update it
test_module_imports - Smoke test to ensure the module imports correctly after migration
References: src/db/referral.py, src/routes/referral.py, src/services/referral.py
Module: Tests for referral database models
class TestReferralCodeGeneration - Test referral code generation
test_generate_referral_code_length - Test referral code has correct length
test_generate_referral_code_uppercase_digits - Test referral code contains only uppercase and digits
test_generate_referral_code_uniqueness - Test that multiple codes are likely unique
class TestUserModel - Test User model
test_user_init - Test User initialization
test_user_init_with_referral - Test User initialization with referral code
test_user_to_dict - Test User to_dict method
test_user_get_remaining_referral_uses_zero_used - Test remaining uses when none used
test_user_get_remaining_referral_uses_some_used - Test remaining uses when partially used
test_user_get_remaining_referral_uses_max_used - Test remaining uses when max reached
test_user_get_remaining_referral_uses_over_max - Test remaining uses when over max (edge case)
class TestCouponUsageModel - Test CouponUsage model
test_coupon_usage_to_dict - Test CouponUsage to_dict method
test_coupon_usage_default_bonus - Test CouponUsage default bonus amount
test_coupon_usage_default_is_valid - Test CouponUsage default is_valid
class TestPurchaseModel - Test Purchase model
test_purchase_to_dict - Test Purchase to_dict method
test_purchase_default_referral_bonus - Test Purchase default referral bonus
test_purchase_default_referral_code - Test Purchase default referral code
References: src/db/roles.py, src/routes/roles.py, src/services/roles.py
Module: Comprehensive tests for role management database layer
class TestPermissionChecking - Test user permission checking
test_check_user_permission_granted - Test permission check when permission is granted
test_check_user_permission_denied - Test permission check when permission is denied
test_check_user_permission_no_data - Test permission check when no data is returned
test_check_user_permission_error - Test permission check with database error
test_get_user_permissions_success - Test retrieving all user permissions
test_get_user_permissions_empty - Test retrieving permissions when user has none
test_get_user_permissions_error - Test error handling when retrieving permissions
class TestUserRoleManagement - Test user role retrieval and updates
test_get_user_role_success - Test retrieving user's role
test_get_user_role_not_found - Test retrieving role for non-existent user
test_get_user_role_error - Test error handling when getting user role
test_update_user_role_success - Test successfully updating user's role
test_update_user_role_with_audit_log - Test role update creates audit log entry
test_update_user_role_invalid_role - Test updating to invalid role
test_update_user_role_no_data - Test role update when no data is returned
test_update_user_role_error - Test error handling during role update
class TestRoleAuditLog - Test role change audit logging
test_get_role_audit_log_all - Test retrieving all audit log entries
test_get_role_audit_log_by_user - Test retrieving audit log for specific user
test_get_role_audit_log_empty - Test retrieving empty audit log
test_get_role_audit_log_error - Test error handling when retrieving audit log
class TestUsersByRole - Test retrieving users by role
test_get_users_by_role_success - Test retrieving users with specific role
test_get_users_by_role_admin - Test retrieving admin users
test_get_users_by_role_empty - Test retrieving users when none have role
test_get_users_by_role_error - Test error handling when retrieving users by role
class TestRolePermissionsManagement - Test role permissions CRUD operations
test_get_role_permissions_success - Test retrieving all permissions for a role
test_get_role_permissions_empty - Test retrieving permissions when role has none
test_add_role_permission_success - Test adding permission to a role
test_add_role_permission_no_data - Test adding permission when no data is returned
test_add_role_permission_error - Test error handling when adding permission
test_remove_role_permission_success - Test removing permission from a role
test_remove_role_permission_no_data - Test removing permission when no data is returned
test_remove_role_permission_error - Test error handling when removing permission
class TestRoleConstants - Test role constants and validation
test_user_role_constants - Test role constant values
test_role_constants_are_strings - Test all role constants are strings
test_role_constants_uniqueness - Test all role constants are unique
References: src/db/shared_chats.py
Module: Tests for the shared_chats database functions.
test_generate_share_token_is_unique - Test that generate_share_token produces unique tokens
test_generate_share_token_is_url_safe - Test that generated tokens are URL-safe
test_create_shared_chat_success - Test successfully creating a shared chat
test_create_shared_chat_with_expiry - Test creating shared chat with expiration date
test_get_shared_chat_by_token_success - Test getting shared chat by token
test_get_shared_chat_by_token_not_found - Test getting non-existent shared chat
test_get_shared_chat_by_token_expired - Test getting expired shared chat returns None
test_get_user_shared_chats_success - Test getting user's shared chats
test_get_user_shared_chats_empty - Test getting shared chats when user has none
test_delete_shared_chat_success - Test deleting a shared chat
test_delete_shared_chat_not_found - Test deleting non-existent shared chat
test_verify_session_ownership_true - Test verifying session ownership returns True for owner
test_verify_session_ownership_false - Test verifying session ownership returns False for non-owner
test_check_share_rate_limit_within_limit - Test rate limit check when within limit
test_check_share_rate_limit_exceeded - Test rate limit check when limit exceeded
test_check_share_rate_limit_custom_limit - Test rate limit check with custom limit
References: src/db/subscription_products.py
Module: Tests for subscription products database module
class TestSubscriptionProducts - Test subscription products database operations
test_get_tier_from_product_id_pro - Test getting tier from PRO product ID
test_get_tier_from_product_id_max - Test getting tier from MAX product ID
test_get_tier_from_product_id_not_found - Test getting tier from unknown product ID defaults to basic
test_get_tier_from_product_id_recovers_from_schema_cache_miss - Ensure schema cache refresh is attempted when PostgREST cannot find the table.
test_get_tier_from_product_id_schema_cache_refresh_failure - Ensure we fall back to basic tier when schema cache refresh cannot run.
test_get_credits_from_tier_pro - Test getting credits for PRO tier
test_get_credits_from_tier_max - Test getting credits for MAX tier
test_get_credits_from_tier_not_found - Test getting credits from unknown tier defaults to 0
test_get_subscription_product - Test getting full product configuration
test_get_all_active_products - Test getting all active products
test_add_subscription_product - Test adding new subscription product
test_add_subscription_product_failure - Test adding product failure
test_update_subscription_product - Test updating subscription product
test_update_subscription_product_no_fields - Test updating product with no fields returns False
test_database_error_handling - Test that database errors are handled gracefully
References: src/db/credit_transactions.py, src/db/plans.py, src/db/subscription_products.py, src/db/users.py, src/services/daily_usage_limiter.py
Module: Tests for tiered subscription credit tracking system.
class TestTieredCreditDeduction - Test credit deduction logic with tiered subscription tracking.
test_deduct_from_allowance_first - Verify allowance is consumed before purchased credits.
test_deduct_split_across_both - Test deduction that spans both allowance and purchased credits.
test_deduct_from_purchased_when_no_allowance - Test deduction when allowance is 0 (basic tier or depleted).
test_insufficient_total_credits_error - Verify error when total balance is insufficient.
class TestAllowanceReset - Test subscription allowance reset on renewal.
test_reset_sets_correct_amount_pro - PRO tier should get $15 allowance on reset.
test_reset_sets_correct_amount_max - MAX tier should get $150 allowance on reset.
class TestSubscriptionCancellation - Test cancellation behavior - allowance forfeited, purchased preserved.
test_allowance_forfeited_on_cancel - Verify allowance is zeroed on cancellation.
test_purchased_credits_preserved_on_cancel - Verify purchased credits remain after cancellation.
class TestAllowanceFromTier - Test getting allowance amounts from tier configuration.
test_get_pro_allowance - PRO tier should return $15 allowance.
test_get_max_allowance - MAX tier should return $150 allowance.
test_get_basic_allowance - Basic tier should return $0 allowance.
References: src/db/trials.py, src/schemas/trials.py
Module: Comprehensive tests for trial management database operations
class TestStartTrial - Test starting trials
test_start_trial_success - Test successfully starting a trial
test_start_trial_custom_duration - Test starting trial with custom duration
test_start_trial_api_key_not_found - Test starting trial with invalid API key
test_start_trial_database_error - Test error handling during trial start
test_start_trial_exception - Test exception handling
class TestGetTrialStatus - Test getting trial status
test_get_trial_status_active - Test getting status for active trial
test_get_trial_status_expired - Test getting status for expired trial
test_get_trial_status_not_trial - Test getting status for non-trial key
test_get_trial_status_api_key_not_found - Test getting status with invalid API key
class TestConvertTrialToPaid - Test converting trials to paid subscriptions
test_convert_trial_to_paid_success - Test successfully converting trial to paid
test_convert_trial_different_plans - Test converting to different plan types
test_convert_trial_api_key_not_found - Test converting with invalid API key
test_convert_trial_exception - Test exception handling during conversion
class TestTrackTrialUsage - Test tracking trial usage
test_track_trial_usage_success - Test successfully tracking trial usage
test_track_trial_usage_custom_requests - Test tracking usage with custom request count
test_track_trial_usage_api_key_not_found - Test tracking usage with invalid API key
class TestTrialAnalytics - Test trial analytics
test_get_trial_analytics_success - Test getting trial analytics
test_get_trial_analytics_no_trials - Test analytics with no trials
test_get_trial_analytics_no_data - Test analytics with no data
test_get_trial_analytics_calculates_averages - Test analytics calculates correct averages
test_get_trial_analytics_exception - Test exception handling in analytics
class TestTrialsIntegration - Test trial workflow integration
test_complete_trial_workflow - Test complete trial workflow: start -> use -> convert
References: src/db/plans.py, src/db/users.py, src/services/daily_usage_limiter.py
Module: Tests for User Credit Updates (Trial Credit: $5 total, $1/day usage limit)
class TestReducedTrialCredits - Test that new users receive $5 trial credits with $1/day usage limit
test_create_enhanced_user_default_credits - Test that default credits are $5 with $1/day limit enforced
test_create_enhanced_user_custom_credits - Test that custom credit amounts can still be specified
class TestDailyLimitEnforcement - Test that daily usage limits are enforced in deduct_credits
test_deduct_credits_checks_daily_limit - Test that deduct_credits calls enforce_daily_usage_limit
test_deduct_credits_raises_on_daily_limit_exceeded - Test that DailyUsageLimitExceeded is converted to ValueError
test_admin_users_bypass_daily_limit - Test that admin users bypass daily usage limits
class TestCreditDeductionWithDailyLimits - Integration tests for credit deduction with daily limits
test_multiple_small_deductions_within_limit - Test multiple small deductions that stay within $1 daily limit
References: src/db/api_keys.py, src/db/credit_transactions.py, src/db/users.py, src/routes/users.py, src/schemas/users.py, src/security/security.py
test_create_enhanced_user_creates_trial_and_primary
test_get_user_prefers_new_api_keys_then_legacy
test_get_user_by_id_and_privy
test_add_credits_and_deduct_credits
test_get_all_users_delete_user_count
test_record_usage_and_metrics
test_admin_monitor_data
test_admin_monitor_data_deduplication - Test that duplicate records in activity_log and usage_records are correctly deduplicated.
test_admin_monitor_data_deduplication_api_key_only - Test deduplication when legacy records only have api_key (no user_id).
test_update_and_get_user_profile
test_get_user_profile_returns_credits_in_cents - Test that get_user_profile returns credits in cents, not dollars.
test_get_user_profile_pro_tier_allowance_in_cents - Test Pro tier user returns $15.00 allowance as 1500 cents.
test_get_user_profile_max_tier_allowance_in_cents - Test Max tier user returns $150.00 allowance as 15000 cents.
test_get_user_profile_basic_tier_zero_allowance - Test Basic tier user returns zero allowance correctly.
test_get_user_profile_fractional_cents - Test that fractional dollar amounts are correctly converted to integer cents.
test_get_user_profile_null_credit_fields - Test that null/missing credit fields default to 0 cents.
test_get_user_profile_large_credits_in_cents - Test that large credit amounts are correctly converted to cents.
test_mark_welcome_email_sent_and_delete_user_account
test_create_enhanced_user_with_privy_id - Test user creation with privy_user_id
test_create_enhanced_user_failure_no_data - Test user creation when insert returns no data
test_create_enhanced_user_exception_handling - Test exception handling in create_enhanced_user
test_get_user_not_found - Test get_user when API key doesn't exist
test_get_user_exception_handling - Test get_user exception handling
test_get_user_by_id_not_found - Test get_user_by_id when user doesn't exist
test_get_user_by_id_exception_handling - Test get_user_by_id exception handling
test_get_user_by_username_not_found - Test get_user_by_username when user doesn't exist
test_get_user_by_username_found - Test get_user_by_username when user exists
test_get_user_by_username_exception_handling - Test get_user_by_username exception handling
test_add_credits_to_user_success - Test add_credits_to_user successfully adds credits
test_add_credits_to_user_exception_handling - Test add_credits_to_user exception handling
test_add_credits_success - Test add_credits function
test_deduct_credits_with_metadata - Test deduct_credits with metadata
test_deduct_credits_user_not_found - Test deduct_credits when user not found
test_deduct_credits_exception_handling - Test deduct_credits exception handling
test_get_all_users_exception_handling - Test get_all_users exception handling
test_delete_user_exception_handling - Test delete_user exception handling
test_get_user_count_exception_handling - Test get_user_count exception handling
test_record_usage_exception_handling - Test record_usage exception handling
test_record_usage_with_latency - Test record_usage with latency parameter
test_get_user_usage_metrics_no_user - Test get_user_usage_metrics when user not found
test_get_user_usage_metrics_exception_handling - Test get_user_usage_metrics exception handling
test_get_admin_monitor_data_exception_handling - Test get_admin_monitor_data exception handling
test_update_user_profile_exception_handling - Test update_user_profile exception handling
test_get_user_profile_not_found - Test get_user_profile when user not found
test_get_user_profile_exception_handling - Test get_user_profile exception handling
test_mark_welcome_email_sent_exception_handling - Test mark_welcome_email_sent exception handling
test_delete_user_account_not_found - Test delete_user_account when user not found
test_delete_user_account_exception_handling - Test delete_user_account exception handling
test_user_cache_hit_on_second_call - Second call to get_user with same API key should use cache (not hit database)
test_user_cache_miss_for_different_keys - Different API keys should have separate cache entries
test_clear_user_cache_specific_key - clear_user_cache(api_key) should clear only that key
test_clear_user_cache_all - clear_user_cache() with no arguments should clear entire cache
test_invalidate_user_cache - invalidate_user_cache should clear cache for specific user
test_user_cache_not_found_user - Cache should not cache None returns for invalid keys
test_is_temporary_api_key_empty - Test _is_temporary_api_key with empty/None values
test_is_temporary_api_key_short_gw_live - Test is_temporary_api_key detects short gw_live keys as temporary
test_is_temporary_api_key_proper_length - Test _is_temporary_api_key does not flag proper-length keys as temporary
test_is_temporary_api_key_non_gw_live_prefix - Test is_temporary_api_key ignores keys without gw_live prefix
test_is_temporary_api_key_realistic_keys - Test _is_temporary_api_key with realistic key formats
test_migrate_legacy_api_key_skips_temporary_keys - Test _migrate_legacy_api_key does not migrate temporary keys
test_get_user_detects_temporary_key_in_legacy - Test get_user correctly detects and handles temporary keys in legacy lookup
References: src/db/webhook_events.py
Module: Comprehensive tests for webhook event database operations with HTTP/2 retry logic
class TestIsEventProcessedWithRetry - Test event checking with HTTP/2 connection error retry
test_is_event_processed_success - Test successful event check (event exists)
test_is_event_processed_not_found - Test event not found
test_is_event_processed_http2_error_retry - Test HTTP/2 connection error triggers retry via execute_with_retry
test_is_event_processed_stream_id_error - Test StreamIDTooLowError is handled by execute_with_retry
test_is_event_processed_generic_error - Test generic database errors
class TestRecordProcessedEventWithRetry - Test event recording with HTTP/2 connection error retry
test_record_processed_event_success - Test successful event recording
test_record_processed_event_no_data_returned - Test recording fails when no data returned
test_record_processed_event_http2_error - Test HTTP/2 connection error during recording
test_record_processed_event_send_headers_error - Test SEND_HEADERS LocalProtocolError
class TestGetProcessedEventWithRetry - Test event retrieval with HTTP/2 connection error retry
test_get_processed_event_success - Test successful event retrieval
test_get_processed_event_not_found - Test event not found
test_get_processed_event_http2_error - Test HTTP/2 connection error during retrieval
class TestCleanupOldEventsWithRetry - Test event cleanup with HTTP/2 connection error retry
test_cleanup_old_events_success - Test successful cleanup
test_cleanup_old_events_no_data - Test cleanup with no events to delete
test_cleanup_old_events_http2_error - Test HTTP/2 connection error during cleanup
test_cleanup_old_events_custom_days - Test cleanup with custom retention period
class TestWebhookEventsIntegration - Integration tests for webhook event idempotency flow
test_idempotency_flow_new_event - Test full flow: check event doesn't exist, then record it
test_idempotency_flow_duplicate_event - Test full flow: detect duplicate event
test_resilience_to_transient_errors - Test that execute_with_retry handles transient errors gracefully
References: src/services/model_transformations.py
Module: End-to-end tests for AllenAI OLMo models via the /v1/chat/completions endpoint.
class TestAllenAIModelsE2E - E2E tests for AllenAI OLMo models via chat completions endpoint.
test_allenai_model_basic_request - Test basic chat completion request for each AllenAI model.
test_allenai_model_streaming - Test streaming chat completion for each AllenAI model.
test_allenai_model_with_system_prompt - Test AllenAI models with system prompt.
test_allenai_model_conversation_history - Test AllenAI models with conversation history.
test_allenai_model_with_temperature - Test AllenAI models with custom temperature.
test_allenai_model_with_top_p - Test AllenAI models with top_p parameter.
test_olmo_instruct_follows_instructions - Test that OLMo instruct model follows instructions.
test_thinking_models_reasoning_task - Test thinking models with a reasoning task.
test_allenai_model_with_openrouter_provider - Test AllenAI models with explicit OpenRouter provider.
test_allenai_model_response_has_usage - Test that AllenAI model responses include usage information.
test_allenai_model_invalid_should_fail - Test that invalid AllenAI model ID returns appropriate error.
class TestAllenAIModelsStreamingE2E - Dedicated streaming tests for AllenAI models.
test_streaming_with_custom_params - Test streaming with custom temperature and max_tokens.
test_streaming_response_structure - Test that streaming responses have correct SSE structure.
class TestAllenAIModelsDeveloperRoleE2E - Tests for AllenAI models with developer role support.
test_allenai_model_with_developer_role - Test AllenAI models with developer role instead of system role.
test_allenai_model_developer_role_streaming - Test streaming with developer role for AllenAI models.
References: src/routes/chat.py
Module: End-to-end tests for /v1/chat/completions endpoint.
class TestChatCompletionsE2E - E2E tests for chat completions endpoint.
test_chat_completions_basic_request - Test basic chat completion request and response.
test_chat_completions_with_system_prompt - Test chat completion with system prompt.
test_chat_completions_with_all_parameters - Test chat completion with all optional parameters.
test_chat_completions_streaming - Test streaming chat completion.
test_chat_completions_with_provider_openrouter - Test chat completion with explicit OpenRouter provider.
test_chat_completions_with_provider_featherless - Test chat completion with Featherless provider.
test_chat_completions_with_provider_fireworks - Test chat completion with Fireworks provider.
test_chat_completions_missing_api_key - Test chat completion without API key.
test_chat_completions_empty_messages - Test chat completion with empty messages array.
test_chat_completions_invalid_role - Test chat completion with invalid message role.
test_chat_completions_empty_content - Test chat completion with empty message content.
test_chat_completions_multiple_messages - Test chat completion with conversation history.
test_chat_completions_with_tools - Test chat completion with function calling tools.
test_chat_completions_response_format_json - Test chat completion with JSON response format.
test_chat_completions_very_long_message - Test chat completion with very long message.
test_chat_completions_session_id_parameter - Test chat completion with session_id query parameter.
test_chat_completions_default_max_tokens - Test that max_tokens defaults to 950.
test_chat_completions_default_temperature - Test that temperature defaults to 1.0.
class TestChatCompletionsDeveloperRoleE2E - E2E tests for chat completions with developer message role.
test_chat_completions_with_developer_role - Test chat completion with developer role message.
test_chat_completions_developer_role_streaming - Test streaming chat completion with developer role.
test_reasoning_models_with_developer_role - Test OpenAI reasoning models (o1/o3) with developer role.
References: src/routes/images.py
Module: End-to-end tests for /v1/images endpoint (Image generation).
class TestImagesE2E - E2E tests for image generation endpoint.
test_images_basic_request - Test basic image generation request and response.
test_images_with_single_image - Test image generation with single image.
test_images_with_multiple_images - Test image generation with multiple images.
test_images_different_sizes - Test image generation with different sizes.
test_images_with_deepinfra_provider - Test image generation with DeepInfra provider.
test_images_with_google_vertex_provider - Test image generation with Google Vertex AI provider.
test_images_with_fal_provider - Test image generation with Fal.ai provider.
test_images_invalid_provider - Test image generation with unsupported provider.
test_images_missing_prompt - Test image generation without prompt.
test_images_empty_prompt - Test image generation with empty prompt.
test_images_missing_api_key - Test image generation without API key.
test_images_very_long_prompt - Test image generation with very long prompt.
test_images_special_characters_in_prompt - Test image generation with special characters in prompt.
test_images_invalid_size - Test image generation with invalid size.
test_images_invalid_number_of_images - Test image generation with invalid n parameter.
test_images_default_size - Test image generation without size (should use default).
test_images_default_provider - Test image generation without provider (should default to deepinfra).
References: src/routes/messages.py
Module: End-to-end tests for /v1/messages endpoint (Anthropic Claude API).
class TestMessagesE2E - E2E tests for messages endpoint.
test_messages_basic_request - Test basic messages API request and response.
test_messages_with_system_prompt - Test messages API with system prompt (Anthropic style).
test_messages_with_all_parameters - Test messages API with all optional parameters.
test_messages_streaming - Test streaming messages API.
test_messages_with_provider_openrouter - Test messages API with explicit OpenRouter provider.
test_messages_with_provider_featherless - Test messages API with Featherless provider.
test_messages_missing_api_key - Test messages API without API key.
test_messages_missing_max_tokens - Test messages API without required max_tokens (Anthropic requirement).
test_messages_invalid_max_tokens - Test messages API with invalid max_tokens.
test_messages_empty_messages - Test messages API with empty messages array.
test_messages_invalid_role - Test messages API with invalid message role (Anthropic only allows user/assistant).
test_messages_empty_content - Test messages API with empty message content.
test_messages_conversation_history - Test messages API with conversation history.
test_messages_with_tools - Test messages API with tool definitions (Claude tool use).
test_messages_with_tool_choice - Test messages API with tool_choice parameter.
test_messages_with_stop_sequences - Test messages API with stop_sequences (Claude feature).
test_messages_with_top_k - Test messages API with top_k parameter (Anthropic-specific).
test_messages_very_long_content - Test messages API with very long message.
test_messages_metadata - Test messages API with metadata parameter.
References: src/routes/ai_sdk.py, src/routes/chat.py
Module: End-to-end tests for /v1/responses endpoint (Unified API).
class TestResponsesE2E - E2E tests for unified responses endpoint.
test_responses_basic_request - Test basic unified responses endpoint request and response.
test_responses_with_all_parameters - Test responses endpoint with all optional parameters.
test_responses_streaming - Test streaming responses endpoint.
test_responses_with_provider - Test responses endpoint with specific provider.
test_responses_with_json_response_format - Test responses endpoint with JSON response format.
test_responses_multimodal_input - Test responses endpoint with multimodal input (text + image).
test_responses_multiple_input_items - Test responses endpoint with multiple input items.
test_responses_missing_api_key - Test responses endpoint without API key.
test_responses_empty_input - Test responses endpoint with empty input array.
test_responses_missing_model - Test responses endpoint without model.
test_responses_with_tools - Test responses endpoint with tool definitions.
test_responses_very_long_input - Test responses endpoint with very long input.
test_responses_default_max_tokens - Test that max_tokens defaults to 950.
test_responses_with_featherless_provider - Test responses endpoint with Featherless provider.
test_responses_with_fireworks_provider - Test responses endpoint with Fireworks provider.
References: src/services/model_transformations.py
Module: End-to-end tests for Simplismart models via the /v1/chat/completions endpoint.
class TestSimplismartModelsE2E - E2E tests for Simplismart models via chat completions endpoint.
test_simplismart_model_basic_request - Test basic chat completion request for each Simplismart model.
test_simplismart_model_streaming - Test streaming chat completion for each Simplismart model.
test_simplismart_model_with_system_prompt - Test Simplismart models with system prompt.
test_simplismart_model_conversation_history - Test Simplismart models with conversation history.
test_simplismart_model_with_temperature - Test Simplismart models with custom temperature.
test_simplismart_model_with_top_p - Test Simplismart models with top_p parameter.
test_simplismart_model_response_has_usage - Test that Simplismart model responses include usage information.
test_simplismart_model_invalid_should_fail - Test that invalid Simplismart model ID returns appropriate error.
class TestSimplismartModelAliasesE2E - E2E tests for Simplismart model aliases.
test_simplismart_alias_resolution - Test that model aliases are correctly resolved.
class TestSimplismartStreamingE2E - Dedicated streaming tests for Simplismart models.
test_streaming_with_custom_params - Test streaming with custom temperature and max_tokens.
test_streaming_response_structure - Test that streaming responses have correct SSE structure.
class TestSimplismartLlamaModelsE2E - Specific tests for Llama models on Simplismart.
test_llama_3_1_8b_instruction_following - Test that Llama 3.1 8B follows instructions.
test_llama_3_3_70b_reasoning_task - Test Llama 3.3 70B with a reasoning task.
class TestSimplismartGemmaModelsE2E - Specific tests for Gemma models on Simplismart.
test_gemma_3_27b_basic - Test Gemma 3 27B basic functionality.
class TestSimplismartQwenModelsE2E - Specific tests for Qwen models on Simplismart.
test_qwen_2_5_32b_basic - Test Qwen 2.5 32B basic functionality.
References: src/routes/chat.py
Module: End-to-end tests for streaming and provider parameter across all endpoints.
class TestStreamingE2E - E2E tests for streaming functionality.
test_streaming_chat_completions - Test streaming on chat completions endpoint.
test_streaming_messages - Test streaming on messages endpoint.
test_streaming_responses - Test streaming on responses endpoint.
test_streaming_with_custom_parameters - Test streaming with custom temperature and tokens.
test_non_streaming_chat_completions - Test non-streaming chat completions.
test_non_streaming_messages - Test non-streaming messages endpoint.
class TestProviderParameterE2E - E2E tests for provider parameter across endpoints.
test_provider_openrouter_chat - Test explicit OpenRouter provider on chat endpoint.
test_provider_featherless_chat - Test explicit Featherless provider on chat endpoint.
test_provider_fireworks_chat - Test explicit Fireworks provider on chat endpoint.
test_provider_together_chat - Test explicit Together provider on chat endpoint.
test_provider_huggingface_chat - Test explicit HuggingFace provider on chat endpoint.
test_provider_openrouter_messages - Test explicit OpenRouter provider on messages endpoint.
test_provider_openrouter_responses - Test explicit OpenRouter provider on responses endpoint.
test_provider_deepinfra_images - Test explicit DeepInfra provider on images endpoint.
test_provider_default_fallback - Test default provider fallback when not specified.
test_provider_auto_detection_from_model_id - Test provider auto-detection from model ID.
test_provider_alias_hug_to_huggingface - Test provider alias normalization (hug -> huggingface).
test_provider_with_streaming - Test provider parameter works with streaming.
test_multiple_providers_in_sequence - Test using different providers in sequence.
References: src/services/model_transformations.py
Module: End-to-end tests for Sybil models via the /v1/chat/completions endpoint.
class TestSybilModelsE2E - E2E tests for Sybil models via chat completions endpoint.
test_sybil_model_basic_request - Test basic chat completion request for each Sybil model.
test_sybil_model_streaming - Test streaming chat completion for each Sybil model.
test_sybil_model_with_tools - Test Sybil model with tool calling (for models that support it).
test_sybil_model_with_json_mode - Test Sybil model with JSON mode (for models that support it).
References: src/routes/health.py, src/services/intelligent_health_monitor.py
Module: Test suite for Gateway Model URL Health Checker script
class TestGatewayHealthChecker - Test suite for gateway health checker
test_gateway_config_exists - Test that gateway configuration is properly defined
test_all_required_gateways_configured - Test that all expected gateways are in the configuration
test_gateway_config_has_required_fields - Test that each gateway config has required fields
test_build_headers_bearer_token - Test header building for bearer token auth
test_build_headers_no_api_key - Test header building when no API key is provided
test_cache_test_with_empty_cache - Test cache testing with empty cache
test_cache_test_with_valid_cache - Test cache testing with valid cached models
test_clear_gateway_cache - Test clearing gateway cache
test_gateway_config_min_expected_models - Test that each gateway has reasonable min_expected_models
test_gateway_urls_are_valid - Test that all gateway URLs are properly formatted
class TestGatewayEndpointChecks - Test gateway endpoint checking functionality
test_endpoint_check_timeout_handling - Test that timeout is handled properly
test_endpoint_check_response_parsing - Test proper response parsing for different formats
class TestGatewayHealthCheckFlow - Test the overall health check flow
test_build_headers_for_all_gateway_types - Test header building for all configured gateway types
test_comprehensive_check_result_structure - Test structure of comprehensive check results
test_gateway_count_consistency - Test that gateway counts add up correctly
class TestGatewayCache - Test gateway cache functionality
test_cache_with_expired_timestamp - Test cache age calculation for expired cache
test_cache_model_count_validation - Test that cache validates minimum model count
class TestGatewayIntegration - Integration tests for gateway checking
test_openrouter_endpoint_reachable - Test that OpenRouter endpoint is reachable
test_script_runs_without_errors - Test that the full script runs without exceptions
References: src/services/model_transformations.py
Module: Integration tests for AllenAI OLMo models.
class TestAllenAIModelsDirectOpenRouter - Test AllenAI models directly via OpenRouter API.
test_model_basic_response - Test that each AllenAI model responds to a basic prompt.
test_model_streaming - Test that each AllenAI model supports streaming.
test_thinking_models_reasoning - Test that thinking models provide reasoning for complex questions.
test_olmo_instruct_follows_instructions - Test that OLMo instruct model follows instructions well.
test_model_with_conversation_history - Test that models handle multi-turn conversations.
class TestAllenAIModelsViaGatewayz - Test AllenAI models via Gatewayz API endpoint.
test_model_via_gatewayz - Test AllenAI models through Gatewayz API.
test_model_streaming_via_gatewayz - Test AllenAI models streaming through Gatewayz API.
References: src/config/supabase_config.py, src/db/users.py, src/enhanced_notification_service.py, src/main.py, src/routes/auth.py, src/schemas/auth.py, src/services/professional_email_templates.py, src/services/referral.py
Module: Integration Tests for Auth/Registration Referral Flow
test_auth_users - Create test users for auth testing
class TestAuthRegistrationReferralIntegration - Test auth/registration integration with referral system
test_new_user_signup_with_referral_code - Test that new user signup WITH referral code:
test_new_user_signup_with_invalid_referral_code - Test that invalid referral code doesn't block signup
test_existing_user_login_ignores_referral_code - Test that existing users logging in don't process referral codes
class TestReferralTrackingIntegration - Test actual referral tracking with database
test_track_referral_signup_creates_pending_record - Test that track_referral_signup creates pending referral record
test_store_referred_by_code_on_signup - Test that referred_by_code is stored correctly
test_multiple_signups_same_code - Test that multiple users can sign up with the same referral code
class TestReferralNotificationIntegration - Test notification sending during signup
test_signup_notification_sent_to_referrer - Test that referrer receives email when someone signs up with their code
test_notification_failure_doesnt_block_signup - Test that notification failures don't block the signup process
test_notification_logs_attempt_before_sending - Test that notification logs attempt before sending email
test_notification_logs_success_with_email - Test that successful notification logs include email address
test_notification_logs_warning_on_failure - Test that failed notification logs warning with email address
class TestReferralBackgroundTaskLogging - Test logging in the background task that processes referral codes
test_background_task_logs_when_referrer_has_no_email - Test that background task logs warning when referrer has no email
test_background_task_logs_success_with_email - Test that background task logs success with referrer email
test_background_task_logs_warning_when_notification_returns_false - Test that background task logs warning when notification returns False
test_background_task_logs_notification_error_with_details - Test that background task logs notification errors with full details
class TestReferralValidationDuringSignup - Test validation logic during signup
test_validate_code_before_signup - Test validating referral code before signup
test_validate_prevents_self_referral_during_signup - Test that validation prevents self-referral during signup
test_validate_checks_usage_limit - Test that validation checks usage limit
References: src/routes.py
Module: Test Braintrust integration
test_braintrust_import - Test that Braintrust can be imported
test_api_key_configured - Test that BRAINTRUST_API_KEY is configured
test_logger_initialization - Test that Braintrust logger can be initialized
test_basic_tracing - Test basic tracing functionality
test_endpoint_tracing_syntax - Test that the chat endpoint tracing syntax is correct
References: src/services/cerebras_client.py
Module: Test Cerebras API with Qwen models
class TestCerebrasQwen - Test suite for Cerebras Qwen model integration
test_cerebras_qwen_basic - Test basic Qwen model inference
test_cerebras_qwen_streaming - Test streaming Qwen model inference
test_cerebras_qwen_system_prompt - Test Qwen model with system prompt
test_cerebras_qwen_multi_turn - Test multi-turn conversation with Qwen model
test_cerebras_qwen_temperature_variation - Test Qwen model with different temperature settings
test_cerebras_qwen_max_tokens - Test Qwen model with different max_tokens settings
test_cerebras_qwen_response_format - Test response format from Qwen model
test_cerebras_qwen_api_direct - Direct test without pytest fixtures - useful for manual testing
References: src/db/users.py, src/main.py, src/services/pricing.py, src/services/provider_failover.py
Module: Integration tests for chat completions error handling.
class TestChatCompletionsErrors - Integration tests for chat completions error handling.
test_invalid_api_key - Test invalid API key returns detailed error.
test_missing_authorization_header - Test missing authorization header returns detailed error.
test_model_not_found - Test model not found returns suggestions.
test_model_typo_suggests_correct_model - Test that typo in model name suggests correct model.
test_empty_messages_array - Test empty messages array error.
test_missing_messages_field - Test missing messages field error.
test_invalid_message_format - Test invalid message format error.
test_invalid_temperature - Test invalid parameter error for temperature.
test_negative_max_tokens - Test invalid max_tokens parameter.
test_insufficient_credits - Test insufficient credits error.
test_request_id_in_all_errors - Test that all errors include request_id.
test_request_id_propagation - Test that provided X-Request-ID is propagated.
test_timestamp_in_errors - Test that all errors include timestamp.
test_docs_url_in_errors - Test that appropriate errors include docs_url.
test_suggestions_in_errors - Test that errors include actionable suggestions.
test_provider_error - Test provider error returns detailed error.
test_streaming_error_format - Test that errors in streaming mode are properly formatted.
test_error_response_structure_complete - Test that error response has all expected fields.
test_concurrent_requests_unique_ids - Test that concurrent requests get unique request IDs.
test_internal_error_detailed - Test that internal errors return detailed responses.
test_no_error_fields_are_none_in_response - Test that no fields with None values are included in response.
References: src/db/chat_history.py
Module: Comprehensive test suite for chat history edge cases and bug scenarios
class TestDuplicateDetection - Tests for duplicate message scenarios
test_exact_duplicate_messages_same_content - Test that exact duplicate messages are detected
test_duplicate_messages_different_timestamps - Test duplicate messages with different timestamps (e.g., retry scenario)
test_similar_but_different_messages - Test that similar but different messages are both saved
class TestHistoryInjection - Tests for conversation history loading and injection
test_history_injection_prepends_correctly - Test that history is prepended to new messages in correct order
test_empty_history_handling - Test handling of session with no history
test_history_respects_user_ownership - Test that users can only access their own chat history
class TestMessageOrdering - Tests for message ordering and sequencing
test_messages_ordered_by_created_at - Test that messages are returned in chronological order
test_alternating_user_assistant_pattern - Test proper conversation flow alternating between user and assistant
class TestEdgeCases - Tests for edge cases and unusual scenarios
test_empty_message_content - Test handling of empty message content
test_very_long_message_content - Test handling of very long messages
test_special_characters_in_content - Test handling of special characters and unicode
test_null_model_field - Test handling when model field is None
test_zero_and_negative_tokens - Test handling of zero and negative token counts
class TestMultimodalContent - Tests for multimodal message content handling
test_text_only_content - Test standard text-only message
test_json_string_content - Test handling of JSON-like string content
class TestSessionManagement - Tests for session creation, updates, and deletion
test_create_session_with_defaults - Test session creation with default values
test_update_session_title - Test updating session title
test_delete_session_soft_delete - Test that delete is a soft delete (sets is_active=False)
test_get_user_sessions_excludes_deleted - Test that deleted sessions are not returned
class TestPagination - Tests for session and message pagination
test_session_pagination - Test pagination of user sessions
test_session_ordering_by_updated_at - Test that sessions are ordered by updated_at desc
class TestSearch - Tests for chat session search
test_search_by_title - Test searching sessions by title
test_search_by_message_content - Test searching sessions by message content
test_search_case_insensitive - Test that search is case-insensitive
class TestStatistics - Tests for chat statistics
test_session_stats_accuracy - Test that session stats are calculated correctly
test_stats_exclude_inactive_sessions - Test that stats exclude deleted (inactive) sessions
class TestConcurrency - Tests for concurrent operations
test_concurrent_message_saves - Test saving messages concurrently to same session
test_concurrent_session_creation - Test creating sessions concurrently
class TestErrorHandling - Tests for error scenarios
test_get_nonexistent_session - Test getting a session that doesn't exist
test_update_nonexistent_session - Test updating a session that doesn't exist
test_delete_nonexistent_session - Test deleting a session that doesn't exist
class TestRealWorldScenarios - Tests for real-world usage patterns
test_full_conversation_flow - Test complete conversation lifecycle
test_multi_session_user - Test user with multiple concurrent sessions
References: src/services/models.py
Module: Test script to verify Chutes.ai integration
test_chutes_models - Test loading Chutes models from the catalog
References: src/db/trials.py, src/db/users.py, src/services/credit_handler.py, src/services/model_transformations.py, src/services/models.py, src/services/pricing.py
Module: Integration tests for credit deduction with specific model types.
class TestCreditDeductionForOpenAIModels - Test credit deduction for OpenAI models (GPT-4o, GPT-3.5, etc.)
test_gpt4o_pricing_lookup - Test that GPT-4o model gets correct pricing
test_gpt4o_without_prefix_uses_alias - Test that 'gpt-4o' (without openai/ prefix) resolves via alias
test_cost_calculation_openai - Test cost calculation for OpenAI models
test_credit_handler_deducts_for_openai - Test that credit handler deducts credits for OpenAI model
class TestCreditDeductionForAnthropicModels - Test credit deduction for Anthropic models (Claude 3, etc.)
test_claude_opus_pricing_lookup - Test that Claude Opus model gets correct pricing
test_claude_sonnet_pricing_lookup - Test that Claude Sonnet model gets correct pricing
test_cost_calculation_anthropic - Test cost calculation for Anthropic Claude Opus
test_credit_handler_deducts_for_anthropic - Test that credit handler deducts credits for Anthropic model
class TestTrialUserCreditHandling - Test that trial users don't get credits deducted
test_trial_user_no_credit_deduction - Test that trial users don't have credits deducted
class TestTrialOverrideForPaidUsers - Test that paid users with stale is_trial flag still get charged
test_paid_user_with_stale_trial_gets_charged - Test that paid users with stale is_trial=True flag get charged
class TestDefaultPricingAlerts - Test that default pricing usage is tracked and alerted
test_unknown_model_uses_default_pricing - Test that unknown models fall back to default pricing and are tracked
test_high_value_model_default_pricing_alert - Test that high-value models (OpenAI, Anthropic) trigger alerts on default pricing
class TestAsyncCostCalculation - Test async cost calculation
test_calculate_cost_async_works - Test that async cost calculation works correctly
test_free_model_returns_zero_cost_async - Test that free models return $0 cost in async version
References: src/main.py
Module: E2E Tests for Coupon Feature
test_api_keys - Create test users with API keys
class TestCouponAdminEndpoints - Test admin coupon management endpoints
test_create_global_coupon - Test creating a global promotional coupon
test_create_user_specific_coupon - Test creating a user-specific coupon
test_create_coupon_invalid_negative_value - Test that negative coupon values are rejected
test_create_user_specific_without_assigned_user - Test that user-specific coupons require assigned_to_user_id
test_list_coupons - Test listing all coupons
test_list_coupons_with_filters - Test listing coupons with filters
test_get_coupon_by_id - Test getting a specific coupon by ID
test_get_nonexistent_coupon - Test getting a coupon that doesn't exist
test_update_coupon - Test updating coupon fields
test_deactivate_coupon - Test deactivating a coupon
class TestCouponUserEndpoints - Test user-facing coupon endpoints
test_get_available_coupons - Test getting available coupons for a user
test_get_available_coupons_unauthorized - Test that getting available coupons requires authentication
test_redeem_global_coupon - Test redeeming a global coupon
test_redeem_user_specific_coupon - Test redeeming a user-specific coupon
test_redeem_coupon_twice_rejected - Test that redeeming the same coupon twice is rejected
test_redeem_invalid_coupon - Test redeeming an invalid coupon code
test_redeem_expired_coupon - Test that expired coupons cannot be redeemed
test_redeem_deactivated_coupon - Test that deactivated coupons cannot be redeemed
test_get_redemption_history - Test getting user redemption history
test_get_redemption_history_with_limit - Test getting redemption history with limit parameter
class TestCouponAnalytics - Test coupon analytics endpoints
test_get_coupon_analytics - Test getting analytics for a specific coupon
test_get_system_stats - Test getting system-wide coupon statistics
class TestCouponEdgeCases - Test edge cases and error scenarios
test_create_coupon_without_admin_key - Test that creating coupons requires admin API key
test_redeem_without_authentication - Test that redeeming requires authentication
test_wrong_user_cannot_redeem_user_specific_coupon - Test that user-specific coupons can only be redeemed by the assigned user
References: src/config/supabase_config.py, src/main.py, src/services/referral.py
Module: E2E Tests for Referral Feature
test_users - Create test users with API keys for referral testing
class TestEndToEndReferralFlow - End-to-end tests for complete referral flow
test_complete_referral_flow_success - Test complete successful referral flow:
test_referral_stats_accuracy - Test that referral stats are accurate after multiple referrals
test_referral_bonus_minimum_purchase_enforcement - Test that referral bonus requires minimum $10 purchase
test_self_referral_prevention - Test that users cannot use their own referral code
test_max_referral_uses_enforcement - Test that referral codes cannot exceed maximum uses
test_one_referral_code_per_user - Test that users can only use one referral code
test_referral_code_uniqueness - Test that all generated referral codes are unique
test_bonus_only_on_first_purchase - Test that referral bonus only applies on first purchase
class TestReferralEdgeCases - Test edge cases and error scenarios
test_invalid_referral_code - Test handling of invalid referral code
test_deleted_referrer - Test handling when referrer is deleted
test_partial_credit_failure_handling - Test handling when credit addition fails for one user
class TestReferralAPIEndpoints - Test referral API endpoints with real database
test_get_referral_code_endpoint - Test GET /referral/code endpoint
test_get_referral_stats_endpoint - Test GET /referral/stats endpoint
test_validate_referral_endpoint - Test POST /referral/validate endpoint
test_generate_referral_endpoint - Test POST /referral/generate endpoint
References: src/services/model_transformations.py
Module: Test end-to-end normalization with actual provider clients
test_fireworks_with_transform - Test that Fireworks accepts model IDs after transformation
test_openrouter_with_transform - Test that OpenRouter still works with transformed model IDs
References: src/db/activity.py, src/db/api_keys.py, src/db/chat_history.py, src/db/payments.py, src/db/plans.py, src/db/ranking.py, src/db/rate_limits.py, src/db/users.py, src/main.py, src/services/openrouter_client.py, src/services/rate_limiting.py, src/services/trial_validation.py
Module: Comprehensive endpoint regression test suite.
class TestHealthEndpoints - Test health check and status endpoints
test_health_endpoint_exists - Regression: GET /health must exist and return 200
test_ping_endpoint_exists - Regression: GET /ping must exist
test_root_endpoint_exists - Regression: GET / must exist
class TestAuthenticationEndpoints - Test authentication and user profile endpoints
test_user_balance_endpoint_exists - Regression: GET /user/balance must exist
test_user_profile_endpoint_exists - Regression: GET /user/profile must exist
test_user_monitor_endpoint_exists - Regression: GET /user/monitor must exist
class TestChatCompletionsEndpoints - Test chat completion endpoints - CRITICAL for business
test_v1_chat_completions_endpoint_exists - CRITICAL: POST /v1/chat/completions must exist and process requests
test_v1_chat_completions_streaming_exists - CRITICAL: POST /v1/chat/completions with stream=true must work
class TestUnifiedResponsesEndpoint - Test the new /v1/responses endpoint (OpenAI unified API)
test_v1_responses_endpoint_exists - CRITICAL: POST /v1/responses must exist (unified API)
test_v1_responses_with_json_format - Regression: /v1/responses must support response_format parameter
class TestAnthropicMessagesEndpoint - Test Anthropic Messages API endpoint (/v1/messages) - Claude compatible
test_v1_messages_endpoint_exists - CRITICAL: POST /v1/messages must exist (Anthropic/Claude API)
test_v1_messages_with_system_parameter - Regression: /v1/messages must support separate 'system' parameter
class TestAPIKeyEndpoints - Test API key management endpoints
test_list_api_keys_endpoint_exists - Regression: GET /user/api-keys must exist
test_create_api_key_endpoint_exists - Regression: POST /user/api-keys must exist
class TestPaymentEndpoints - Test Stripe payment endpoints
test_stripe_checkout_session_endpoint_exists - Regression: Stripe checkout session endpoint must exist
test_list_payments_endpoint_exists - Regression: Payments list endpoint must exist
class TestChatHistoryEndpoints - Test chat history and session management endpoints
test_list_chat_sessions_endpoint_exists - Regression: Chat sessions list endpoint must exist
test_create_chat_session_endpoint_exists - Regression: Chat session creation endpoint must exist
test_get_chat_session_endpoint_exists - Regression: GET /chat-history/sessions/{session_id} must exist
class TestCatalogEndpoints - Test model catalog endpoints
test_catalog_models_endpoint_exists - Regression: GET /v1/models must exist
test_catalog_providers_endpoint_exists - Regression: GET /v1/provider must exist
class TestRankingEndpoints - Test model and app ranking endpoints
test_ranking_models_endpoint_exists - Regression: GET /ranking/models must exist
test_ranking_apps_endpoint_exists - Regression: GET /ranking/apps must exist
class TestAdminEndpoints - Test admin endpoints
test_admin_list_users_endpoint_exists - Regression: Admin list users endpoint must exist
test_admin_add_credits_endpoint_exists - Regression: Admin add credits endpoint must exist
class TestRateLimitEndpoints - Test rate limiting endpoints
test_get_rate_limits_endpoint_exists - Regression: Rate limits endpoint must exist
class TestActivityEndpoints - Test activity tracking endpoints
test_get_activity_endpoint_exists - Regression: Activity endpoint must exist
class TestNotificationEndpoints - Test notification endpoints
test_get_notifications_endpoint_exists - Regression: Notifications endpoint must exist
References: src/db/api_keys.py, src/db/chat_history.py, src/db/credit_transactions.py, src/db/payments.py, src/db/plans.py, src/db/ranking.py, src/db/rate_limits.py, src/db/users.py, src/main.py, src/routes/chat.py, src/services/openrouter_client.py, src/services/rate_limiting.py, src/services/trial_validation.py
Module: Comprehensive endpoint tests to ensure all critical endpoints exist and function correctly.
class TestHealthEndpoints - Test health check and basic endpoints
test_health_check - Test GET /health returns 200
test_root_endpoint - Test GET / returns welcome message
test_ping_endpoint - Test GET /ping returns pong
class TestAuthEndpoints - Test authentication endpoints
test_auth_endpoint_exists - Test POST /auth endpoint exists
test_user_balance_endpoint - Test GET /user/balance requires authentication
class TestChatEndpoints - Test chat completion endpoints - the most critical functionality
test_chat_completions_endpoint_exists - Test POST /v1/chat/completions endpoint exists and processes requests
test_chat_completions_requires_auth - Test /v1/chat/completions requires authentication
class TestUserEndpoints - Test user management endpoints
test_credit_transactions_endpoint - Test GET /user/credit-transactions endpoint exists
test_user_profile_endpoint - Test GET /user/profile endpoint exists
test_user_monitor_endpoint - Test GET /user/monitor endpoint exists
class TestPaymentEndpoints - Test payment and Stripe endpoints
test_stripe_checkout_session_endpoint - Test POST /api/stripe/checkout-session endpoint exists
test_stripe_webhook_endpoint_exists - Test POST /api/stripe/webhook endpoint exists
test_payments_list_endpoint - Test GET /api/stripe/payments endpoint exists
class TestRankingEndpoints - Test ranking endpoints
test_ranking_models_endpoint - Test GET /ranking/models endpoint exists
test_ranking_apps_endpoint - Test GET /ranking/apps endpoint exists
class TestAPIKeyEndpoints - Test API key management endpoints
test_list_api_keys_endpoint - Test GET /api-keys endpoint exists
test_create_api_key_endpoint - Test POST /api-keys endpoint exists
class TestAdminEndpoints - Test admin endpoints
test_admin_list_users_endpoint - Test GET /admin/users endpoint exists
test_admin_add_credits_endpoint - Test POST /admin/users/{user_id}/credits endpoint exists
class TestCatalogEndpoints - Test model catalog endpoints
test_catalog_models_endpoint - Test GET /catalog/models endpoint exists
test_catalog_providers_endpoint - Test GET /catalog/providers endpoint exists
test_public_models_endpoint - Test GET /models endpoint mirrors catalog route
test_public_model_detail_endpoint_with_hf_developer - Test GET /models/{developer}/{model} supports Hugging Face-style slugs
test_catalog_models_groq_gateway - Test GET /catalog/models with Groq gateway parameter
test_public_models_groq_gateway - Test GET /models with Groq gateway parameter
class TestChatHistoryEndpoints - Test chat history endpoints
test_chat_sessions_list_endpoint - Test GET /chat/sessions endpoint exists
test_create_chat_session_endpoint - Test POST /chat/sessions endpoint exists
class TestIntegration - Integration tests for critical user flows
test_full_chat_flow - Test complete chat completion flow: auth -> credit check -> API call -> deduct credits
References: src/services/fireworks_client.py
Module: Test Fireworks API directly to diagnose the issue
test_fireworks_api - Test Fireworks API directly
References: src/db/activity.py, src/db/api_keys.py, src/db/plans.py, src/db/rate_limits.py, src/db/users.py, src/main.py, src/routes/chat.py, src/services/pricing.py, src/services/rate_limiting.py, src/services/trial_validation.py
Module: Integration tests for function calling/tools support
class TestFunctionCallingIntegration - Integration tests for function calling
test_chat_completions_with_tools - Test that tools parameter is passed through to provider
test_huggingface_with_tools - Test that HuggingFace receives tools parameter
References: src/db/plans.py, src/db/users.py, src/main.py, src/services/google_models_config.py, src/services/google_vertex_client.py, src/services/rate_limiting.py, src/services/trial_validation.py
Module: End-to-End Integration Test for Google Vertex AI
test_user - Create a test user with initial credits
class TestGoogleVertexE2E - End-to-end integration tests for Google Vertex AI
test_01_vertex_credentials_available - Test that Google Vertex credentials are properly configured
test_02_create_user_and_purchase_credits - Test user creation and credit purchase
test_03_call_all_gemini_models - Test all Google Gemini models via Vertex AI
test_04_verify_streaming_support - Test streaming support for Google Gemini models
class TestVertexAIDirectCall - Direct tests of Vertex AI client without mocking
test_vertex_client_can_make_request - Test that the Vertex AI client can make a direct API call
References: src/services/huggingface_client.py
Module: Test HuggingFace case sensitivity with actual working models
test_hf_case
References: src/services/huggingface_models.py
Module: Test script for Hugging Face Models API integration.
test_fetch_models - Test fetching models from Hugging Face API
test_cache - Test caching mechanism
test_fetch_hug_wrapper - Test the fetch_models_from_hug wrapper function
test_search - Test search functionality
test_model_info - Test fetching specific model info
References: src/services/model_transformations.py
Module: Test the model transformation feature
test_api - Test the API with different model formats
References: src/services/model_transformations.py
Module: Test Near AI API with Qwen3-30B-A3B-Instruct-2507 model to verify timeout fix
test_near_qwen_api - Test Near AI API with Qwen3-30B model
References: src/services/model_transformations.py, src/services/openrouter_client.py
Module: Live test for openrouter/auto endpoint
test_openrouter_auto - Send a test message to openrouter/auto
test_code_only - Test just the code paths without making API calls
References: src/services/openrouter_client.py
Module: Test OpenRouter directly to check if sao10k/l3-euryale-70b is available
test_model
References: src/config/supabase_config.py, src/main.py, src/services/payments.py, src/services/referral.py
Module: Integration Tests for Payment Webhook Referral Bonus Application
test_users_for_webhook - Create test users for webhook testing
class TestPaymentWebhookReferralIntegration - Test payment webhook integration with referral system
test_checkout_completed_triggers_referral_bonus - Test that checkout.session.completed webhook triggers referral bonus
test_checkout_below_minimum_no_bonus - Test that purchases < $10 do NOT trigger referral bonus
test_second_purchase_no_bonus - Test that second purchase does NOT trigger referral bonus
test_payment_without_referral_code - Test that users without referral code just get payment credits
test_payment_succeeds_even_if_referral_fails - CRITICAL: Test that payment processing succeeds even if referral bonus fails
class TestWebhookEdgeCases - Test webhook edge cases
test_webhook_with_missing_metadata - Test webhook handling when metadata is missing
test_webhook_with_invalid_user_id - Test webhook handling with non-existent user ID
References: src/services/model_transformations.py
Module: Test case sensitivity requirements for all gateway providers
test_provider_case_sensitivity
References: src/db/users.py, src/enhanced_notification_service.py, src/services/professional_email_templates.py, src/services/referral.py
Module: Comprehensive tests for the enhanced referral system including:
class TestReferralSignupTracking - Test referral tracking when users sign up
test_track_referral_signup_success - Test successful tracking of referral signup
test_track_referral_signup_invalid_code - Test tracking with invalid referral code
test_track_referral_signup_self_referral - Test that self-referral is blocked
test_track_referral_signup_max_uses_reached - Test that codes with max uses cannot be used
class TestReferralNotifications - Test referral notification system
test_send_referral_signup_notification - Test sending signup notification to referrer
test_send_referral_bonus_notification - Test sending bonus notification to both users
class TestReferralBonusWithPendingRecord - Test bonus application updates pending referral records
test_apply_bonus_updates_pending_referral - Test that bonus application updates existing pending referral record
class TestTrialCredits - Test that new users get $5 trial credits
test_new_user_gets_trial_credits - Test new users receive $5 trial credits
class TestEndToEndReferralFlow - End-to-end tests for complete referral flow
test_complete_referral_flow - Test complete referral flow:
test_constants - Verify referral system constants
References: src/config/supabase_config.py, src/services/referral.py
Module: Database Integrity and Concurrency Tests for Referral System
test_db_users - Create test users for database testing
class TestReferralCodeUniqueness - Test that referral codes are unique in database
test_referral_code_uniqueness_constraint - Test that database enforces referral code uniqueness
test_generate_unique_codes_at_scale - Test that code generation produces unique codes at scale
class TestConcurrentReferralUsage - Test concurrent usage scenarios
test_concurrent_signup_same_code - Test multiple users signing up with same code concurrently
test_concurrent_bonus_application - Test concurrent bonus applications (edge case)
class TestReferralUsageLimits - Test usage limit enforcement with database
test_usage_limit_enforced_in_database - Test that database correctly tracks and enforces usage limits
class TestReferralStatusTransitions - Test referral status transitions in database
test_pending_to_completed_transition - Test that referral status correctly transitions from pending to completed
test_cannot_complete_twice - Test that referral cannot be completed twice
class TestDataIntegrity - Test data integrity and consistency
test_orphaned_referral_prevention - Test that referral records reference valid users
test_referral_code_persistence - Test that referral codes persist correctly in database
test_referred_by_code_persistence - Test that referred_by_code persists correctly
class TestCreditTransactionIntegrity - Test credit transaction integrity
test_credit_transactions_created_for_bonuses - Test that credit transactions are created when bonuses are applied
References: src/main.py, src/routes/chat.py, src/services/model_transformations.py, src/services/multi_provider_registry.py, src/services/provider_selector.py
Module: Comprehensive integration tests for the routing refactor
class TestProviderSelectorBasics - Test basic ProviderSelector functionality
test_selector_initialization - Test that selector initializes correctly
test_get_model_providers - Test getting available providers for a model
test_check_provider_health - Test checking provider health status
class TestMultiProviderRegistry - Test multi-provider registry functionality
test_registry_initialization - Test that registry initializes and has models
test_get_model - Test retrieving a model from registry
test_select_provider - Test provider selection logic
test_get_fallback_providers - Test getting fallback providers
class TestProviderFailover - Test automatic provider failover
test_execute_with_failover_success - Test successful execution with failover
test_execute_with_failover_retry - Test failover retries with different providers
test_execute_with_failover_all_fail - Test when all providers fail
class TestCircuitBreaker - Test circuit breaker health tracking
test_health_tracker_records_success - Test recording successful requests
test_health_tracker_records_failure - Test recording failed requests
test_circuit_breaker_opens_after_threshold - Test circuit breaker opens after failure threshold
class TestModelPrompting - Test actual model prompting through the API
test_prompt_gpt4_through_api - Test prompting GPT-4 through the chat API
test_model_routing_with_mock_providers - Test model routing with mocked provider responses
class TestProviderSpecificRouting - Test routing to specific providers
test_route_to_openrouter - Test explicit routing to OpenRouter
test_route_gemini_model - Test routing Google Gemini model
class TestModelTransformation - Test model ID transformation and provider detection
test_detect_provider_from_model_id - Test automatic provider detection from model ID
class TestCatalogIntegration - Test integration with model catalog
test_get_models_catalog - Test retrieving models catalog
test_catalog_includes_multi_provider_models - Test that catalog includes multi-provider model info
References: src/routes/chat.py
Module: Comprehensive streaming test - Smoke test for production validation
test_streaming_response - Test 1: Streaming Response
test_non_streaming_response - Test 2: Non-Streaming Response (for comparison)
test_all_smoke_tests_summary - Final summary after all smoke tests
References: src/main.py, src/routes/payments.py, src/schemas/payments.py, src/services/payments.py
Module: Integration tests for Stripe webhook metadata handling
class TestCheckoutSessionMetadata - Test metadata handling in checkout session creation
test_checkout_session_includes_credits_cents_field - Verify checkout session metadata includes 'credits_cents' field
test_checkout_session_includes_backward_compatible_credits_field - Verify checkout session metadata includes 'credits' field for backward compatibility
class TestWebhookMetadataHandling - Test webhook processing with metadata
test_webhook_parses_credits_cents_field - Verify webhook correctly parses credits_cents from metadata
test_webhook_handles_missing_credits_cents_fallback_to_credits - Verify webhook falls back to 'credits' field if 'credits_cents' missing
test_webhook_always_marked_processed_even_on_error - Verify webhook is marked as processed even if handler fails
class TestWebhookHttpStatus - Test webhook HTTP status code handling
test_webhook_endpoint_returns_200_on_success - Verify webhook endpoint returns HTTP 200 on success
test_webhook_endpoint_returns_200_on_error - Verify webhook endpoint returns HTTP 200 even on processing errors
class TestMetadataExtraction - Test utility functions for metadata extraction
test_coerce_to_int_handles_string_numbers - Verify _coerce_to_int correctly converts string numbers
test_coerce_to_int_handles_float_numbers - Verify _coerce_to_int correctly converts floats
test_coerce_to_int_handles_none - Verify _coerce_to_int handles None gracefully
test_metadata_to_dict_handles_various_formats - Verify _metadata_to_dict handles various metadata formats
References: src/config/config.py, src/config/supabase_config.py
Module: Integration tests for Supabase authentication via httpx client.
class TestSupabaseHttpxClientAuth - Integration tests that verify the httpx client can authenticate with Supabase.
test_httpx_client_with_auth_headers_can_query_supabase - Test that an httpx client configured like our production client can query Supabase.
test_httpx_client_without_auth_headers_fails - Test that an httpx client WITHOUT auth headers fails to authenticate.
test_httpx_client_with_only_apikey_header_succeeds - Test that apikey header alone is sufficient for read operations.
test_supabase_client_initialization_succeeds - Test that the actual Supabase client can be initialized and query the database.
test_postgrest_session_has_correct_headers - Test that after initialization, the postgrest session has the correct auth headers.
class TestSupabaseConnectionHealth - Health check tests for Supabase connection.
test_can_reach_supabase_health_endpoint - Test basic connectivity to Supabase.
test_supabase_url_has_correct_format - Test that SUPABASE_URL is correctly formatted.
References: src/routes/chat.py
Module: Test script to debug DeepSeek R1 thinking tags in streaming
test_thinking_tags_in_stream - Test that thinking tags are properly passed through in streaming responses
References: src/config/usage_limits.py, src/db/plans.py, src/db/users.py, src/services/daily_usage_limiter.py
Module: Integration Tests for Trial Credits with Daily Usage Limits
class TestTrialCreditAllocation - Test that trial users receive correct credit allocation
test_new_user_receives_5_dollar_credits - Test that new users receive $5 in trial credits
test_trial_credits_configuration - Test that configuration is set correctly
class TestDailyLimitDuringTrial - Test that daily usage limits are enforced during trial period
test_trial_user_limited_to_1_dollar_per_day - Test that trial users can only use $1/day even with $5 credits
test_multiple_small_requests_within_daily_limit - Test multiple small API calls that stay within $1/day limit
class TestTrialPeriodScenario - Test realistic 3-day trial period scenario
test_three_day_trial_usage_pattern - Simulate a user using $1/day for 3 days during trial
class TestPostTrialCredits - Test that users can continue using remaining credits after trial
test_post_trial_credits_available - Test that $2 remaining credits can be used after trial expires
class TestFraudMitigation - Test that daily limits prevent fraud scenarios
test_bot_cannot_drain_credits_instantly - Test that a bot cannot drain all $5 credits in one day
class TestConfigurationValues - Test that configuration values are correctly set
test_trial_credits_is_5_dollars - Verify TRIAL_CREDITS_AMOUNT is $5
test_daily_limit_is_1_dollar - Verify DAILY_USAGE_LIMIT is $1
test_trial_duration_is_3_days - Verify TRIAL_DURATION_DAYS is 3
test_trial_daily_limit_is_1_dollar - Verify TRIAL_DAILY_LIMIT is $1
References: src/db/models_catalog_db.py, src/main.py, src/services/models.py
Module: Basic integration test for unique models feature.
class TestUniqueModelsBasic - Basic tests for unique models functionality.
test_database_layer_get_unique_models - Test database layer can fetch unique models.
test_database_layer_transformation - Test database-to-API transformation.
test_service_layer_unique_models_cache - Test service layer unique models caching.
test_service_layer_get_cached_models_flat_mode - Test backward compatibility - flat mode.
test_service_layer_get_cached_models_unique_mode - Test unique mode via get_cached_models.
test_route_layer_default_behavior - Test /models endpoint default behavior (backward compatibility).
test_route_layer_unique_models_false - Test /models endpoint with unique_models=false.
test_route_layer_unique_models_true - Test /models endpoint with unique_models=true.
test_route_layer_unique_models_with_specific_gateway - Test that unique_models is ignored for provider-specific queries.
test_deduplication_check - Test that unique_models=true actually deduplicates.
References: src/config.py, src/routes.py, src/services/model_transformations.py, src/services/provider_failover.py, src/services/vercel_ai_gateway_client.py
Module: Test script to verify Vercel AI Gateway integration
test_vercel_config - Test that Vercel AI Gateway configuration is present
test_vercel_client_import - Test that the Vercel AI Gateway client can be imported
test_vercel_model_mappings - Test Vercel AI Gateway model ID transformations
test_vercel_provider_detection - Test that Vercel AI Gateway models are properly detected
test_failover_chain - Test that Vercel AI Gateway is in the failover chain
test_chat_route_integration - Test that Vercel AI Gateway is integrated in the chat route
References: src/middleware/auto_sentry_middleware.py
Module: Tests for Auto-Sentry Middleware
class TestAutoSentryMiddleware - Test Auto-Sentry Middleware functionality
test_successful_request_no_sentry_capture - Test that successful requests don't trigger Sentry
test_error_captured_to_sentry - Test that errors are captured to Sentry with context
test_request_context_extraction - Test that request context is properly extracted
test_endpoint_type_detection_inference - Test endpoint type detection for inference endpoints
test_endpoint_type_detection_payment - Test endpoint type detection for payment endpoints
test_endpoint_type_detection_auth - Test endpoint type detection for auth endpoints
test_sensitive_headers_sanitized - Test that sensitive headers are sanitized
test_revenue_critical_tag - Test that revenue-critical endpoints are tagged
test_http_exception_handling - Test that HTTPExceptions are not captured by middleware to avoid duplicates
test_http_exception_filtering_in_middleware - Test that 4xx HTTPException is filtered in middleware exception handler
test_http_exception_various_status_codes - Test that all HTTPExceptions (4xx and 5xx) are filtered to avoid duplicates
test_regular_exception_still_captured - Test that non-HTTPException errors are still captured
test_sentry_unavailable_no_error - Test that middleware works even when Sentry is unavailable
test_slow_request_breadcrumb - Test that slow requests generate breadcrumbs
test_middleware_determines_endpoint_types_correctly - Test endpoint type determination logic
test_middleware_categorizes_http_errors_correctly - Test HTTP error categorization
class TestAutoSentryMiddlewareEdgeCases - Test edge cases and error conditions
test_missing_client_info - Test handling when request.client is None
test_empty_headers - Test handling of empty headers
test_sanitize_headers_edge_cases - Test header sanitization with various input
References: src/middleware/security_middleware.py
Module: Tests for Security Middleware
class TestVelocityModeActivation - Test velocity mode activation logic
test_velocity_mode_not_activated_below_threshold - Test that velocity mode does not activate when error rate is below threshold
test_velocity_mode_activated_above_threshold - Test that velocity mode activates when error rate exceeds threshold
test_velocity_mode_not_activated_insufficient_samples - Test that velocity mode requires minimum sample size
test_velocity_mode_activation_increments_counter - Test that activation counter increments
class TestVelocityModeDeactivation - Test velocity mode deactivation logic
test_velocity_mode_deactivates_after_cooldown - Test that velocity mode deactivates after cooldown period
test_velocity_mode_stays_active_during_cooldown - Test that velocity mode stays active during cooldown
class TestErrorClassification - Test error classification logic
test_4xx_errors_not_counted - Test that 4xx errors (client errors) are not counted as system errors
test_5xx_errors_counted - Test that 5xx errors (server errors) are counted
test_499_slow_requests_counted - Test that 499 errors with slow duration (>5s) are counted
test_499_fast_requests_not_counted - Test that 499 errors with fast duration (<5s) are not counted
class TestRateLimitCalculation - Test rate limit calculations with velocity mode
test_effective_limit_normal_mode - Test that limits are normal when velocity mode is inactive
test_effective_limit_velocity_mode - Test that limits are reduced during velocity mode
test_effective_limit_minimum_of_one - Test that effective limit never goes below 1
class TestAuthenticatedUserExemption - Test authenticated user exemption from IP-based rate limiting
test_bearer_token_detected - Test that Bearer token format is detected
test_gw_api_key_detected - Test that Gatewayz API key format is detected
test_generic_api_key_detected - Test that generic long API keys are detected
test_no_authorization_header - Test that request without auth header is not authenticated
test_short_authorization_header - Test that short auth headers are not considered authenticated
class TestIPTierDetection - Test datacenter IP detection
test_datacenter_user_agent_detected - Test that datacenter IPs are detected by user agent
test_proxy_headers_detected - Test that proxy headers indicate datacenter IP
test_residential_ip_not_detected_as_datacenter - Test that residential IPs are not flagged as datacenter
class TestFingerprintGeneration - Test behavioral fingerprint generation
test_fingerprint_generation - Test that fingerprint is generated from headers
test_same_headers_produce_same_fingerprint - Test that identical headers produce identical fingerprints
test_different_headers_produce_different_fingerprints - Test that different headers produce different fingerprints
class TestRequestOutcomeRecording - Test request outcome recording and log cleanup
test_request_outcome_recorded - Test that request outcomes are recorded
test_old_entries_cleaned - Test that old entries are removed from request log
class TestRateLimitHeaders - Test that rate limit headers are returned in 429 responses
test_rate_limit_headers_present - Test that 429 responses include rate limit headers
class TestIntegrationScenarios - Integration test scenarios
test_health_endpoint_bypasses_security - Test that health endpoints bypass security checks
test_normal_request_allowed - Test that normal requests are allowed
class TestVelocityModeConfiguration - Test velocity mode configuration constants
test_configuration_constants - Test that configuration constants are set correctly
test_ip_limit_constants - Test that IP limit constants are set correctly
class TestEdgeCases - Test edge cases and error conditions
test_empty_request_log - Test that velocity mode doesn't activate with empty log
test_deque_max_length_respected - Test that request log respects maxlen (10000)
test_client_ip_extraction - Test client IP extraction from X-Forwarded-For
References: src/middleware/selective_gzip_middleware.py
Module: Tests for Selective GZip Middleware
class TestSelectiveGZipMiddleware - Test SelectiveGZipMiddleware functionality
test_small_json_not_compressed - Test that small JSON responses are NOT compressed (below minimum_size)
test_large_json_compressed - Test that large JSON responses ARE compressed
test_sse_stream_not_compressed - Test that SSE streaming responses are NOT compressed
test_sse_stream_without_explicit_headers_not_compressed - Test that SSE streaming is NOT compressed even without explicit headers
test_ndjson_stream_not_compressed - Test that NDJSON streaming responses are NOT compressed
test_binary_stream_with_no_buffering_header_not_compressed - Test that responses with X-Accel-Buffering: no are NOT compressed
test_no_gzip_without_accept_encoding - Test that responses are NOT compressed when client doesn't accept gzip
test_vary_header_added_for_compressed_response - Test that Vary: Accept-Encoding header is added for compressed responses
class TestSelectiveGZipMiddlewareEdgeCases - Test edge cases for SelectiveGZipMiddleware
test_streaming_media_types_constant - Test that STREAMING_MEDIA_TYPES contains expected types
test_compress_method - Test the internal _compress method
class TestStreamingHeadersIntegration - Test that streaming headers are correctly set in production-like scenarios
test_sse_headers_complete - Test that all expected SSE headers are present
test_chunks_received_individually - Test that SSE chunks can be received as they stream
References: src/db/activity.py, src/main.py, src/routes/activity.py, src/security/deps.py
Module: Comprehensive tests for activity tracking API endpoints
class TestActivityStatsEndpoint - Test /user/activity/stats endpoint
test_get_activity_stats_default - Test getting activity stats with defaults
test_get_activity_stats_with_days - Test getting activity stats with days parameter
test_get_activity_stats_with_date_range - Test getting activity stats with date range
test_get_activity_stats_empty - Test getting stats when no activity exists
test_get_activity_stats_error - Test error handling in stats endpoint
test_get_activity_stats_days_validation - Test validation for days parameter
class TestActivityLogEndpoint - Test /user/activity/log endpoint
test_get_activity_log_default - Test getting activity log with defaults
test_get_activity_log_with_pagination - Test activity log with pagination
test_get_activity_log_with_page_number - Test activity log with page number
test_get_activity_log_with_date_filters - Test activity log with date filtering
test_get_activity_log_with_model_filter - Test activity log with model filter
test_get_activity_log_with_provider_filter - Test activity log with provider filter
test_get_activity_log_combined_filters - Test activity log with multiple filters
test_get_activity_log_empty - Test getting empty activity log
test_get_activity_log_error - Test error handling in log endpoint
test_get_activity_log_limit_validation - Test validation for limit parameter
test_get_activity_log_offset_validation - Test validation for offset parameter
test_get_activity_log_page_validation - Test validation for page parameter
class TestActivityAuthentication - Test authentication requirements
test_stats_requires_authentication - Test that stats endpoint requires authentication
test_log_requires_authentication - Test that log endpoint requires authentication
class TestActivityIntegration - Test activity endpoint integration scenarios
test_stats_and_log_consistency - Test that stats and log endpoints return consistent data
test_pagination_workflow - Test pagination workflow across multiple pages
test_date_range_filtering - Test date range filtering across stats endpoint
References: src/db/rate_limits.py, src/db/users.py, src/enhanced_notification_service.py, src/main.py, src/routes/admin.py, src/schemas/admin.py, src/security/deps.py
Module: Tests for Admin Route Endpoints
class TestUserCreation - Test user creation endpoint
test_create_user_success - Successfully create a new user
test_create_user_invalid_environment - Create user fails with invalid environment tag
test_create_user_missing_fields - Create user fails with missing required fields
class TestAdminAuthentication - Test admin authentication and authorization
test_admin_endpoint_requires_authentication - Admin endpoint rejects requests without authentication
test_admin_endpoint_rejects_non_admin_user - Regular user cannot access admin endpoints
test_admin_endpoint_accepts_valid_admin - Admin user can access admin endpoints
class TestCreditManagement - Test credit management operations
test_add_credits_success - Admin can add credits to user
test_add_credits_user_not_found - Add credits fails when user not found
test_add_negative_credits - Admin can add negative credits (deduct)
class TestRateLimitManagement - Test rate limit management
test_set_rate_limits_success - Admin can set user rate limits
class TestSystemOperations - Test system operations
test_get_all_users - Admin can view all users
class TestAdminValidation - Test admin endpoint validation
test_add_credits_requires_api_key - Add credits requires api_key field
test_add_credits_requires_credits_amount - Add credits requires credits amount
class TestAdminEdgeCases - Test edge cases
test_add_zero_credits - Adding zero credits should work
class TestPricingSchedulerStatus - Test GET /admin/pricing/scheduler/status endpoint (Phase 3)
test_get_scheduler_status_success - Admin can get scheduler status
test_get_scheduler_status_requires_admin - Scheduler status requires admin role
test_get_scheduler_status_requires_authentication - Scheduler status requires authentication
test_get_scheduler_status_handles_error - Scheduler status handles errors gracefully
class TestPricingSchedulerTrigger - Test POST /admin/pricing/scheduler/trigger endpoint (Phase 3)
test_trigger_manual_sync_success - Admin can trigger manual pricing sync
test_trigger_manual_sync_failure - Manual sync handles failure gracefully
test_trigger_manual_sync_requires_admin - Manual sync trigger requires admin role
test_trigger_manual_sync_requires_authentication - Manual sync trigger requires authentication
test_trigger_manual_sync_logs_admin_user - Manual sync logs the admin user who triggered it
test_trigger_manual_sync_handles_exception - Manual sync handles unexpected exceptions
class TestPricingSchedulerIntegration - Integration tests for pricing scheduler endpoints
test_status_after_manual_trigger - Status endpoint shows updated state after manual trigger
test_multiple_admin_users_can_trigger - Multiple admin users can trigger manual sync
References: src/config.py, src/main.py, src/routes/ai_sdk.py
Module: Tests for the Vercel AI SDK endpoint.
class TestAISDKEndpoint - Tests for the AI SDK chat completion endpoint
test_ai_sdk_endpoint_exists - Test that the AI SDK endpoint is registered
test_ai_sdk_completions_endpoint_exists - Test that the AI SDK completions endpoint is registered
test_ai_sdk_chat_completion_success - Test successful AI SDK chat completion request
test_ai_sdk_missing_api_key - Test that missing API key returns proper error and captures to Sentry
test_ai_sdk_invalid_request_format - Test that invalid request format returns proper error
test_ai_sdk_streaming_request - Test streaming response from AI SDK endpoint
test_ai_sdk_endpoint_schema - Test that endpoint properly validates request schema
test_ai_sdk_completions_endpoint_works - Test that the -completions variant endpoint works identically
class TestAISDKConfiguration - Tests for AI SDK configuration
test_ai_sdk_config_variable_exists - Test that AI_SDK_API_KEY is available in Config
test_ai_sdk_config_loading - Test that AI_SDK_API_KEY is properly loaded from environment
class TestAISDKModelRouting - Tests for AI SDK model routing (openrouter/* models go directly to OpenRouter)
test_is_openrouter_model_auto - Test that openrouter/auto is detected correctly
test_is_openrouter_model_other_openrouter_models - Test that other openrouter/* models are detected
test_is_openrouter_model_case_insensitive - Test that openrouter/* detection is case insensitive
test_is_openrouter_model_false_for_regular_models - Test that regular models are not detected as openrouter/*
test_is_openrouter_model_false_for_empty - Test that empty model returns False
test_is_openrouter_model_false_for_none - Test that None model returns False
test_openrouter_auto_routes_to_openrouter - Test that openrouter/auto is routed directly to OpenRouter
test_openrouter_auto_streaming_routes_to_openrouter - Test that openrouter/auto streaming is routed directly to OpenRouter
class TestAISDKErrorMessages - Tests for proper error messages based on the service that failed
test_openrouter_missing_api_key_shows_openrouter_error - Test that missing OpenRouter API key shows OpenRouter-specific error message
test_ai_sdk_missing_api_key_shows_ai_sdk_error - Test that missing AI SDK API key shows AI SDK-specific error message
test_openrouter_error_captured_to_sentry - Test that OpenRouter configuration errors are captured to Sentry
test_openrouter_streaming_missing_api_key_returns_503 - Test that missing OpenRouter API key returns HTTP 503 for streaming requests
test_openrouter_streaming_error_captured_to_sentry - Test that OpenRouter streaming config errors are captured to Sentry
class TestAISDKStreamingReasoningContent - Tests for reasoning/thinking content in AI SDK streaming responses
test_streaming_with_reasoning_content - Test streaming response includes reasoning_content for thinking models
test_streaming_only_reasoning_content - Test streaming response with only reasoning content (no text)
test_streaming_with_reasoning_via_reasoning_attr - Test streaming response with 'reasoning' attribute (alternative name)
test_openrouter_streaming_with_reasoning_content - Test OpenRouter streaming response includes reasoning_content
test_streaming_empty_reasoning_content_no_fallback - Test that empty reasoning_content doesn't incorrectly fallback to reasoning attribute
class TestAISDKAuthentication - Tests for AI SDK endpoint authentication requirement
test_ai_sdk_requires_authentication - Test that AI SDK endpoints require API key authentication
test_ai_sdk_completions_requires_authentication - Test that AI SDK completions endpoint requires API key authentication
test_ai_sdk_invalid_api_key - Test that invalid API key returns 401
test_ai_sdk_trial_expired - Test that expired trial returns 403
class TestAISDKCreditDeduction - Tests for AI SDK endpoint credit deduction
test_ai_sdk_deducts_credits_for_non_trial - Test that AI SDK deducts credits for non-trial users
test_ai_sdk_tracks_trial_usage - Test that AI SDK tracks trial usage for trial users
test_ai_sdk_streaming_deducts_credits - Test that AI SDK streaming deducts credits after stream completes
class TestAISDKSentryIntegration - Tests for Sentry error capture in AI SDK endpoint
test_general_error_captured_to_sentry - Test that general errors are captured to Sentry
test_error_when_sentry_unavailable - Test that errors still work when Sentry is unavailable
test_streaming_error_captured_to_sentry - Test that streaming errors are captured to Sentry
class TestTrialStatusOverride - Tests for the trial status override defense-in-depth logic
test_check_trial_override_non_trial_user_returns_false - Test that non-trial users return False (should be billed)
test_check_trial_override_legitimate_trial_user_returns_true - Test that legitimate trial users return True (should NOT be billed)
test_check_trial_override_paid_user_with_stale_trial_flag - Test that paid users with stale is_trial=TRUE are overridden to billing path
test_check_trial_override_pro_tier_with_trial_flag - Test that pro tier users with is_trial=TRUE are overridden
test_check_trial_override_max_tier_with_trial_flag - Test that max tier users with is_trial=TRUE are overridden
test_check_trial_override_admin_tier_with_trial_flag - Test that admin tier users with is_trial=TRUE are overridden
test_check_trial_override_inactive_subscription_not_overridden - Test that cancelled subscriptions are NOT overridden
test_check_trial_override_null_user_returns_trial_status - Test that None user returns original trial status
test_check_trial_override_empty_trial_dict_returns_false - Test that empty trial dict is treated as non-trial
References: src/routes/alibaba_debug.py
Module: Comprehensive tests for Alibaba Debug routes
class TestAlibabaDebugRoutes - Test Alibaba Debug route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/main.py, src/routes/analytics.py, src/security/deps.py, src/services/analytics.py
Module: Comprehensive tests for analytics endpoints
class TestLogEvent - Test single analytics event logging
test_log_event_authenticated_user - Test logging event with authenticated user
test_log_event_anonymous_user - Test logging event without authentication
test_log_event_with_provided_user_id - Test logging event with user_id in payload
test_log_event_without_metadata - Test logging event without optional metadata
test_log_event_statsig_error - Test error handling when Statsig fails
test_log_event_missing_event_name - Test validation error for missing event_name
class TestLogBatchEvents - Test batch analytics event logging
test_log_batch_events_authenticated_user - Test logging multiple events for authenticated user
test_log_batch_events_mixed_user_ids - Test batch events with different user IDs
test_log_batch_events_empty_list - Test batch logging with empty events list
test_log_batch_events_large_batch - Test logging large batch of events
test_log_batch_events_partial_failure - Test error handling when batch logging fails
class TestEventMetadataHandling - Test metadata handling in analytics events
test_event_with_complex_metadata - Test event with complex nested metadata
test_event_with_array_metadata - Test event with array in metadata
class TestUserIDPriority - Test user ID determination priority
test_user_id_priority_authenticated_over_provided - Test authenticated user ID takes priority over provided user_id
test_user_id_priority_provided_over_anonymous - Test provided user_id takes priority over anonymous
test_user_id_default_to_anonymous - Test default to anonymous when no user info available
class TestAnalyticsIntegration - Test analytics integration scenarios
test_single_and_batch_consistency - Test that single and batch endpoints produce consistent results
References: src/db/api_keys.py, src/db_security.py, src/main.py, src/routes/api_keys.py, src/schemas/api_keys.py, src/security/deps.py, src/security/security.py
Module: Comprehensive tests for API key management endpoints
class TestApiKeyCreation - Test API key creation
test_create_api_key_success - Test successful API key creation
test_create_api_key_insufficient_permissions - Test API key creation with insufficient permissions
test_create_api_key_invalid_environment - Test API key creation with invalid environment tag
test_create_api_key_negative_expiration - Test API key creation with negative expiration days
test_create_api_key_duplicate_name - Test API key creation with duplicate name
class TestApiKeyUpdate - Test API key update and rotation
test_update_api_key_success - Test successfully updating API key
test_update_api_key_not_found - Test updating non-existent API key
test_update_api_key_no_fields - Test updating API key with no fields
test_rotate_api_key_success - Test successfully rotating API key
test_bulk_rotate_api_keys_success - Test bulk rotation of API keys
class TestApiKeyListing - Test API key listing
test_list_api_keys_success - Test successfully listing API keys
test_list_api_keys_insufficient_permissions - Test listing keys with insufficient permissions
test_list_api_keys_empty - Test listing when no API keys exist
class TestApiKeyDeletion - Test API key deletion
test_delete_api_key_success - Test successfully deleting API key
test_delete_api_key_wrong_confirmation - Test deleting API key with wrong confirmation
test_delete_api_key_not_found - Test deleting non-existent API key
test_delete_api_key_failed - Test API key deletion failure
class TestApiKeyUsage - Test API key usage statistics
test_get_usage_stats_success - Test successfully getting usage statistics
test_get_usage_stats_failed - Test usage statistics retrieval failure
test_get_usage_stats_invalid_key - Test usage stats with invalid API key
class TestApiKeyIntegration - Test API key management integration scenarios
test_complete_key_lifecycle - Test complete API key lifecycle: create -> list -> use
test_update_then_delete_workflow - Test updating then deleting a key
References: src/main.py, src/routes/api_models.py
Module: Integration tests for /api/models/detail endpoint
class TestApiModelsDetailEndpoint - Test /api/models/detail endpoint
test_get_model_detail_with_model_id - Test getting model detail using modelId parameter
test_get_model_detail_with_separate_params - Test getting model detail using separate developer and modelName params
test_get_model_detail_with_all_params - Test with both modelId and separate params (separate params take precedence)
test_get_model_detail_with_gateway - Test getting model detail with specific gateway
test_get_model_detail_not_found - Test 404 when model not found
test_get_model_detail_missing_params - Test 400 when required parameters are missing
test_get_model_detail_only_developer - Test 400 when only developer is provided
test_get_model_detail_only_model_name - Test when only modelName is provided without developer
test_get_model_detail_with_huggingface - Test including HuggingFace data
test_get_model_detail_returns_providers_list - Test that providers list is included in response
test_get_model_detail_url_encoded_model_id - Test with URL-encoded modelId (as sent by frontend)
test_get_model_detail_with_complex_model_name - Test with complex model names containing multiple slashes
class TestApiModelsDetailErrorHandling - Test error handling in /api/models/detail endpoint
test_internal_error_handling - Test that internal errors return 500
References: src/main.py, src/routes/audio.py
Module: Tests for audio transcription routes.
class TestAudioTranscriptions - Tests for POST /v1/audio/transcriptions endpoint.
test_transcription_endpoint_exists - Test that the transcription endpoint is registered.
test_transcription_requires_file - Test that file parameter is required.
test_transcription_success - Test successful transcription.
test_transcription_with_language_hint - Test transcription with language hint.
test_transcription_with_prompt - Test transcription with prompt context.
test_transcription_empty_file - Test that empty files are rejected.
test_transcription_unsupported_format - Test that unsupported formats are handled.
class TestAudioTranscriptionsBase64 - Tests for POST /v1/audio/transcriptions/base64 endpoint.
test_base64_endpoint_exists - Test that the base64 transcription endpoint is registered.
test_base64_transcription_success - Test successful base64 transcription.
test_base64_data_url_format - Test handling of data URL format.
test_base64_invalid_data - Test that invalid base64 data is rejected.
test_base64_empty_data - Test that empty base64 data is rejected.
class TestAudioFormats - Tests for supported audio format handling.
test_supported_formats - Test that various audio formats are accepted.
class TestAudioErrorHandling - Tests for error handling in audio transcription.
test_whisper_api_error - Test handling of Whisper API errors.
test_client_unavailable - Test handling when OpenAI client is unavailable.
References: src/main.py, src/routes/audit.py, src/security/deps.py
Module: Comprehensive tests for audit log endpoints
class TestGetAuditLogs - Test audit log retrieval endpoint
test_get_audit_logs_success - Test successfully retrieving audit logs
test_get_audit_logs_with_key_filter - Test retrieving audit logs filtered by key_id
test_get_audit_logs_with_action_filter - Test retrieving audit logs filtered by action
test_get_audit_logs_with_date_range - Test retrieving audit logs with date range
test_get_audit_logs_with_limit - Test retrieving audit logs with custom limit
test_get_audit_logs_combined_filters - Test retrieving audit logs with multiple filters
test_get_audit_logs_empty - Test retrieving empty audit logs
class TestAuditAuthentication - Test authentication and authorization
test_get_audit_logs_invalid_api_key - Test retrieving audit logs with invalid API key
test_get_audit_logs_insufficient_permissions - Test retrieving audit logs without sufficient permissions
test_get_audit_logs_missing_api_key - Test retrieving audit logs without API key
class TestAuditDateValidation - Test date parsing and validation
test_get_audit_logs_invalid_start_date - Test invalid start_date format
test_get_audit_logs_invalid_end_date - Test invalid end_date format
test_get_audit_logs_valid_iso_dates - Test valid ISO date formats
class TestAuditErrorHandling - Test error handling
test_get_audit_logs_database_error - Test error handling when database fails
test_get_audit_logs_user_lookup_error - Test error handling when user lookup fails
class TestAuditIntegration - Test audit log integration scenarios
test_audit_log_filtering_workflow - Test complete filtering workflow
test_audit_log_pagination - Test pagination with limit parameter
test_audit_log_metadata_structure - Test audit log response includes metadata
References: src/routes/auth.py, src/schemas/auth.py
Module: Tests for auth route error handling improvements.
class TestAuthConfigurationErrorHandling - Test auth endpoint error handling for configuration issues
test_returns_503_for_url_protocol_error - Test that URL protocol errors return 503 with user-friendly message
test_returns_503_for_supabase_url_must_start_with_error - Test that SUPABASE_URL validation errors return 503
test_returns_503_for_supabase_url_not_set_error - Test that missing SUPABASE_URL error returns 503
test_returns_500_for_other_errors - Test that non-configuration errors return 500
test_does_not_expose_internal_details_for_config_errors - Test that configuration errors don't expose internal URL values to users
References: src/config/supabase_config.py, src/db/activity.py, src/db/api_keys.py, src/db/users.py, src/main.py, src/routes/auth.py
Module: Tests for temporary email bot status assignment
test_privy_auth_temp_email_sets_bot_status - Test that new users with temporary email get subscription_status='bot'
test_privy_auth_normal_email_sets_trial_status - Test that new users with normal email get subscription_status='trial'
test_privy_auth_10minutemail_sets_bot_status - Test that 10minutemail.com domain gets bot status
test_privy_auth_guerrillamail_sets_bot_status - Test that guerrillamail.com domain gets bot status
test_privy_auth_mailinator_sets_bot_status - Test that mailinator.com domain gets bot status
test_privy_auth_yopmail_sets_bot_status - Test that yopmail.com domain gets bot status
test_privy_auth_blocked_domain_still_rejected - Test that blocked abuse domains are still rejected (not just marked as bot)
test_register_temp_email_sets_bot_status - Test that direct registration with temporary email sets bot status
test_register_normal_email_sets_trial_status - Test that direct registration with normal email sets trial status
test_register_blocked_domain_still_rejected - Test that blocked domains are still rejected in registration
test_temp_email_user_stored_with_bot_status_in_db - Verify that users with temp email are stored with bot status in database
test_normal_email_user_stored_with_trial_status_in_db - Verify that users with normal email are stored with trial status in database
References: src/config/supabase_config.py, src/db/activity.py, src/db/api_keys.py, src/db/users.py, src/main.py, src/routes/auth.py
Module: Comprehensive tests for authentication endpoints using dependency injection
test_privy_auth_existing_user_success - Test successful authentication for existing user
test_privy_auth_existing_user_no_api_keys_uses_legacy - Test existing user with no API keys falls back to legacy key
test_privy_auth_fallback_to_username_lookup - Test fallback to username lookup if Privy ID not found
test_privy_auth_new_user_creation - Test creating new user via Privy auth
test_privy_auth_email_address_field_extraction - Email accounts that use address field should still be processed
test_privy_auth_google_oauth - Test authentication with Google OAuth
test_privy_auth_github - Test authentication with GitHub
test_privy_auth_email_fallback - Test email fallback when no email found in linked accounts
test_privy_auth_phone_new_user - Test authentication with phone number for new user
test_privy_auth_phone_existing_user - Test authentication with phone number for existing user
test_privy_auth_phone_with_email - Test user with both phone and email linked accounts
test_privy_auth_phone_camelcase_field - Test that phoneNumber (camelCase from Privy) is properly handled
test_privy_auth_phone_international_format - Test phone authentication with various international phone number formats
test_privy_auth_phone_short_number - Test phone authentication with a short phone number (edge case)
test_privy_auth_phone_first_then_email_added - Test user who signed up with phone, then added email later
test_privy_auth_phone_with_google_oauth - Test user with phone and Google OAuth linked
test_privy_auth_phone_returning_user_updates_phone - Test returning user who adds phone number to their account
test_privy_auth_phone_only_no_email_placeholder - Test that phone-only users get proper placeholder email in database
test_privy_auth_phone_username_generation_various_lengths - Test username generation from phone numbers of various lengths
test_register_user_success - Test successful user registration
test_register_user_duplicate_email - Test registration with existing email
test_register_user_duplicate_username - Test registration with existing username
test_request_password_reset_success - Test successful password reset request
test_request_password_reset_user_not_found - Test password reset for non-existent email (security - don't reveal)
test_reset_password_with_valid_token - Test resetting password with valid token
test_reset_password_with_invalid_token - Test resetting password with invalid token
test_reset_password_with_expired_token - Test resetting password with expired token
test_reset_password_with_used_token - Test resetting password with already used token
References: src/routes/availability.py
Module: Comprehensive tests for Availability routes
class TestAvailabilityRoutes - Test Availability route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/main.py, src/routes/catalog.py
Module: Integration tests for catalog endpoints
class TestGetProvidersEndpoint - Test /v1/provider endpoint
test_get_providers_openrouter_default - Test getting OpenRouter providers (default)
test_get_providers_with_gateway_param - Test with specific gateway parameter
test_get_providers_all_gateways - Test getting providers from all gateways
test_get_providers_huggingface - Test getting Hugging Face providers
test_get_providers_huggingface_alias - Test huggingface gateway alias
test_get_providers_moderated_only - Test filtering for moderated providers
test_get_providers_with_limit - Test pagination with limit parameter
test_get_providers_with_offset - Test pagination with offset parameter
test_get_providers_empty_data - Test when no provider data available - should return 200 with empty response (graceful degradation)
test_get_providers_none_data - Test when provider data is None - should return 200 with empty response (graceful degradation)
test_get_providers_groq - Test Groq gateway
test_get_providers_fireworks - Test Fireworks gateway
test_get_providers_together - Test Together gateway - should return 200 with empty data (graceful degradation)
test_get_providers_cerebras - Test Cerebras gateway - should return 200 with empty data (graceful degradation)
test_get_providers_xai - Test xAI gateway
test_get_providers_deepinfra - Test DeepInfra gateway
test_get_providers_featherless - Test Featherless gateway - should return 200 with empty data (graceful degradation)
class TestModelsEndpoint - Test coverage for the unified /models endpoint used by the UI.
test_nebius_gateway_returns_empty_catalog - Requests for Nebius should return 200 even if no catalog is available.
test_xai_gateway_returns_empty_catalog - Requests for xAI should return 200 even if no catalog is available.
test_models_page_all_gateway_loads - Ensure the models catalog page can load aggregated data.
class TestMergeProviderLists - Test provider list merging
test_merge_providers_from_multiple_sources - Test that providers from multiple gateways are merged
class TestMergeModelsBySlug - Test model merging by slug
test_models_merged_correctly - Test that duplicate models are handled
class TestGetGatewaysEndpoint - Test /v1/gateways endpoint for gateway auto-discovery
test_get_gateways_returns_list - Test that gateways endpoint returns a list of gateways
test_get_gateways_has_required_fields - Test that each gateway has required fields
test_get_gateways_includes_simplismart - Test that SimpliSmart is in the gateway list
test_get_gateways_includes_major_providers - Test that major providers are included
test_get_gateways_fast_gateways_first - Test that fast gateways are sorted before slow gateways
test_get_gateways_logo_urls - Test that gateways have logo URLs generated from site URLs
test_get_gateways_aliases - Test that gateways with aliases include them
References: src/main.py, src/routes/catalog.py
Module: Test for the simplismart_models UnboundLocalError fix
class TestSimplismartModelsInitialization - Test that simplismart_models variable is properly initialized
test_get_models_alpaca_gateway_no_unbound_error - Test that requesting models from 'alpaca' gateway doesn't cause UnboundLocalError.
test_get_models_clarifai_gateway_no_unbound_error - Test that requesting models from 'clarifai' gateway doesn't cause UnboundLocalError.
test_get_models_huggingface_gateway_no_unbound_error - Test that requesting models from 'huggingface' gateway doesn't cause UnboundLocalError.
test_get_models_multiple_non_simplismart_gateways - Test multiple different gateways to ensure robustness of the fix.
test_get_models_simplismart_gateway_still_works - Verify that the fix doesn't break the intended simplismart gateway functionality.
test_get_models_all_gateway_includes_simplismart - Verify that 'all' gateway properly includes simplismart models.
class TestRootModelsEndpoint - Test the /models endpoint (without /v1 prefix) which delegates to get_models.
test_root_models_endpoint_alpaca_no_error - Test /models endpoint with alpaca gateway
test_root_models_endpoint_clarifai_no_error - Test /models endpoint with clarifai gateway
class TestProviderDerivationBlock - Test the provider derivation block that references simplismart_models.
test_provider_derivation_with_all_gateway - Test that 'all' gateway properly handles provider derivation for simplismart.
References: src/routes/catalog.py
Module: Tests for catalog utility functions
class TestNormalizeDeveloperSegment - Test developer segment normalization
test_normalize_none
test_normalize_empty_string
test_normalize_with_at_symbol
test_normalize_regular_string
test_normalize_non_string
class TestNormalizeModelSegment - Test model segment normalization
test_normalize_none
test_normalize_empty_string
test_normalize_preserves_casing
test_normalize_strips_whitespace
test_normalize_non_string
class TestAnnotateProviderSources - Test provider source annotation
test_annotate_empty_list
test_annotate_none
test_annotate_single_provider
test_annotate_multiple_providers
test_annotate_preserves_existing_data
test_annotate_doesnt_duplicate_source
test_annotate_adds_new_source
References: src/db/api_keys.py, src/db/chat_history.py, src/db/plans.py, src/db/rate_limits.py, src/db/users.py, src/routes/chat.py, src/schemas/chat.py, src/security/deps.py, src/services/model_availability.py, src/services/model_transformations.py, src/services/pricing.py, src/services/prometheus_metrics.py, src/services/trial_validation.py
test_happy_path_openrouter - Test successful chat completion with OpenRouter
test_invalid_api_key - Test that invalid API key returns 401
test_plan_limit_exceeded_precheck - Test that plan limit exceeded returns 429
test_rate_limit_exceeded_precheck - Test that rate limit exceeded returns 429
test_insufficient_credits_non_trial - Test that insufficient credits returns 402
test_trial_valid_usage_tracked - Test that trial user usage is tracked correctly
test_trial_expired_403 - Test that expired trial returns 403
test_expired_trial_can_use_free_model - Test that expired trial can still access free models (with :free suffix)
test_trial_limits_exceeded_can_use_free_model - Test that trial with exceeded limits can still access free models
test_expired_trial_blocked_for_non_free_model - Test that expired trial is blocked for non-free models
test_valid_trial_free_model_usage_tracked - Test that free model usage is tracked for valid trials too
test_paid_user_free_model_usage_tracked - Test that free model usage is tracked for paid users
test_is_free_model_with_free_suffix - Test that models with :free suffix are detected as free
test_is_free_model_without_free_suffix - Test that models without :free suffix are not detected as free
test_is_free_model_edge_cases - Test edge cases for is_free_model
test_validate_trial_with_free_model_bypass_does_not_mutate_original_dict - Test that validate_trial_with_free_model_bypass does not mutate the original trial dict.
test_validate_trial_with_free_model_bypass_cache_isolation - Test that multiple calls don't share state through the original dict.
test_upstream_429_maps_429 - Test that upstream 429 error is properly mapped to 429
test_upstream_401_maps_500_in_your_code - Test that upstream 401 error is mapped to 500
test_upstream_request_error_maps_503 - Test that upstream request error is mapped to 503
test_upstream_timeout_maps_504 - Test that upstream timeout is handled properly
test_saves_chat_history_when_session_id - Test that chat history is saved when session_id is provided
test_streaming_response - Test streaming response
test_provider_failover_to_huggingface - Test provider failover from featherless to huggingface
test_provider_failover_on_404_to_huggingface - Test provider failover on 404 from featherless to huggingface
test_response_with_none_choices - Test that responses with None choices are handled safely (no NoneType error)
test_response_with_empty_choices - Test that responses with empty choices are handled safely
test_response_with_none_message - Test that responses with None message in choices are handled safely
test_happy_path_onerouter - Test successful chat completion with OneRouter provider
test_onerouter_streaming_response - Test streaming response with OneRouter provider
test_onerouter_upstream_error_handling - Test that OneRouter upstream errors are properly handled
test_onerouter_versioned_model_format - Test OneRouter with @ versioned model format (e.g., claude-3-5-sonnet@20240620)
test_onerouter_network_error_handling - Test that OneRouter network errors are properly handled
test_fal_model_rejected_in_chat_completions - Test that FAL models (image/video generation only) are rejected with a clear error in chat completions
test_fal_model_rejected_various_models - Test that various FAL model IDs are rejected with a clear error
References: src/routes/chat.py, src/schemas/proxy.py, src/security/deps.py, src/services/query_classifier.py, src/services/tools/base.py
Module: Tests for Auto Web Search functionality in Chat Completions.
class TestAutoWebSearchParameter - Tests for auto_web_search parameter handling.
test_auto_web_search_default_is_auto - Test that auto_web_search defaults to 'auto'.
test_auto_web_search_can_be_true - Test that auto_web_search can be set to True.
test_auto_web_search_can_be_false - Test that auto_web_search can be set to False.
test_web_search_threshold_default - Test that web_search_threshold defaults to 0.5.
test_web_search_threshold_validation - Test that web_search_threshold is validated between 0 and 1.
class TestQueryClassifierIntegration - Tests for query classifier integration in chat completions.
test_travel_query_triggers_classification - Test that travel queries are classified correctly.
test_code_query_does_not_trigger_search - Test that code queries don't trigger search.
test_greeting_does_not_trigger_search - Test that simple greetings don't trigger search.
class TestMessageAugmentation - Tests for message augmentation with search results.
test_search_context_format - Test that search context is formatted correctly.
test_system_message_insertion_logic - Test that system message is inserted after existing system messages.
test_system_message_insertion_no_existing_system - Test system message insertion when no existing system message.
class TestAutoWebSearchModes - Tests for different auto_web_search modes.
test_auto_mode_uses_classifier - Test that 'auto' mode uses the query classifier.
test_explicit_true_always_searches - Test that auto_web_search=True would always search.
test_explicit_false_never_searches - Test that auto_web_search=False never searches.
class TestErrorHandling - Tests for error handling in auto web search.
test_search_failure_continues_request - Test that search failure doesn't break the request.
test_classifier_exception_handled - Test that classifier exceptions are handled gracefully.
test_empty_search_results_handled - Test handling of empty search results.
class TestRealWorldScenarios - Integration tests for real-world scenarios.
test_various_queries - Test classification of various query types.
test_conversation_context_uses_last_message - Test that classification uses the last user message.
class TestThresholdBehavior - Tests for threshold configuration behavior.
test_low_threshold_more_searches - Test that lower threshold triggers more searches.
test_threshold_boundary_cases - Test behavior at threshold boundaries.
References: src/routes/chat.py, src/services/butter_client.py
Module: Tests for Butter.dev integration in chat completions route.
class TestButterProviderConfig - Test BUTTER_PROVIDER_CONFIG mapping.
test_openrouter_config_exists - Test that OpenRouter config is defined.
test_together_config_exists - Test that Together config is defined.
test_fireworks_config_exists - Test that Fireworks config is defined.
test_groq_config_exists - Test that Groq config is defined.
test_onerouter_config_exists - Test that OneRouter/Infron config is defined - this is the default provider.
test_all_compatible_providers_have_config - Test that all providers in BUTTER_COMPATIBLE_PROVIDERS have a config entry.
test_all_configs_have_required_fields - Test that all provider configs have required fields.
class TestMakeButterProxiedStream - Test the make_butter_proxied_stream function.
test_raises_for_unknown_provider - Test that unknown provider raises ValueError.
test_raises_for_missing_api_key - Test that missing API key raises ValueError.
test_creates_butter_client_correctly - Test that Butter client is created with correct parameters.
class TestButterIntegrationInChatCompletions - Test Butter.dev integration in the chat completions flow.
test_should_use_butter_cache_is_imported - Test that should_use_butter_cache is available in chat module.
test_butter_cache_timer_is_imported - Test that ButterCacheTimer is available in chat module.
test_get_butter_pooled_async_client_is_imported - Test that get_butter_pooled_async_client is available in chat module.
class TestButterCacheHeaderLogic - Test the X-Butter-Cache header logic.
test_butter_reason_codes - Test all possible reason codes from should_use_butter_cache.
References: src/main.py
Module: Comprehensive test suite for Chat Completions endpoint.
class TestChatCompletionsBasic - Test basic chat completions functionality.
test_chat_completions_simple_request - Test simple chat completions request with minimal parameters.
test_chat_completions_system_message - Test chat completions with system message.
test_chat_completions_multiple_messages - Test chat completions with conversation history.
class TestChatCompletionsParameters - Test chat completions with various parameters.
test_chat_completions_with_temperature - Test chat completions with temperature parameter.
test_chat_completions_with_top_p - Test chat completions with top_p parameter.
test_chat_completions_with_frequency_penalty - Test chat completions with frequency penalty.
test_chat_completions_with_presence_penalty - Test chat completions with presence penalty.
test_chat_completions_with_seed - Test chat completions with seed parameter for reproducibility.
test_chat_completions_with_user_identifier - Test chat completions with user identifier.
class TestChatCompletionsStreaming - Test streaming chat completions.
test_chat_completions_streaming - Test streaming chat completions.
class TestChatCompletionsResponseStructure - Test response structure and format.
test_chat_completions_response_structure - Test that chat completions response has correct structure.
test_chat_completions_usage_information - Test that response includes usage information when available.
class TestChatCompletionsErrorHandling - Test error handling in chat completions.
test_chat_completions_missing_model - Test error when model is missing.
test_chat_completions_missing_messages - Test error when messages are missing.
test_chat_completions_invalid_auth - Test error with invalid authentication.
class TestChatCompletionsPerformance - Test performance of chat completions.
test_chat_completions_response_time - Test that chat completions responds in reasonable time.
References: src/config/supabase_config.py, src/db/activity.py, src/db/api_keys.py, src/db/chat_history.py, src/db/plans.py, src/db/rate_limits.py, src/db/users.py, src/main.py, src/routes/chat.py, src/security/deps.py, src/services/openrouter_client.py, src/services/pricing.py, src/services/rate_limiting.py, src/services/trial_validation.py, src/services/vercel_ai_gateway_client.py
Module: Comprehensive tests for routes/chat.py
test_chat_completions_no_api_key - Test chat endpoint without API key - anonymous access is now allowed
test_chat_completions_invalid_api_key - Test chat endpoint with invalid API key
test_chat_completions_success - Test successful chat completion
test_chat_completions_with_optional_params - Test chat completion with optional parameters
test_chat_completions_credits_deducted - Test that credits are properly deducted
test_chat_completions_streaming_success - Test successful streaming chat completion
test_chat_completions_streaming_anonymous_success - Test streaming chat completion for anonymous users (no API key).
test_chat_completions_insufficient_credits - Test chat endpoint with insufficient credits
test_chat_completions_trial_user_success - Test chat endpoint with trial user
test_chat_completions_trial_expired - Test chat endpoint with expired trial
test_chat_completions_missing_model - Test chat endpoint without model parameter
test_chat_completions_missing_messages - Test chat endpoint without messages parameter
test_chat_completions_empty_messages - Test chat endpoint with empty messages array
test_unified_responses_success - Test /v1/responses endpoint
test_unified_responses_with_multimodal_input - Test /v1/responses with multimodal input
test_unified_responses_with_output_text_transformation - Test /v1/responses properly transforms output_text content type to text.
test_chat_completions_with_session_id - Test chat endpoint with session ID for history
test_chat_completions_with_specific_provider - Test chat endpoint with specific provider
test_chat_completions_very_long_message - Test chat endpoint with very long message
test_chat_completions_multiple_messages - Test chat endpoint with conversation history
References: src/db/activity.py, src/db/api_keys.py, src/db/plans.py, src/db/rate_limits.py, src/db/users.py, src/main.py, src/routes/chat.py, src/services/pricing.py, src/services/rate_limiting.py, src/services/trial_validation.py
Module: Route tests for function calling support
class TestChatCompletionsFunctionCalling - Test function calling in /v1/chat/completions endpoint
test_tools_parameter_extracted - Test that tools parameter is correctly extracted from request
test_tools_parameter_validation - Test that invalid tools parameter is rejected
test_multiple_tools_passed_through - Test that multiple tools are passed through correctly
References: src/config/supabase_config.py, src/db/chat_history.py, src/routes/chat_history.py
Module: Test script to verify chat history endpoints are working
test_chat_history_data - Check if chat history data exists in database
References: src/main.py, src/routes/chat_history.py
Module: Integration tests for chat history routes with performance optimizations
class TestChatHistoryPerformance - Test chat history endpoints with performance optimizations
test_create_session_uses_cached_user_lookup - create_session should use cached user lookup
test_create_session_uses_background_logging - create_session should use background activity logging
test_create_session_logs_performance_metrics - create_session should log performance metrics
test_create_session_returns_success_response - create_session should return successful response
test_create_session_invalid_api_key - create_session should return 401 for invalid API key
test_create_session_handles_background_logging_failure - create_session should handle background logging failures gracefully
test_create_session_background_logging_includes_metadata - create_session should pass metadata to background logging
test_create_session_with_default_model - create_session should work without explicit model
test_create_session_with_empty_title - create_session should generate title if not provided
References: src/main.py, src/routes/chat_metrics.py
Module: Test suite for Chat Completions Metrics endpoints.
class TestTokensPerSecondEndpoints - Test tokens per second metrics endpoints.
test_get_all_tokens_per_second_valid - Test /tokens-per-second/all with valid parameters.
test_get_all_tokens_per_second_missing_provider - Test /tokens-per-second/all without provider_id parameter.
test_get_all_tokens_per_second_missing_model - Test /tokens-per-second/all without model_id parameter.
test_get_tokens_per_second_valid_hour - Test /tokens-per-second with valid hour parameter.
test_get_tokens_per_second_valid_week - Test /tokens-per-second with valid week parameter.
test_get_tokens_per_second_valid_month - Test /tokens-per-second with valid month parameter.
test_get_tokens_per_second_valid_1year - Test /tokens-per-second with valid 1year parameter.
test_get_tokens_per_second_valid_2year - Test /tokens-per-second with valid 2year parameter.
test_get_tokens_per_second_invalid_time - Test /tokens-per-second with invalid time parameter.
test_get_tokens_per_second_missing_time - Test /tokens-per-second without time parameter.
test_get_tokens_per_second_missing_model - Test /tokens-per-second without model_id parameter.
test_get_tokens_per_second_missing_provider - Test /tokens-per-second without provider_id parameter.
class TestPrometheusFormat - Test Prometheus format output.
test_prometheus_format_valid_structure - Test that response follows Prometheus text format.
test_prometheus_format_labels - Test that metrics include proper labels.
test_content_type_is_plain_text - Test that content-type is text/plain.
class TestErrorHandling - Test error handling.
test_invalid_model_id_type - Test error handling with invalid model_id type.
test_empty_provider_id - Test error handling with empty provider_id.
test_server_error_handling - Test that server errors return 500.
References: src/routes/chat.py
Module: QA Test Suite: Chat Requests Monitoring Endpoints
class TestChatRequestsCountsEndpoint - Test /api/monitoring/chat-requests/counts endpoint
test_counts_endpoint_returns_200 - Test that counts endpoint returns 200 OK
test_counts_endpoint_response_structure - Test that response has correct structure
test_counts_endpoint_uses_real_data - Test that endpoint returns real data from database (not mock)
test_counts_endpoint_data_is_sorted - Test that data is sorted by request count (descending)
test_counts_endpoint_metadata_accuracy - Test that metadata totals match actual data
test_counts_endpoint_timestamp_is_valid - Test that timestamp is valid ISO-8601
class TestChatRequestsModelsEndpoint - Test /api/monitoring/chat-requests/models endpoint
test_models_endpoint_returns_200 - Test that models endpoint returns 200 OK
test_models_endpoint_response_structure - Test that response has correct structure
test_models_endpoint_returns_real_model_data - Test that endpoint returns real models with stats (not mock)
test_models_endpoint_provider_data - Test that provider information is included and real
test_models_endpoint_sorted_by_requests - Test that models are sorted by request count (descending)
class TestChatRequestsEndpoint - Test /api/monitoring/chat-requests endpoint
test_chat_requests_returns_200 - Test that endpoint returns 200 OK
test_chat_requests_response_structure - Test that response has correct structure
test_chat_requests_returns_real_requests - Test that endpoint returns real request data (not mock)
test_chat_requests_pagination - Test that pagination works correctly
test_chat_requests_filter_by_model_id - Test filtering by model_id
test_chat_requests_filter_by_model_name - Test filtering by model name
test_chat_requests_returned_count_matches_data - Test that returned_count matches actual data length
test_chat_requests_limit_validation - Test that limit parameter is validated
class TestChatRequestsDataIntegrity - Cross-endpoint tests for data integrity
test_counts_and_models_consistency - Test that counts and models endpoints return consistent data
test_all_endpoints_return_real_timestamps - Test that all endpoints return valid timestamps
test_no_mock_data_markers - Test that responses don't contain mock/test data markers
test_success_flag_always_true_on_200 - Test that 200 responses always have success=true
References: src/db/users.py, src/routes/chat.py, src/security/deps.py, src/services/pricing.py, src/services/trial_validation.py
Module: Tests for trial status defense-in-depth override in chat endpoint.
class TestTrialStatusDefenseInDepth - Tests for trial status override to prevent revenue leak
test_trial_override_with_active_stripe_subscription - Test that user with is_trial=TRUE but active Stripe subscription is charged
test_trial_override_with_pro_tier - Test that user with is_trial=TRUE but pro tier is charged
test_trial_override_with_max_tier - Test that user with is_trial=TRUE but max tier is charged
test_trial_override_with_admin_tier - Test that user with is_trial=TRUE but admin tier is charged
test_legitimate_trial_user_not_charged - Test that legitimate trial users are NOT charged
test_paid_user_with_correct_is_trial_false - Test that paid users with correct is_trial=FALSE are charged normally
References: src/db/coupons.py, src/routes/coupons.py, src/schemas/coupons.py
Module: Comprehensive tests for Coupons routes
class TestCouponsRoutes - Test Coupons route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/main.py, src/routes/credits.py, src/security/deps.py
Module: Tests for Credits Route Endpoints
class TestCreditsAdd - Test credit addition endpoint
test_add_credits_success - Successfully add credits to a user
test_add_credits_user_not_found - Return 404 when user not found
test_add_credits_negative_amount - Reject negative credit amounts
class TestCreditsAdjust - Test credit adjustment endpoint
test_adjust_credits_positive - Successfully add credits via adjustment
test_adjust_credits_negative_prevents_overdraft - Prevent adjustment that would result in negative balance
class TestCreditsBulkAdd - Test bulk credit addition endpoint
test_bulk_add_credits_success - Successfully add credits to multiple users with different balances
test_bulk_add_deduplicates_user_ids - Duplicate user IDs are deduplicated to prevent incorrect balance tracking
test_bulk_add_empty_list - Reject empty user list
class TestCreditsRefund - Test credit refund endpoint
test_refund_credits_success - Successfully refund credits to a user
class TestCreditsSummary - Test credits summary endpoint
test_get_summary_for_user - Get credit summary for specific user
test_get_system_summary - Get system-wide credit summary
class TestCreditsTransactions - Test credit transactions listing endpoint
test_get_transactions_default - Get transactions with default parameters
test_get_transactions_with_filters - Get transactions with filters applied
test_get_transactions_invalid_direction - Reject invalid direction filter
class TestCreditsAuthorization - Test authorization for credits endpoints
test_add_credits_requires_admin - Credits endpoints require admin authentication
References: src/routes/error_monitor.py, src/services/error_monitor.py
Module: Comprehensive tests for Error Monitor routes
class TestErrorMonitorRoutes - Test Error Monitor route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/db/feedback.py, src/routes/chat_history.py, src/security/deps.py
Module: Tests for the message feedback API endpoints.
test_submit_thumbs_up_feedback - Test submitting thumbs up feedback
test_submit_thumbs_down_with_comment - Test submitting thumbs down feedback with comment
test_submit_regenerate_feedback - Test submitting regenerate feedback
test_submit_feedback_invalid_type - Test submitting feedback with invalid type returns 422 (Pydantic validation)
test_submit_feedback_invalid_rating - Test submitting feedback with invalid rating returns 422 (Pydantic validation)
test_submit_feedback_session_not_found - Test submitting feedback with non-existent session returns 404
test_submit_feedback_message_not_owned - Test submitting feedback with message_id not owned by user returns 404
test_submit_feedback_message_not_in_session - Test submitting feedback with message_id not in specified session returns 404
test_submit_feedback_unauthorized - Test submitting feedback without auth returns 401
test_get_user_feedback_all - Test getting all user feedback
test_get_user_feedback_filtered_by_type - Test getting user feedback filtered by type
test_get_user_feedback_filtered_by_session - Test getting user feedback filtered by session
test_get_user_feedback_pagination - Test getting user feedback with pagination
test_get_feedback_stats - Test getting feedback statistics
test_get_feedback_stats_with_model_filter - Test getting feedback statistics filtered by model
test_get_session_feedback - Test getting feedback for a specific session
test_get_session_feedback_not_found - Test getting feedback for non-existent session returns 404
test_update_feedback_success - Test updating feedback successfully
test_update_feedback_not_found - Test updating non-existent feedback returns 404
test_update_feedback_invalid_type - Test updating feedback with invalid type returns 422 (Pydantic validation)
test_delete_feedback_success - Test deleting feedback successfully
test_delete_feedback_not_found - Test deleting non-existent feedback returns 404
References: src/services/model_transformations.py
Module: Test for the Gemini model routing fix.
test_gemini_provider_detection_logic - Unit test for the provider detection logic used in chat.py
test_gemini_with_vertex_credentials - Test that google/gemini models route to Google Vertex when credentials exist.
test_various_gemini_models_route_correctly - Test that various Google/Gemini model IDs are routed correctly.
References: src/main.py, src/routes/catalog.py, src/services/models.py, src/services/pricing.py
Module: GPT-5.1 Pricing Availability Tests
class TestGPT51ModelAvailability - Test that GPT-5.1 models are available in the catalog
test_gpt51_in_openrouter_models - Test that GPT-5.1 is returned from OpenRouter models endpoint
class TestGPT51Pricing - Test GPT-5.1 pricing data structure and availability
test_gpt51_pricing_structure - Test that GPT-5.1 pricing has correct structure
test_gpt51_pricing_lookup - Test that pricing can be looked up for GPT-5.1
test_gpt51_pricing_lookup_hyphen_alias - Test that pricing lookup works for hyphenated GPT-5-1 alias
test_gpt51_cost_calculation - Test that costs are calculated correctly for GPT-5.1
class TestGPT51ModelCatalogEndpoints - Test GPT-5.1 availability through various catalog endpoints
test_models_endpoint_includes_gpt51 - Test that /v1/models includes GPT-5.1
test_provider_endpoint_openai_includes_gpt51 - Test that /v1/provider returns OpenAI provider with GPT-5.1
test_catalog_page_loads_gpt51 - Test that models catalog page loads with GPT-5.1
class TestGPT51DynamicPricing - Test that GPT-5.1 pricing is dynamically fetched from OpenRouter
test_fetch_gpt51_from_openrouter_api - Test that GPT-5.1 can be fetched from OpenRouter API
test_gpt51_pricing_sanitization_filters_dynamic - Test that negative pricing values return None (filter model)
test_gpt51_pricing_sanitization_all_negative_filters - Test sanitization when all pricing is dynamic (-1)
class TestGPT51Variants - Test different GPT-5 model variants are all available with pricing
test_all_gpt5_variants_have_pricing - Test that all GPT-5 variants have pricing
class TestGPT51CostEstimation - Test accurate cost estimation for GPT-5.1 requests
test_estimate_gpt51_request_cost - Test cost estimation for typical GPT-5.1 requests
test_large_gpt51_request_cost - Test cost estimation for large GPT-5.1 requests
test_gpt51_hyphen_alias_cost - Test cost estimation works for hyphen alias input
class TestGPT51Integration - Integration tests for GPT-5.1 with the full system
test_gpt51_in_full_catalog_response - Test GPT-5.1 appears in full catalog response
References: src/config/supabase_config.py, src/main.py, src/models/health_models.py, src/routes/health.py, src/security/deps.py, src/services/model_availability.py, src/services/model_health_monitor.py, src/services/models.py
Module: Tests for Health Check Routes
class TestBasicHealthCheck - Test basic health check endpoint
test_health_check_returns_200 - Basic health check returns 200 OK
test_health_check_returns_healthy_status - Health check returns healthy status
test_health_check_no_auth_required - Health check doesn't require authentication
test_health_check_response_format - Health check returns valid JSON
class TestQuickHealthCheck - Test /health/quick endpoint - ultra-fast health check with no I/O operations
test_quick_health_returns_200 - Quick health check returns 200 OK
test_quick_health_returns_healthy_status - Quick health check returns healthy status
test_quick_health_has_timestamp - Quick health check includes timestamp
test_quick_health_no_auth_required - Quick health check doesn't require authentication
test_quick_health_response_is_minimal - Quick health check returns minimal response (no database info)
class TestSystemHealth - Test system health endpoint
test_system_health_success - Successfully get system health metrics
test_system_health_requires_auth - System health requires authentication
test_system_health_no_data_available - System health handles no data gracefully
test_system_health_error_captured_to_sentry - Test that system health errors are captured to Sentry
class TestProvidersHealth - Test providers health endpoint
test_get_all_providers_health - Get health for all providers
test_filter_providers_by_gateway - Filter providers by gateway parameter
class TestModelsHealth - Test models health endpoint
test_get_all_models_health - Get health for all models
test_filter_models_by_provider - Filter models by provider parameter
test_filter_models_by_status - Filter models by status parameter
class TestSpecificModelHealth - Test specific model health endpoint
test_get_model_health_success - Get health for specific model
test_get_model_health_not_found - Model not found returns 404
class TestSpecificProviderHealth - Test specific provider health endpoint
test_get_provider_health_success - Get health for specific provider
test_get_provider_health_not_found - Provider not found returns 404
class TestHealthSummary - Test health summary endpoint
test_get_health_summary - Get comprehensive health summary
class TestHealthCheck - Test health check trigger endpoints
test_perform_health_check - Trigger background health check
test_perform_immediate_health_check - Perform immediate health check
class TestUptimeMetrics - Test uptime metrics endpoint
test_get_uptime_metrics - Get uptime metrics for status page
test_get_uptime_metrics_error_captured_to_sentry - Test that uptime metrics errors are captured to Sentry but return graceful degradation
test_get_uptime_metrics_no_data_available - Test that uptime metrics handles no data gracefully
class TestHealthDashboard - Test health dashboard endpoint
test_get_health_dashboard - Get complete health dashboard
test_get_health_dashboard_error_captured_to_sentry - Test that dashboard errors are captured to Sentry
class TestHealthStatus - Test simple health status endpoint
test_get_health_status - Get simple health status
class TestMonitoringControls - Test monitoring control endpoints
test_get_monitoring_status - Get monitoring service status
test_start_health_monitoring - Start health monitoring service
test_stop_health_monitoring - Stop health monitoring service
class TestHealthErrorHandling - Test error handling
test_system_health_error_handling - Handle errors in system health gracefully
test_health_check_always_works - Basic health check should never fail
class TestHealthEdgeCases - Test edge cases
test_empty_models_list - Handle empty models list
test_empty_providers_list - Handle empty providers list
test_invalid_model_id_special_chars - Handle special characters in model ID
class TestModelHealthMonitorScheduling - Tests for model health monitor batching and scheduling
test_get_models_to_check_returns_full_gateway_catalog
test_perform_health_checks_batches_models
class TestDatabaseHealth - Test database health endpoint with initialization status
test_database_health_success - Test database health returns success with initialization status
test_database_health_failure - Test database health returns failure with error details
test_database_health_captures_to_sentry - Test database health errors are captured to Sentry
test_database_health_timeout - Test database health returns degraded status when query times out
test_database_health_timeout_constant_exists - Test that HEALTH_CHECK_TIMEOUT_SECONDS constant is defined
class TestHealthAllEndpoint - Test /health/all endpoint
test_get_all_health_success - Successfully get all health information
test_get_all_health_degraded - Return degraded status when providers are unhealthy
class TestHealthModelsStatsEndpoint - Test /health/models/stats endpoint
test_get_models_stats_success - Successfully get model health stats
test_get_models_stats_empty - Handle empty model health data
class TestHealthProvidersStatsEndpoint - Test /health/providers/stats endpoint
test_get_providers_stats_success - Successfully get provider health stats
test_get_providers_stats_empty - Handle empty provider health data
References: src/config.py, src/routes.py
Module: Tests for health_timeline route import fixes.
test_health_timeline_imports_successfully - Test that health_timeline module imports without errors.
test_health_timeline_uses_correct_supabase_import - Test that health_timeline uses get_supabase_client (not get_supabase_admin).
test_supabase_config_exports - Test that supabase_config exports the correct functions.
test_health_timeline_endpoints_defined - Test that health timeline endpoints are properly defined.
References: src/main.py, src/routes/images.py, src/security/deps.py, src/services/user_lookup_cache.py
Module: Comprehensive tests for image generation endpoint
class TestImageGenerationSuccess - Test successful image generation
test_generate_image_deepinfra_success - Test successful image generation with DeepInfra
test_generate_multiple_images - Test generating multiple images
test_generate_image_fal_success - Test successful image generation with Fal.ai
class TestImageGenerationAuth - Test authentication and authorization
test_generate_image_no_auth_header - Test request without Authorization header
test_generate_image_invalid_api_key - Test request with invalid API key
class TestImageGenerationCredits - Test credit validation and deduction
test_generate_image_insufficient_credits - Test request with insufficient credits
test_generate_image_insufficient_credits_multiple - Test request for multiple images with insufficient credits
class TestImageGenerationValidation - Test request validation
test_generate_image_missing_prompt - Test request without required prompt
test_generate_image_empty_prompt - Test request with empty prompt
test_generate_image_invalid_size - Test request with invalid size
test_generate_image_invalid_n - Test request with invalid n value
class TestImageGenerationProviders - Test provider selection and routing
test_default_provider_is_deepinfra - Test that DeepInfra is the default provider
test_unsupported_provider_error - Test error handling for unsupported providers
class TestImageGenerationResponseProcessing - Test response processing
test_response_includes_gateway_usage - Test that response includes gateway usage metadata
test_response_timing_tracked - Test that request timing is tracked
class TestImageGenerationErrorHandling - Test error handling
test_provider_error_handling - Test handling of provider errors
test_credit_deduction_failure_returns_402 - Test that credit deduction failures return 402 Payment Required.
test_unexpected_billing_error_returns_500 - Test that unexpected billing errors return 500 and don't give away free images.
class TestImageGenerationPricing - Test pricing calculations and fallback behavior
test_get_image_cost_known_model - Test cost calculation for known model
test_get_image_cost_multiple_images - Test cost calculation for multiple images
test_get_image_cost_unknown_model_uses_provider_default - Test that unknown models use provider default pricing and flag as fallback
test_get_image_cost_unknown_provider_uses_conservative_default - Test that unknown providers use conservative high default to avoid revenue loss
test_get_image_cost_fal_flux_models - Test pricing for Fal flux models
test_get_image_cost_resolution_multipliers - Test resolution-aware pricing multipliers
test_get_image_cost_unknown_size_uses_default_multiplier - Test that unknown sizes default to 1.0 multiplier
test_get_image_cost_none_size_uses_default_multiplier - Test backwards compatibility: None size uses 1.0 multiplier
test_get_image_cost_resolution_with_multiple_images - Test resolution multiplier applies correctly to multiple images
References: src/config/opentelemetry_config.py, src/main.py, src/routes/instrumentation.py
Module: Tests for instrumentation and observability endpoints.
class TestInstrumentationHealth - Tests for /api/instrumentation/health endpoint.
test_health_returns_status - Test health endpoint returns expected structure.
test_health_includes_loki_config - Test health endpoint includes Loki configuration.
test_health_includes_tempo_config - Test health endpoint includes Tempo configuration.
class TestTraceContext - Tests for /api/instrumentation/trace-context endpoint.
test_trace_context_returns_ids - Test trace context returns trace and span IDs.
class TestOtelInitialize - Tests for /api/instrumentation/otel/initialize endpoint.
test_initialize_requires_auth - Test initialize endpoint requires admin API key.
test_initialize_with_valid_auth - Test initialize endpoint with valid admin API key.
test_initialize_already_initialized - Test initialize endpoint when OTel is already initialized.
class TestOtelReinitialize - Tests for /api/instrumentation/otel/reinitialize endpoint.
test_reinitialize_requires_auth - Test reinitialize endpoint requires admin API key.
test_reinitialize_shuts_down_existing - Test reinitialize shuts down existing provider before reinitializing.
class TestOtelStatus - Tests for /api/instrumentation/otel/status endpoint.
test_status_requires_auth - Test status endpoint requires admin API key.
test_status_returns_config_info - Test status endpoint returns configuration information.
class TestStartupRetryLogic - Tests for the startup retry logic in tempo_otlp initialization.
test_retry_logic_eventually_succeeds - Test that retry logic eventually succeeds when Tempo becomes available.
test_retry_logic_gives_up_after_max_attempts - Test that retry logic gives up after max attempts.
class TestEndpointReachabilityCheck - Tests for the endpoint reachability check logic.
test_check_endpoint_reachable_valid_endpoint - Test reachability check with valid endpoint.
test_check_endpoint_reachable_dns_failure - Test reachability check with DNS failure.
test_check_endpoint_reachable_connection_refused - Test reachability check with connection refused.
test_check_endpoint_reachable_timeout - Test reachability check with timeout.
class TestRailwayEndpointHandling - Tests for Railway-specific endpoint URL handling.
test_railway_internal_dns_port_handling - Test that Railway internal DNS URLs get port 4318 added.
test_railway_public_url_removes_port - Test that Railway public URLs have ports removed.
test_railway_public_url_ensures_https - Test that Railway public URLs use HTTPS.
References: src/main.py, src/routes/messages.py, src/schemas.py, src/services/anthropic_transformer.py
Module: Comprehensive tests for Anthropic Messages API endpoint (Claude API)
class TestAnthropicTransformer - Test transformation between Anthropic and OpenAI formats
test_transform_anthropic_to_openai_basic - Test basic message transformation
test_transform_anthropic_to_openai_with_system - Test transformation with system message
test_transform_anthropic_to_openai_with_params - Test transformation with all parameters
test_transform_anthropic_to_openai_content_blocks - Test transformation with content blocks
test_transform_anthropic_to_openai_single_text_block - Test transformation with single text block (should unwrap)
test_transform_anthropic_to_openai_image_blocks - Test transformation with image content blocks
test_transform_openai_to_anthropic_basic - Test OpenAI to Anthropic response transformation
test_transform_openai_to_anthropic_finish_reasons - Test finish reason mapping
test_transform_openai_to_anthropic_tool_calls - Test OpenAI to Anthropic transformation with tool_calls
test_extract_text_from_string_content - Test extracting text from string content
test_extract_text_from_content_blocks - Test extracting text from content blocks
test_extract_text_from_mixed_blocks - Test extracting text from mixed content blocks
test_extract_text_from_empty_blocks - Test extracting text from empty content
class TestMessagesEndpointSuccess - Test successful message completions
test_messages_endpoint_basic_success - Test successful Claude API message completion
class TestMessagesEndpointAuth - Test authentication and authorization
test_messages_endpoint_no_auth_header - Test request without Authorization header
test_messages_endpoint_invalid_api_key - Test request with invalid API key
class TestMessagesEndpointCredits - Test credit validation and deduction
test_messages_endpoint_insufficient_credits - Test request with insufficient credits
class TestMessagesEndpointRateLimiting - Test rate limiting enforcement
test_messages_endpoint_rate_limit_exceeded - Test rate limit exceeded
class TestMessagesEndpointPlanLimits - Test plan limit enforcement
test_messages_endpoint_plan_limit_exceeded - Test plan limit exceeded
class TestMessagesEndpointTrialValidation - Test trial access validation
test_messages_endpoint_trial_expired - Test expired trial access
class TestMessagesEndpointValidation - Test request validation
test_messages_endpoint_missing_max_tokens - Test request without required max_tokens
test_messages_endpoint_empty_messages - Test request with empty messages array
test_messages_endpoint_invalid_role - Test request with invalid message role
class TestMessagesEndpointFailover - Test provider failover logic
test_messages_endpoint_provider_failover_success - Test successful failover to backup provider
class TestAnthropicTransformerNewFeatures - Test new Anthropic API features added for Claude API alignment
test_transform_system_as_array - Test transformation with system as array of content blocks
test_transform_document_content_block - Test transformation of document content blocks
test_transform_tool_result_content_block - Test transformation of tool_result content blocks
test_transform_tool_choice_types - Test transformation of different tool_choice types
test_transform_tools_to_openai_format - Test transformation of Anthropic tool definitions to OpenAI format
test_transform_assistant_tool_use_message - Test transformation of assistant messages with tool_use blocks
test_transform_openai_to_anthropic_with_stop_sequence - Test response transformation detecting stop sequences
test_transform_openai_to_anthropic_with_thinking - Test response transformation with reasoning/thinking content
test_transform_openai_to_anthropic_refusal - Test response transformation mapping content_filter to refusal
class TestMessagesRequestSchema - Test MessagesRequest schema validation
test_valid_request_with_string_system - Test valid request with string system prompt
test_valid_request_with_array_system - Test valid request with array system prompt
test_valid_request_with_service_tier - Test valid request with service_tier parameter
test_valid_request_with_thinking_config - Test valid request with thinking configuration
test_thinking_config_budget_validation - Test thinking budget_tokens validation (must be >= 1024)
test_temperature_validation - Test temperature must be between 0.0 and 1.0
test_content_block_types - Test ContentBlock supports all required types
test_tool_choice_models - Test tool choice model types
class TestMessagesResponseSchema - Test MessagesResponse schema
test_response_schema_structure - Test MessagesResponse has correct structure
test_usage_response_with_cache_fields - Test UsageResponse with cache-related fields
test_stop_reason_values - Test all valid stop_reason values
References: src/main.py, src/routes/monitoring.py
Module: Tests for monitoring API endpoints.
class TestHealthEndpoints - Test provider health endpoints
test_get_all_provider_health - Test getting health scores for all providers
test_get_provider_health - Test getting health score for specific provider
class TestErrorEndpoints - Test error tracking endpoints
test_get_provider_errors - Test getting recent errors for a provider
test_get_provider_errors_with_limit - Test error endpoint with custom limit
class TestStatsEndpoints - Test statistics endpoints
test_get_realtime_stats - Test getting real-time statistics
test_get_hourly_stats - Test getting hourly stats for a provider
class TestCircuitBreakerEndpoints - Test circuit breaker endpoints
test_get_all_circuit_breakers - Test getting all circuit breaker states
test_get_provider_circuit_breakers - Test getting circuit breakers for specific provider
class TestProviderComparisonEndpoint - Test provider comparison endpoint
test_get_provider_comparison - Test provider comparison endpoint
class TestLatencyEndpoints - Test latency tracking endpoints
test_get_latency_percentiles - Test getting latency percentiles
test_get_latency_percentiles_custom - Test latency percentiles with custom percentiles
test_get_latency_trends - Test getting latency trends
class TestAnomalyEndpoints - Test anomaly detection endpoints
test_get_anomalies - Test anomaly detection endpoint
class TestBusinessMetricsEndpoints - Test business metrics endpoints
test_get_trial_analytics - Test trial analytics endpoint
test_get_cost_analysis - Test cost analysis endpoint
test_get_error_rates - Test error rates endpoint
test_get_token_efficiency - Test token efficiency endpoint
class TestSentryTunnelEndpoint - Test Sentry tunnel endpoint for frontend error tracking
test_sentry_tunnel_empty_body - Test that empty body returns 400
test_sentry_tunnel_invalid_envelope - Test that invalid envelope format returns 400
test_sentry_tunnel_no_dsn - Test that envelope without DSN returns 400
test_sentry_tunnel_blocked_host - Test that non-Sentry hosts are blocked
test_sentry_tunnel_ssrf_prevention_suffix_attack - Test that SSRF attacks via suffix matching are blocked (e.g., evil-sentry.io)
test_sentry_tunnel_ssrf_prevention_malicious_subdomain - Test that SSRF attacks via malicious subdomains are blocked
test_sentry_tunnel_null_hostname - Test that malformed DSN with null hostname is rejected
test_sentry_tunnel_non_dict_json - Test that non-dict JSON envelope returns 400
test_sentry_tunnel_string_json - Test that string JSON envelope returns 400
test_sentry_tunnel_number_json - Test that number JSON envelope returns 400
test_sentry_tunnel_null_json - Test that null JSON envelope returns 400
test_sentry_tunnel_valid_envelope - Test that valid Sentry envelope is forwarded
References: src/routes/nosana.py
Module: Tests for Nosana GPU Computing Network routes
class TestNosanaCreditsEndpoints - Test Nosana credits endpoints
test_get_credit_balance - Test getting credit balance
class TestNosanaDeploymentEndpoints - Test Nosana deployment endpoints
test_list_deployments - Test listing deployments
test_get_deployment - Test getting deployment details
test_create_deployment - Test creating a deployment
class TestNosanaQuickDeployEndpoints - Test Nosana quick deploy endpoints
test_deploy_llm_inference - Test deploying LLM inference
test_deploy_image_generation - Test deploying image generation
test_deploy_whisper - Test deploying Whisper transcription
class TestNosanaJobsEndpoints - Test Nosana jobs endpoints
test_create_job - Test creating a job
test_get_job - Test getting job details
test_extend_job - Test extending job duration
test_stop_job - Test stopping a job
class TestNosanaMarketsEndpoints - Test Nosana markets endpoints
test_list_markets - Test listing markets
test_get_market - Test getting market details
test_get_market_resources - Test getting market resource requirements
class TestNosanaConfigEndpoint - Test Nosana config endpoint
test_get_config - Test getting Nosana configuration
class TestNosanaRouteValidation - Test Nosana route input validation
test_deployment_create_model_validation - Test deployment creation model validation
test_deployment_create_timeout_bounds - Test deployment timeout bounds
test_deployment_replicas_bounds - Test deployment replicas bounds
test_job_create_model_validation - Test job creation model validation
test_llm_inference_job_create_validation - Test LLM inference job creation validation
References: src/routes/notifications.py
Module: Comprehensive tests for Notifications routes
class TestNotificationsRoutes - Test Notifications route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/routes/optimization_monitor.py
Module: Comprehensive tests for Optimization Monitor routes
class TestOptimizationMonitorRoutes - Test Optimization Monitor route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/main.py, src/routes/partner_trials.py
Module: Tests for Partner Trials API Routes
class TestPartnerTrialsPublicEndpoints - Tests for public (unauthenticated) endpoints
test_get_partner_config_success - Test getting partner config for valid partner
test_get_partner_config_not_found - Test getting config for non-existent partner
test_check_partner_code_valid - Test checking a valid partner code
test_check_partner_code_invalid - Test checking a user referral code (not a partner code)
class TestPartnerTrialsAuthenticatedEndpoints - Tests for authenticated endpoints
test_start_partner_trial_success - Test starting a partner trial
test_start_partner_trial_invalid_partner - Test starting trial with invalid partner code
test_get_partner_trial_status_active - Test getting active partner trial status
test_get_partner_trial_status_no_trial - Test getting status when no partner trial exists
test_get_daily_limit_partner_user - Test getting daily limit for partner trial user
test_get_daily_limit_unlimited - Test getting daily limit for paid subscriber (unlimited)
class TestPartnerTrialsAdminEndpoints - Tests for admin endpoints
test_get_partner_analytics - Test getting partner analytics
test_force_expire_trial - Test force expiring a trial
References: src/db/payments.py, src/main.py, src/routes/payments.py, src/schemas/payments.py, src/services/payments.py
Module: Comprehensive tests for Stripe payment endpoints
class TestStripeWebhooks - Test Stripe webhook handling
test_webhook_success - Test successful webhook processing
test_webhook_missing_signature - Test webhook without signature
test_webhook_invalid_signature - Test webhook with invalid signature
class TestCheckoutSessions - Test checkout session operations
test_create_checkout_session_success - Test successfully creating checkout session
test_create_checkout_session_validation_error - Test checkout session with validation error
test_get_checkout_session_success - Test retrieving checkout session
class TestPaymentIntents - Test payment intent operations
test_create_payment_intent_success - Test successfully creating payment intent
test_get_payment_intent_success - Test retrieving payment intent
class TestCreditPackages - Test credit package listing
test_get_credit_packages_success - Test getting available credit packages
class TestRefunds - Test refund operations
test_create_refund_as_admin_success - Test admin creating refund
test_create_refund_as_non_admin_fails - Test non-admin cannot create refund
class TestPaymentHistory - Test payment history retrieval
test_get_payment_history_success - Test getting payment history
test_get_payment_history_with_pagination - Test payment history with pagination
test_get_payment_details_success - Test getting specific payment details
test_get_payment_details_not_found - Test getting non-existent payment
test_get_payment_details_wrong_user - Test accessing another user's payment
class TestPaymentsIntegration - Test payment workflow integration
test_complete_payment_flow - Test complete payment workflow
References: src/db/ping.py, src/routes/ping.py, src/services/ping.py
Module: Comprehensive tests for Ping routes
class TestPingRoutes - Test Ping route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/db/plans.py, src/routes/plans.py, src/schemas/plans.py
Module: Comprehensive tests for Plans routes
class TestPlansRoutes - Test Plans route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/routes/provider_credits.py
Module: Tests for provider credit balance API endpoints.
class TestGetProviderCreditBalances - Test GET /api/provider-credits/balance endpoint
test_get_balances_as_admin - Test retrieving all provider credit balances as admin
test_get_balances_with_warning_status - Test retrieving balance with warning status
test_get_balances_unauthorized - Test that non-admin users cannot access credit balances
class TestGetSpecificProviderBalance - Test GET /api/provider-credits/balance/{provider} endpoint
test_get_openrouter_balance - Test retrieving OpenRouter specific balance
test_get_unsupported_provider - Test retrieving balance for unsupported provider
test_get_balance_with_error - Test handling errors during balance retrieval
class TestClearProviderCreditCache - Test POST /api/provider-credits/balance/clear-cache endpoint
test_clear_all_cache - Test clearing all provider credit caches
test_clear_specific_provider_cache - Test clearing cache for specific provider
test_clear_cache_unauthorized - Test that non-admin users cannot clear cache
References: src/db/ranking.py, src/routes/ranking.py
Module: Comprehensive tests for Ranking routes
class TestRankingRoutes - Test Ranking route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/db/rate_limits.py, src/main.py, src/routes/rate_limits.py, src/security/deps.py
Module: Comprehensive tests for Rate Limits routes
class TestRateLimitsRoutes - Test Rate Limits route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
test_default_config_defined - Test that default rate limit config is defined
class TestAdminRateLimitsConfig - Test admin rate limit config endpoints
test_get_config_default - Get default rate limit configuration
test_get_config_for_specific_key - Get rate limit config for specific API key
test_set_config_success - Successfully set rate limit configuration
class TestAdminRateLimitsReset - Test admin rate limit reset endpoint
test_reset_config_success - Successfully reset rate limit to defaults
test_reset_config_missing_key - Reset config requires api_key parameter
class TestAdminRateLimitsUpdate - Test admin rate limit update endpoint
test_update_rate_limits_success - Successfully update rate limits
class TestAdminRateLimitsDelete - Test admin rate limit delete endpoint
test_delete_rate_limits_success - Successfully delete rate limits
class TestAdminRateLimitsUsers - Test admin rate limits users listing endpoint
test_get_users_rate_limits - Get rate limits for all users
test_get_users_rate_limits_with_filters - Get rate limits with filters
class TestAdminRateLimitsAlerts - Test admin rate limit alerts endpoint
test_get_alerts_success - Get rate limit alerts
class TestAdminRateLimitsSystem - Test admin system rate limits endpoint
test_get_system_stats - Get system-wide rate limit stats
class TestRateLimitsAuthorization - Test authorization for rate limits endpoints
test_admin_endpoints_require_admin - Admin rate limit endpoints require admin authentication
References: src/db/referral.py, src/main.py, src/routes/referral.py, src/security/deps.py, src/services/referral.py
Module: Tests for Referral System Routes
class TestReferralStats - Test referral stats endpoint
test_get_referral_stats_success - Successfully get referral stats
test_get_referral_stats_requires_auth - Referral stats requires authentication
test_get_referral_stats_invalid_api_key - Invalid API key returns 401
class TestReferralValidation - Test referral code validation
test_validate_referral_code_success - Successfully validate a referral code
test_validate_invalid_referral_code - Invalid referral code returns error
test_validate_referral_requires_code - Validate referral requires code parameter
class TestReferralCodeGeneration - Test referral code generation
test_referral_code_format - Referral code should follow format
test_referral_code_uniqueness - Referral codes should be unique
class TestSelfReferralPrevention - Test self-referral prevention
test_cannot_use_own_referral_code - User cannot use their own referral code
class TestReferralRewards - Test referral reward distribution
test_referrer_reward_calculation - Calculate referrer reward correctly
test_first_purchase_bonus - Additional bonus on first purchase
test_total_referral_earnings - Calculate total referral earnings
class TestReferralLimits - Test referral limits
test_referral_code_usage_limit - Referral code should have usage limit
test_referral_code_exhausted - Exhausted referral code should be invalid
test_user_can_only_use_one_code - User can only use one referral code
class TestReferralAnalytics - Test referral analytics
test_referral_list_includes_details - Referral stats should include referral details
class TestReferralEdgeCases - Test edge cases
test_referral_code_case_insensitive - Referral codes should be case insensitive
test_referral_code_with_spaces - Referral code with spaces should be trimmed
test_empty_referral_code - Empty referral code should be rejected
test_very_long_referral_code - Very long referral code should be rejected
class TestInviteLink - Test invite link generation
test_invite_link_format - Invite link should have correct format
class TestReferralSecurity - Test referral security
test_referral_code_sql_injection - SQL injection in referral code should be prevented
test_referral_code_xss_prevention - XSS in referral code should be prevented
References: src/routes/ai_sdk.py, src/routes/chat.py
Module: Test script for the new /v1/responses endpoint (OpenAI unified API)
test_chat_completions_endpoint - Test the legacy /v1/chat/completions endpoint
test_responses_endpoint - Test the new /v1/responses endpoint
test_responses_with_json_format - Test /v1/responses with JSON response format
test_streaming_responses - Test /v1/responses with streaming
References: src/db/roles.py, src/main.py, src/routes/roles.py, src/services/roles.py
Module: Comprehensive tests for role management API endpoints
class TestUpdateUserRole - Test role update endpoint
test_update_role_success - Test successfully updating user's role
test_update_role_without_reason - Test updating role without providing reason
test_update_role_invalid_role - Test updating to invalid role
test_update_role_failure - Test role update failure
test_update_role_exception - Test role update with exception
test_update_role_missing_fields - Test validation error for missing required fields
class TestGetUserRoleInfo - Test retrieving user role and permissions
test_get_user_role_info_success - Test retrieving user's role and permissions
test_get_user_role_info_not_found - Test retrieving role for non-existent user
test_get_user_role_info_no_permissions - Test user with role but no permissions
test_get_user_role_info_exception - Test exception handling
class TestRoleAuditLog - Test role change audit log retrieval
test_get_audit_log_all - Test retrieving all audit log entries
test_get_audit_log_by_user - Test retrieving audit log for specific user
test_get_audit_log_with_limit - Test retrieving audit log with custom limit
test_get_audit_log_empty - Test retrieving empty audit log
test_get_audit_log_exception - Test exception handling
class TestListUsersByRole - Test listing users by role
test_list_users_by_role_success - Test listing users with specific role
test_list_users_by_role_with_limit - Test listing users with custom limit
test_list_users_invalid_role - Test listing users with invalid role
test_list_users_empty - Test listing users when none have role
test_list_users_exception - Test exception handling
class TestGetRolePermissions - Test retrieving role permissions
test_get_role_permissions_success - Test retrieving permissions for a role
test_get_role_permissions_invalid_role - Test getting permissions for invalid role
test_get_role_permissions_empty - Test getting permissions when role has none
test_get_role_permissions_exception - Test exception handling
class TestRoleAuthorization - Test admin-only authorization
test_all_endpoints_require_admin - Test that all role endpoints require admin access
class TestRoleManagementIntegration - Test role management integration scenarios
test_role_update_creates_audit_entry - Test that updating role creates audit log entry
test_role_update_affects_user_list - Test that role updates affect user list by role
References: src/routes/root.py
Module: Comprehensive tests for Root routes
class TestRootRoutes - Test Root route handlers
test_router_exists - Test that router is defined
test_module_imports - Test that module imports successfully
References: src/routes/share.py, src/schemas/share.py, src/security/deps.py
Module: Tests for the chat share API endpoints.
test_create_share_link_success - Test successfully creating a share link
test_create_share_link_unauthenticated - Test creating share link without authentication
test_create_share_link_rate_limited - Test creating share link when rate limited
test_create_share_link_session_not_found - Test creating share link for non-existent session
test_create_share_link_with_expiry - Test creating share link with expiration date
test_get_my_share_links_success - Test getting user's share links
test_get_my_share_links_empty - Test getting share links when none exist
test_get_my_share_links_unauthenticated - Test getting share links without authentication
test_get_shared_chat_success - Test getting shared chat by token (public endpoint)
test_get_shared_chat_not_found - Test getting non-existent shared chat
test_get_shared_chat_expired - Test getting expired shared chat
test_delete_share_link_success - Test deleting a share link
test_delete_share_link_not_found - Test deleting non-existent share link
test_delete_share_link_unauthenticated - Test deleting share link without authentication
test_create_share_link_missing_session_id - Test creating share link without session_id
test_create_share_link_invalid_expiry_format - Test creating share link with invalid expiry format
References: src/routes.py, src/security.py
Module: Tests for share route import fixes.
test_share_imports_successfully - Test that share module imports without errors.
test_share_uses_correct_security_imports - Test that share uses get_optional_api_key (not get_api_key_optional).
test_security_deps_exports - Test that security.deps exports the correct functions.
test_share_endpoints_defined - Test that share endpoints are properly defined.
test_share_router_prefix - Test that share router has the correct prefix.
test_share_router_tags - Test that share router has the correct tags.
References: src/main.py, src/services/simplismart_client.py
Module: Test SimpliSmart models loading through the catalog endpoint.
class TestSimplismartModelsLoading - Test suite for SimpliSmart models loading.
test_simplismart_client_returns_models_with_source_gateway - Verify SimpliSmart client returns models with source_gateway field.
test_catalog_endpoint_returns_simplismart_models - Verify /models endpoint returns SimpliSmart models when gateway=simplismart.
test_catalog_endpoint_includes_simplismart_in_all_gateway - Verify /models endpoint includes SimpliSmart models when gateway=all.
test_simplismart_models_have_required_fields - Verify SimpliSmart models have all required fields for frontend filtering.
test_v1_models_endpoint_returns_simplismart - Verify /v1/models endpoint also returns SimpliSmart models.
References: src/routes/chat.py
Module: Unit tests for stream_generator function.
class TestStreamGeneratorAnonymous - Tests for stream_generator with anonymous users.
test_anonymous_stream_does_not_call_enforce_plan_limits - Test that anonymous streaming does not call enforce_plan_limits.
test_anonymous_stream_completes_successfully - Test that anonymous streaming completes without errors.
class TestStreamGeneratorAuthenticated - Tests for stream_generator with authenticated users.
test_authenticated_stream_calls_enforce_plan_limits - Test that authenticated streaming calls enforce_plan_limits.
test_authenticated_stream_plan_limit_exceeded - Test that plan limit exceeded error is properly returned.
References: src/main.py, src/routes/payments.py
Module: Tests for Subscription Management Routes
class TestGetCurrentSubscriptionEndpoint - Tests for GET /api/stripe/subscription endpoint
test_get_subscription_success - Test successful subscription retrieval
test_get_subscription_no_subscription - Test getting subscription when user has none
class TestUpgradeSubscriptionEndpoint - Tests for POST /api/stripe/subscription/upgrade endpoint
test_upgrade_success - Test successful subscription upgrade
test_upgrade_no_subscription - Test upgrading when user has no subscription
test_upgrade_invalid_proration_behavior - Test upgrade with invalid proration behavior
class TestDowngradeSubscriptionEndpoint - Tests for POST /api/stripe/subscription/downgrade endpoint
test_downgrade_success - Test successful subscription downgrade
test_downgrade_no_subscription - Test downgrading when user has no subscription
class TestCancelSubscriptionEndpoint - Tests for POST /api/stripe/subscription/cancel endpoint
test_cancel_at_period_end_success - Test canceling subscription at end of billing period
test_cancel_immediately_success - Test canceling subscription immediately
test_cancel_default_behavior - Test cancel with default behavior (cancel at period end)
test_cancel_no_subscription - Test canceling when user has no subscription
class TestSubscriptionEndpointsAuthentication - Tests for authentication on subscription endpoints
test_get_subscription_no_auth - Test getting subscription without authentication
test_upgrade_no_auth - Test upgrading without authentication
test_downgrade_no_auth - Test downgrading without authentication
test_cancel_no_auth - Test canceling without authentication
References: src/main.py, src/routes/system.py
Module: Comprehensive tests for system endpoints (cache management and gateway health)
class TestCacheStatus - Test cache status retrieval
test_get_cache_status_with_data - Test cache status with data
test_get_cache_status_empty_cache - Test cache status with empty cache
test_get_cache_status_stale_cache - Test cache status with stale cache
test_get_cache_status_error_handling - Test cache status error handling
class TestCacheRefresh - Test cache refresh operations
test_refresh_gateway_cache_success - Test successful cache refresh
test_refresh_gateway_cache_skip_if_valid - Test cache refresh skipped if cache is still valid
test_refresh_gateway_cache_force - Test forced cache refresh
test_refresh_gateway_cache_invalid_gateway - Test refresh with invalid gateway
test_refresh_gateway_cache_deepinfra_not_supported - Test that DeepInfra doesn't support bulk refresh
class TestCacheClear - Test cache clearing operations
test_clear_specific_gateway_cache - Test clearing cache for specific gateway
test_clear_all_caches - Test clearing all caches
test_clear_cache_error_handling - Test cache clear error handling
class TestGatewayHealth - Test gateway health monitoring
test_check_all_gateways_healthy - Test all gateways healthy
test_check_all_gateways_some_unhealthy - Test with some gateways unhealthy
test_check_all_gateways_timeout - Test gateway timeout handling
test_check_single_gateway_success - Test checking single gateway
test_check_single_gateway_not_found - Test checking non-existent gateway
class TestModelzCacheManagement - Test Modelz cache management endpoints
test_get_modelz_cache_status_success - Test getting Modelz cache status
test_get_modelz_cache_status_error - Test Modelz cache status error handling
test_refresh_modelz_cache_success - Test refreshing Modelz cache
test_refresh_modelz_cache_error - Test Modelz cache refresh error handling
test_clear_modelz_cache_success - Test clearing Modelz cache
test_clear_modelz_cache_error - Test Modelz cache clear error handling
class TestSystemIntegration - Test system endpoint integration scenarios
test_cache_workflow_status_refresh_status - Test complete cache workflow: check status, refresh, check again
test_health_check_with_cache_info - Test health check includes cache information
class TestGatewayHealthDashboard - Test gateway health dashboard status calculations
test_gateway_dashboard_endpoint_and_cache_pass - Test gateway marked healthy when both endpoint and cache pass
test_gateway_dashboard_endpoint_pass_cache_fail - Test gateway marked healthy when endpoint passes but cache fails (OR logic)
test_gateway_dashboard_endpoint_fail_cache_pass - Test gateway marked healthy when cache passes but endpoint fails (cache-only gateway)
test_gateway_dashboard_both_fail - Test gateway marked unhealthy when both endpoint and cache fail
test_gateway_dashboard_unconfigured - Test gateway marked unconfigured when no API key is configured
test_gateway_dashboard_html_rendering_with_or_logic - Test HTML dashboard rendering applies OR logic correctly for health status
class TestCacheInvalidation - Test cache invalidation endpoint
test_invalidate_cache_specific_gateway - Test invalidating cache for a specific gateway
test_invalidate_cache_gateway_case_insensitive - Test that gateway name is converted to lowercase
test_invalidate_cache_type_models - Test invalidating all model caches
test_invalidate_cache_type_providers - Test invalidating providers cache
test_invalidate_cache_type_pricing - Test invalidating pricing cache
test_invalidate_cache_all - Test invalidating all caches when no parameters provided
test_invalidate_cache_error_handling - Test error handling in cache invalidation
test_invalidate_cache_response_format - Test that response has correct format
References: src/main.py, src/routes/transaction_analytics.py
Module: Comprehensive tests for transaction analytics endpoints
class TestGetTransactionAnalytics - Test /analytics/transactions endpoint
test_get_transaction_analytics_success - Test successfully fetching transaction analytics
test_get_transaction_analytics_different_windows - Test fetching analytics with different time windows
test_get_transaction_analytics_default_window - Test default window parameter
test_get_transaction_analytics_missing_cookie - Test that missing OPENROUTER_COOKIE returns 503
test_get_transaction_analytics_auth_failure - Test handling OpenRouter authentication failure
test_get_transaction_analytics_api_error - Test handling OpenRouter API errors
test_get_transaction_analytics_timeout - Test handling request timeout
test_get_transaction_analytics_request_error - Test handling request errors
class TestGetTransactionSummary - Test /analytics/transactions/summary endpoint
test_get_transaction_summary_success - Test successfully generating transaction summary
test_get_transaction_summary_aggregation - Test summary data aggregation
test_get_transaction_summary_token_stats - Test token statistics calculation
test_get_transaction_summary_usage_stats - Test usage (cost) statistics calculation
test_get_transaction_summary_empty_data - Test summary with empty data
test_get_transaction_summary_with_reasoning_tokens - Test summary correctly handles reasoning tokens
test_get_transaction_summary_rounding - Test numerical rounding in summary
test_get_transaction_summary_error_propagation - Test error propagation from get_transaction_analytics
class TestTransactionAnalyticsIntegration - Test transaction analytics integration scenarios
test_raw_and_summary_consistency - Test raw data and summary are consistent
test_multiple_window_queries - Test querying multiple time windows
test_public_access_no_auth_required - Test that endpoints are public (no authentication required)
References: src/db/users.py, src/main.py, src/routes/users.py, src/schemas/users.py
Module: Comprehensive tests for user endpoints
class TestUserBalance - Test user balance retrieval
test_get_balance_active_user - Test getting balance for active (non-trial) user
test_get_balance_trial_user - Test getting balance for trial user
test_get_balance_invalid_api_key - Test balance retrieval with invalid API key
test_get_balance_error_handling - Test balance error handling
class TestUserMonitor - Test user monitoring and metrics
test_user_monitor_success - Test successful user monitoring
test_user_monitor_no_rate_limits - Test user monitoring when no rate limits configured
test_user_monitor_failed_metrics - Test monitoring when metrics retrieval fails
test_user_monitor_invalid_key - Test monitoring with invalid API key
class TestRateLimits - Test rate limit retrieval
test_get_rate_limits_configured - Test getting configured rate limits
test_get_rate_limits_default - Test getting default rate limits when none configured
test_get_rate_limits_invalid_key - Test rate limits with invalid API key
class TestUserProfile - Test user profile management
test_get_profile_success - Test successfully getting user profile
test_get_profile_with_max_tier_tiered_credits - Test profile response includes tiered credit fields for MAX tier user
test_get_profile_tier_display_names - Test that tier display names are correctly mapped for all tiers
test_get_profile_not_found - Test getting profile when profile retrieval fails
test_update_profile_success - Test successfully updating user profile
test_update_profile_no_fields - Test updating profile with no fields provided
test_update_profile_failed - Test profile update failure
test_update_profile_multiple_fields - Test updating multiple profile fields
class TestAccountDeletion - Test account deletion
test_delete_account_success - Test successfully deleting account
test_delete_account_wrong_confirmation - Test account deletion with wrong confirmation
test_delete_account_failed - Test account deletion failure
test_delete_account_invalid_key - Test account deletion with invalid API key
class TestCreditTransactions - Test credit transaction history
test_get_transactions_success - Test successfully getting credit transactions
test_get_transactions_with_filters - Test getting transactions with filters
test_get_transactions_pagination - Test transaction pagination
test_get_transactions_invalid_key - Test transactions with invalid API key
test_get_transactions_error_handling - Test transaction retrieval error handling
class TestUserIntegration - Test user endpoint integration scenarios
test_complete_user_info_flow - Test getting complete user information
test_profile_update_workflow - Test complete profile update workflow
test_trial_user_journey - Test trial user accessing balance
References: src/routes/admin.py, src/schemas.py, src/schemas/admin.py
Module: Comprehensive tests for Admin schemas
class TestAdminSchemas - Test Admin schema models
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/db/api_keys.py, src/routes/api_keys.py, src/schemas.py, src/schemas/api_keys.py
Module: Comprehensive tests for Api Keys schemas
class TestApiKeysSchemas - Test Api Keys schema models
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/schemas.py, src/schemas/common.py
Module: Comprehensive tests for Common schemas
class TestCommonSchemas - Test Common schema models
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/db/coupons.py, src/routes/coupons.py, src/schemas.py, src/schemas/coupons.py
Module: Comprehensive tests for Coupons schemas
class TestCouponsSchemas - Test Coupons schema models
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/schemas/errors.py
Module: Unit tests for error response Pydantic schemas.
class TestErrorContext - Test suite for ErrorContext schema.
test_minimal_context - Test creating minimal context with no fields.
test_model_not_found_context - Test context for model not found error.
test_insufficient_credits_context - Test context for insufficient credits error.
test_rate_limit_context - Test context for rate limit error.
test_parameter_validation_context - Test context for parameter validation error.
test_provider_error_context - Test context for provider error.
test_internal_error_context - Test context for internal error.
test_api_key_context - Test context for API key error.
test_serialization - Test context serialization to dict.
test_allowed_values_list - Test that allowed_values accepts list.
class TestErrorDetail - Test suite for ErrorDetail schema.
test_minimal_error_detail - Test creating minimal valid error detail.
test_full_error_detail - Test fully populated error detail.
test_missing_required_fields_raises_error - Test that missing required fields raises ValidationError.
test_suggestions_is_list - Test that suggestions must be a list.
test_status_is_integer - Test that status must be an integer.
test_invalid_status_type_raises_error - Test that invalid status type raises ValidationError.
test_serialization_excludes_none - Test that None values can be excluded from serialization.
test_context_nested_serialization - Test that nested context is properly serialized.
class TestErrorResponse - Test suite for ErrorResponse schema.
test_minimal_error_response - Test creating minimal valid error response.
test_full_error_response - Test fully populated error response.
test_missing_error_field_raises_error - Test that missing error field raises ValidationError.
test_serialization_structure - Test that serialization has correct structure.
test_json_serialization - Test that error response can be serialized to JSON.
test_nested_context_in_response - Test that nested context is properly included.
test_example_model_not_found_response - Test example model not found error response.
test_example_insufficient_credits_response - Test example insufficient credits error response.
test_example_rate_limit_response - Test example rate limit error response.
test_timestamp_validation - Test that timestamp format is validated.
test_urls_are_strings - Test that URL fields accept strings.
References: src/schemas.py, src/schemas/notification.py, src/services/notification.py
Module: Comprehensive tests for Notification schemas
class TestNotificationSchemas - Test Notification schema models
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/schemas/payments.py
Module: Test payment credit value validation logic
test_credit_value_minimum_100_percent - Test that credit value must be at least 100% of payment amount.
test_credit_value_less_than_payment_rejected - Test that credit value less than payment amount is rejected.
test_credit_value_bonus_packages_allowed - Test that bonus packages (credit > payment) are allowed.
test_credit_value_exceeds_3x_rejected - Test that credit value exceeding 3x payment is rejected.
test_credit_value_exactly_3x_allowed - Test that credit value exactly 3x payment is allowed.
test_credit_value_none_allowed - Test that None credit_value is allowed (uses payment amount as credit).
test_various_payment_amounts_with_valid_credits - Test various payment amounts with valid credit values.
test_edge_case_just_below_minimum - Test edge case just below 100% credit value.
test_edge_case_just_above_maximum - Test edge case just above 3x credit value.
test_fractional_credit_values - Test that fractional credit values are handled correctly.
test_promotional_packages_realistic_scenarios - Test realistic promotional package scenarios.
References: src/schemas/proxy.py
Module: Tests for proxy schemas, including tools field validation.
class TestMessageSchema - Test Message schema for OpenAI compatibility
test_basic_message - Test basic message creation
test_message_with_tool_calls - Test assistant message with tool_calls
test_tool_response_message - Test tool response message
test_function_message - Test function message with name
test_multimodal_content - Test message with multimodal content array
test_valid_roles - Test all valid message roles
test_invalid_role - Test invalid message role raises error
test_user_message_requires_content - Test that user messages must have content
test_system_message_requires_content - Test that system messages must have content
test_developer_message_requires_content - Test that developer messages must have content
test_developer_message_basic - Test basic developer message creation (OpenAI developer role)
test_tool_message_requires_content - Test that tool messages must have content (the response)
test_function_message_requires_content - Test that function messages must have content
test_assistant_message_allows_null_content_with_tool_calls - Test that assistant messages can have null content when tool_calls is present
test_assistant_message_requires_content_or_tool_calls - Test that assistant messages need either content or tool_calls
class TestProxyRequestOpenAIAlignment - Test ProxyRequest schema alignment with OpenAI Chat Completions API
test_minimal_request - Test minimal valid request
test_all_sampling_parameters - Test all sampling parameters
test_penalty_parameters - Test frequency and presence penalty parameters
test_penalty_bounds - Test penalty parameter bounds (-2.0 to 2.0)
test_temperature_bounds - Test temperature parameter bounds (0 to 2)
test_streaming_parameters - Test streaming parameters
test_tool_choice_string - Test tool_choice with string value
test_tool_choice_object - Test tool_choice with object value (specific function)
test_parallel_tool_calls - Test parallel_tool_calls parameter
test_response_format_json_object - Test response_format with json_object type
test_response_format_json_schema - Test response_format with json_schema type
test_logprobs_parameters - Test logprobs and top_logprobs parameters
test_top_logprobs_bounds - Test top_logprobs bounds (0 to 20)
test_logit_bias - Test logit_bias parameter
test_seed_parameter - Test seed parameter for deterministic sampling
test_user_parameter - Test user parameter for end-user identification
test_service_tier_parameter - Test service_tier parameter
test_stop_string - Test stop parameter with single string
test_stop_list - Test stop parameter with list of strings
test_stop_max_sequences - Test stop parameter enforces max 4 sequences
test_gateway_provider_parameter - Test gateway-specific provider parameter
test_extra_fields_allowed - Test that extra fields are allowed for forward compatibility
test_empty_messages_rejected - Test that empty messages list is rejected
test_n_minimum_value - Test n parameter minimum value (1)
test_complete_request - Test complete request with all parameters
class TestStreamOptionsSchema - Test StreamOptions schema
test_stream_options_include_usage - Test StreamOptions with include_usage
test_stream_options_default - Test StreamOptions with default values
class TestProxyRequestTools - Test ProxyRequest schema with tools field
test_proxy_request_without_tools - Test ProxyRequest without tools field
test_proxy_request_with_tools - Test ProxyRequest with tools field
test_proxy_request_with_empty_tools_list - Test ProxyRequest with empty tools list
test_proxy_request_tools_extra_fields - Test ProxyRequest accepts tools via extra fields (backward compatibility)
class TestResponseRequestTools - Test ResponseRequest schema with tools field
test_response_request_without_tools - Test ResponseRequest without tools field
test_response_request_with_tools - Test ResponseRequest with tools field
test_response_request_with_multiple_tools - Test ResponseRequest with multiple tools
class TestMessageContentValidation - Test empty string validation for message content
test_user_message_rejects_empty_string - Test that user messages reject empty string content
test_user_message_rejects_whitespace_only - Test that user messages reject whitespace-only content
test_system_message_rejects_empty_string - Test that system messages reject empty string content
test_developer_message_rejects_empty_string - Test that developer messages reject empty string content
test_developer_message_rejects_whitespace_only - Test that developer messages reject whitespace-only content
test_tool_message_rejects_empty_string - Test that tool messages reject empty string content
test_assistant_message_rejects_empty_without_tool_calls - Test that assistant messages reject empty content without tool_calls
test_assistant_message_allows_null_with_tool_calls - Test that assistant messages allow null content with tool_calls
test_user_message_allows_non_empty_string - Test that user messages accept valid non-empty content
test_multimodal_content_empty_list_rejected - Test that empty list content is rejected for user messages
References: src/db/trials.py, src/schemas.py, src/schemas/trials.py
Module: Comprehensive tests for Trials schemas
class TestTrialsSchemas - Test Trials schema models
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/db/users.py, src/routes/users.py, src/schemas.py, src/schemas/users.py
Module: Comprehensive tests for Users schemas
class TestUsersSchemas - Test Users schema models
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/routes/admin.py, src/security/deps.py
Module: Test script to verify admin endpoint security
test_admin_endpoints - Test that all admin endpoints require proper authentication
References: src/db_security.py
Module: Comprehensive tests for db_security module
class TestCreateSecureAPIKey - Test secure API key creation
test_create_secure_api_key_success - Test successful secure API key creation
test_create_secure_api_key_with_expiration - Test API key creation with expiration
test_create_secure_api_key_with_ip_allowlist - Test API key creation with IP allowlist
test_create_secure_api_key_duplicate_name - Test API key creation with duplicate name fails
test_create_secure_api_key_insert_fails - Test API key creation when insert fails
test_create_secure_api_key_with_custom_permissions - Test API key creation with custom scope permissions
class TestValidateSecureAPIKey - Test secure API key validation
test_validate_secure_api_key_success - Test successful API key validation
test_validate_secure_api_key_not_found - Test validation with non-existent key
test_validate_secure_api_key_inactive - Test validation with inactive key
test_validate_secure_api_key_expired - Test validation with expired key
test_validate_secure_api_key_ip_not_allowed - Test validation with disallowed IP
test_validate_secure_api_key_domain_not_allowed - Test validation with disallowed domain
test_validate_secure_api_key_request_limit_exceeded - Test validation when request limit is exceeded
test_validate_plain_text_key - Test validation of plain text (legacy) key
class TestRotateAPIKey - Test API key rotation
test_rotate_api_key_success - Test successful API key rotation
test_rotate_api_key_not_found - Test rotation with non-existent key
test_rotate_api_key_with_new_name - Test API key rotation with new name
class TestGetAuditLogs - Test audit log retrieval
test_get_audit_logs_all - Test getting all audit logs
test_get_audit_logs_by_user - Test getting audit logs filtered by user
test_get_audit_logs_by_action - Test getting audit logs filtered by action
test_get_audit_logs_with_date_range - Test getting audit logs within date range
test_get_audit_logs_empty - Test getting audit logs when none exist
test_get_audit_logs_error_handling - Test audit logs error handling
class TestBulkRotateUserKeys - Test bulk key rotation
test_bulk_rotate_user_keys_success - Test successful bulk rotation
test_bulk_rotate_user_keys_no_keys - Test bulk rotation when user has no keys
test_bulk_rotate_user_keys_with_environment - Test bulk rotation filtered by environment
test_bulk_rotate_user_keys_partial_failure - Test bulk rotation when some keys fail
class TestCheckKeyNameUniqueness - Test key name uniqueness checking
test_check_key_name_uniqueness_unique - Test that unique name returns True
test_check_key_name_uniqueness_duplicate - Test that duplicate name returns False
test_check_key_name_uniqueness_with_exclusion - Test uniqueness check excluding a specific key
test_check_key_name_uniqueness_error_handling - Test that errors return False (safe default)
References: src/security/deps.py
test_get_api_key_missing_credentials
test_get_api_key_missing_token
test_get_api_key_valid_logs_usage
test_get_api_key_valueerror_mapped
test_get_api_key_unexpected
test_get_api_key_no_violation_log_when_disabled - Verify that log_security_violations=False prevents security violation logging.
test_get_api_key_logs_violation_by_default - Verify that log_security_violations=True (default) logs security violations.
test_get_current_user_happy
test_get_current_user_404
test_require_admin_via_flag
test_require_admin_via_role
test_require_admin_denied_logs_violation
test_get_optional_api_key_no_credentials_returns_none
test_get_optional_api_key_invalid_returns_none
test_get_optional_api_key_valid_returns_key
test_get_optional_api_key_does_not_log_violations - Verify that get_optional_api_key does NOT log security violations.
test_get_optional_user_no_credentials_returns_none
test_get_optional_user_invalid_returns_none
test_get_optional_user_valid_returns_user
test_get_optional_user_does_not_log_violations - Verify that get_optional_user does NOT log security violations.
test_require_active_subscription_ok
test_require_active_subscription_forbidden
test_check_credits_ok
test_check_credits_402
test_get_user_id
test_verify_key_permissions_none_required_returns_key
test_verify_key_permissions_ok
test_verify_key_permissions_user_missing
test_verify_key_permissions_forbidden
References: src/main.py
Module: Tests for Injection Attack Prevention
class TestSQLInjectionPrevention - Test SQL injection prevention in all endpoints
test_sql_injection_in_username - SQL injection attempt in username parameter
test_sql_injection_in_email - SQL injection attempt in email parameter
test_sql_injection_in_search - SQL injection attempt in search parameters
test_sql_injection_union_attack - SQL injection UNION attack attempt
test_sql_injection_in_api_key_lookup - SQL injection in API key lookup
class TestXSSPrevention - Test XSS (Cross-Site Scripting) prevention
test_xss_in_username - XSS attempt in username
test_xss_in_chat_message - XSS attempt in chat message
test_xss_in_json_response - Verify JSON responses don't contain unescaped HTML
test_xss_svg_attack - XSS via SVG payload
class TestCommandInjection - Test command injection prevention
test_command_injection_in_model_name - Command injection via model name
test_command_injection_backticks - Command injection using backticks
test_command_injection_pipe - Command injection using pipe operator
class TestPathTraversal - Test path traversal prevention
test_path_traversal_in_filename - Path traversal attempt in filename parameter
test_path_traversal_encoded - Path traversal with URL encoding
test_path_traversal_windows_style - Path traversal with Windows path separators
class TestLDAPInjection - Test LDAP injection prevention
test_ldap_injection_in_username - LDAP injection attempt
test_ldap_wildcard_injection - LDAP wildcard injection
class TestHeaderInjection - Test HTTP header injection prevention
test_crlf_injection_in_header - CRLF injection attempt in custom header
class TestJSONInjection - Test JSON injection and manipulation
test_json_injection_nested_objects - Attempt to inject malicious JSON structures
test_json_with_null_bytes - JSON with null bytes
class TestInputValidation - Test general input validation
test_oversized_input - Reject or handle oversized inputs
test_special_unicode_characters - Handle special Unicode characters safely
test_null_character_injection - Null character injection attempt
class TestAPIKeyValidation - Test API key validation against injection
test_api_key_format_validation - Validate API key format
test_api_key_length_validation - API key length should be validated
class TestErrorMessagesSecurity - Test that error messages don't expose sensitive information
test_database_errors_not_exposed - Database errors should not expose internal details
test_stack_traces_not_exposed - Stack traces should not be exposed to clients
test_file_paths_not_exposed - File paths should not be exposed in errors
References: src/security/security.py
Module: Tests for Security Module
class TestAPIKeyHashing - Test API key hashing
test_hash_api_key_success - Successfully hash an API key
test_hash_api_key_deterministic - Same API key should produce same hash
test_hash_different_keys_different_hashes - Different API keys should produce different hashes
test_hash_api_key_requires_salt - Hashing should require API_GATEWAY_SALT
test_hash_api_key_rejects_short_salt - Hashing should reject salt shorter than 16 characters
class TestAPIKeyGeneration - Test API key generation
test_generate_api_key_success - Successfully generate an API key
test_generate_api_key_unique - Generated API keys should be unique
test_generate_api_key_test_environment - Generate test environment API key
test_generate_api_key_staging_environment - Generate staging environment API key
test_generate_api_key_development_environment - Generate development environment API key
test_generate_api_key_custom_length - Generate API key with custom length
test_generate_api_key_format - API key should follow expected format
class TestIPAllowlist - Test IP allowlist validation
test_validate_ip_exact_match - Validate IP with exact match
test_validate_ip_not_in_allowlist - Validate IP not in allowlist
test_validate_ip_cidr_range - Validate IP in CIDR range
test_validate_ip_outside_cidr_range - Validate IP outside CIDR range
test_validate_ip_empty_allowlist - Empty allowlist should allow all IPs
test_validate_ip_mixed_formats - Validate IP with mixed exact and CIDR formats
class TestDomainReferrers - Test domain referrer validation
test_validate_domain_exact_match - Validate domain with exact match
test_validate_domain_with_subdomain - Validate subdomain
test_validate_domain_not_in_allowlist - Validate domain not in allowlist
test_validate_domain_empty_allowlist - Empty allowlist should allow all domains
test_validate_domain_no_referer - No referer should be handled gracefully
class TestSecurityConstants - Test security-related constants and configurations
test_api_key_prefix - API keys should use gw_ prefix
test_salt_minimum_length - Salt should meet minimum length requirements
class TestSecurityEdgeCases - Test edge cases and error handling
test_hash_empty_string - Hash empty string should work or raise appropriate error
test_validate_ip_invalid_ip - Validate with invalid IP format
test_validate_domain_invalid_url - Validate with invalid URL format
class TestAuditLoggerFormat - Test AuditLogger log format for Railway compatibility
test_audit_logger_exists - AuditLogger should be available via get_audit_logger()
test_audit_logger_has_handlers - AuditLogger should have configured handlers
test_audit_logger_format_has_level_before_audit - Log format should have levelname before [AUDIT] for Railway compatibility
test_audit_logger_log_methods_exist - AuditLogger should have all required logging methods
test_audit_logger_uses_info_level_for_normal_events - Normal events should use INFO level
test_audit_logger_uses_warning_level_for_security_violations - Security violations should use WARNING level
References: src/services/aihubmix_client.py
Module: Unit tests for AiHubMix client
class TestGetAihubmixClient - Tests for get_aihubmix_client function
test_get_client_success - Test successful client initialization
test_get_client_missing_api_key - Test error when API key is missing
test_get_client_missing_app_code - Test error when APP-Code is missing
class TestMakeAihubmixRequest - Tests for making requests to AiHubMix
test_make_request_success - Test successful non-streaming request
test_make_streaming_request_success - Test successful streaming request
test_make_request_error - Test request error handling
class TestProcessAihubmixResponse - Tests for processing AiHubMix responses
test_process_response_success - Test successful response processing
test_process_response_no_usage - Test processing response without usage data
test_process_response_error - Test error handling in response processing
References: src/services/models.py
Module: Tests for AiHubMix model normalization with direct pricing
class TestAiHubMixNormalizationWithPricing - Test AiHubMix model normalization with pricing data
test_normalize_model_with_pricing - Test normalizing a model with pricing from AiHubMix API
test_normalize_model_filters_zero_pricing - Test that models with zero pricing are filtered out
test_normalize_model_filters_empty_pricing - Test that models with empty pricing are filtered out
test_normalize_model_missing_id - Test that models without ID are filtered out
test_normalize_model_with_small_pricing - Test normalizing a model with small pricing values
test_normalize_model_context_length - Test that context length is properly extracted
test_normalize_model_default_context_length - Test that default context length is used when not provided
test_normalize_model_with_model_id_field - Test normalizing a model with 'model_id' instead of 'id' field
test_normalize_model_with_desc_field - Test normalizing a model with 'desc' instead of 'description' field
test_missing_id_logs_debug_not_warning - Test that models missing both 'id' and 'model_id' log at DEBUG level, not WARNING
test_model_id_field_works_without_warnings - Test that models with 'model_id' field are processed without warnings
References: src/cache.py, src/config.py, src/services/models.py
Module: Test AIMO model fetching resilience and timeout handling.
class TestAIMOTimeoutConfiguration - Test that AIMO timeout configuration is in place.
test_aimo_fetch_timeout_configured - Verify AIMO fetch timeout is configurable and reasonable.
test_aimo_connect_timeout_configured - Verify AIMO connect timeout is configured.
test_aimo_max_retries_configured - Verify AIMO max retries is configured.
test_aimo_http_fallback_enabled_by_default - Verify HTTP fallback is enabled by default.
test_aimo_base_urls_configured - Verify multiple AIMO base URLs are available.
class TestAIMOStaleWhileRevalidate - Test stale-while-revalidate caching pattern.
test_fresh_cached_models_returns_fresh_data - Fresh cache within TTL should be returned.
test_fresh_cached_models_returns_none_when_expired - Fresh cache beyond TTL should return None.
test_stale_while_revalidate_returns_fresh_data - Should return data within fresh TTL.
test_stale_while_revalidate_returns_stale_data - Should return stale data within stale_ttl window.
test_stale_while_revalidate_returns_none_when_both_expired - Should return None when both fresh and stale caches expired.
test_stale_while_revalidate_returns_none_with_no_cache - Should return None when no cached data exists.
class TestAIMORetryLogic - Test AIMO retry logic with multiple URLs.
test_successful_fetch_on_first_try - Should return data on successful first attempt.
test_retry_on_timeout - Should retry on timeout exception.
test_http_fallback_when_https_fails - Should try HTTP fallback when HTTPS fails.
test_uses_stale_cache_when_all_retries_fail - Should return stale cache when all retries fail.
test_returns_empty_list_when_no_cache_and_all_fail - Should return empty list when no cache and all retries fail.
class TestRedirectHandling - Test HTTP redirect handling for AIMO model fetching.
test_follows_http_308_redirect - Should follow HTTP 308 redirects by using follow_redirects=True.
test_redirect_preserves_headers - Should preserve authorization headers during redirect.
class TestTimeoutConfiguration - Test that timeout configuration prevents thread blocking.
test_timeout_uses_httpx_timeout_object - Verify implementation uses httpx.Timeout with separate connect timeout.
test_fetch_timeout_is_short - Verify fetch timeout is significantly shorter than original 20s.
References: src/services/akash_client.py
Module: Tests for Akash ML client
class TestAkashClient - Test Akash client functionality
test_get_akash_client - Test getting Akash client
test_get_akash_client_no_key - Test getting Akash client without API key
test_make_akash_request_openai - Test making request to Akash
test_make_akash_request_openai_stream - Test making streaming request to Akash
test_process_akash_response - Test processing Akash response
test_process_akash_response_no_usage - Test processing Akash response without usage data
test_make_akash_request_with_kwargs - Test making request to Akash with additional parameters
References: src/cache.py, src/config.py, src/services.py, src/services/alibaba_cloud_client.py
test_list_alibaba_models_retries_with_china_region - Ensure we fall back to the China endpoint when the default fails with auth errors.
test_chat_request_prefers_cached_region - Subsequent requests should re-use the inferred working region to avoid extra failures.
test_get_client_uses_region_specific_keys - Ensure region-specific API keys override the default key.
test_region_attempt_order_uses_available_key_for_all_regions - When only one region-specific key is set, all regions should still be attempted.
test_failover_with_misconfigured_region_key - Test that failover works when user sets international key but it only works for China.
test_is_quota_error_detects_quota_exceeded - Test that _is_quota_error detects various quota error messages.
test_is_quota_error_does_not_match_auth_errors - Test that _is_quota_error doesn't match auth errors.
test_quota_error_raises_quota_exceeded_error - Test that quota errors are converted to QuotaExceededError.
test_quota_error_does_not_retry_other_regions - Test that quota errors don't trigger region failover.
class TestAlibabaQuotaErrorCaching - Tests for quota error caching in fetch_models_from_alibaba.
test_quota_error_caches_failure_state - Test that QuotaExceededError triggers caching of failure state.
test_quota_error_backoff_skips_api_calls - Test that subsequent calls during backoff period skip API calls.
test_quota_error_backoff_expires - Test that backoff expires after the configured duration.
test_successful_fetch_clears_quota_error - Test that a successful fetch clears the quota error state.
test_quota_error_backoff_not_overridden_by_cache_ttl - Test that quota error backoff (15 min) is not overridden by cache TTL (1 hour).
References: src/routes/analytics.py, src/services/analytics.py
Module: Tests for analytics service
class TestTrialAnalytics - Test trial analytics function
test_get_trial_analytics_returns_dict - Test that get_trial_analytics returns a dictionary
test_get_trial_analytics_has_required_keys - Test that result has all required keys
test_get_trial_analytics_default_values - Test that default values are zero (TODO implementation)
test_get_trial_analytics_value_types - Test that values have correct types
References: src/services/anonymous_rate_limiter.py
Module: Tests for anonymous rate limiter service.
class TestModelWhitelist - Tests for anonymous model whitelist
test_free_model_allowed - Test that whitelisted free models are allowed
test_non_free_model_rejected - Test that non-free models are rejected
test_free_suffix_but_not_whitelisted - Test that models with :free suffix but not whitelisted are rejected
test_model_case_sensitivity - Test model matching is case-insensitive
test_empty_and_none_models - Test handling of empty and None models
test_model_without_free_suffix - Test that models without :free suffix are rejected even if similar
class TestAnonymousRateLimiting - Tests for IP-based rate limiting
test_rate_limit_allows_first_request - Test that first request from IP is allowed
test_rate_limit_blocks_after_limit - Test that requests are blocked after daily limit
test_rate_limit_decrements_remaining - Test that remaining count decreases properly
test_different_ips_tracked_separately - Test that different IPs have separate rate limits
class TestValidateAnonymousRequest - Tests for full request validation (model + rate limit)
test_valid_request_allowed - Test that valid request with free model and under limit is allowed
test_non_free_model_rejected - Test that non-free model request is rejected
test_rate_limit_exceeded_rejected - Test that request over rate limit is rejected
class TestRecordAnonymousRequest - Tests for recording anonymous requests
test_record_increments_count - Test that recording a request increments the counter
test_record_returns_remaining - Test that recording returns correct remaining count
class TestIPHashing - Tests for IP hashing (privacy)
test_ip_hashing_consistency - Test that same IP produces same hash
test_ip_hashing_different_ips - Test that different IPs produce different hashes
test_ip_hash_length - Test that hash is truncated to expected length
class TestAnonymousStats - Tests for anonymous usage statistics
test_stats_empty - Test stats when no usage
test_stats_with_usage - Test stats after some usage
class TestRedisIntegration - Tests for Redis integration
test_redis_increment - Test Redis INCR operation
test_redis_get - Test Redis GET operation
test_redis_fallback_on_error - Test fallback to memory when Redis errors
References: src/services/anthropic_client.py
Module: Tests for Anthropic client.
class TestAnthropicClient - Test Anthropic client functionality
test_get_anthropic_client - Test getting Anthropic client
test_get_anthropic_client_no_key - Test getting Anthropic client without API key
test_make_anthropic_request - Test making request to Anthropic
test_make_anthropic_request_with_kwargs - Test making request to Anthropic with additional parameters
test_make_anthropic_request_error - Test handling errors from Anthropic
test_make_anthropic_request_stream - Test making streaming request to Anthropic
test_make_anthropic_request_stream_with_kwargs - Test making streaming request to Anthropic with additional parameters
test_make_anthropic_request_stream_error - Test handling streaming errors from Anthropic
test_process_anthropic_response - Test processing Anthropic response
test_process_anthropic_response_no_usage - Test processing Anthropic response without usage data
test_process_anthropic_response_multiple_choices - Test processing Anthropic response with multiple choices
class TestAnthropicModels - Test Anthropic model-specific functionality
test_claude_3_5_sonnet_request - Test making request with Claude 3.5 Sonnet
test_claude_3_opus_request - Test making request with Claude 3 Opus
test_claude_3_haiku_request - Test making request with Claude 3 Haiku
References: src/services/models.py
Module: Tests for Anthropic model fetching and normalization.
class TestFetchModelsFromAnthropic - Test Anthropic model fetching from API
test_fetch_models_no_api_key - Test fetching models without API key returns None
test_fetch_models_success - Test successful model fetching from API
test_fetch_models_pagination - Test model fetching with pagination
test_fetch_models_filters_non_claude - Test that non-Claude models are filtered out
test_fetch_models_http_error - Test handling of HTTP errors
class TestNormalizeAnthropicModel - Test Anthropic model normalization
test_normalize_basic_model - Test normalizing a basic model from API response
test_normalize_model_with_vision - Test that Claude 3+ models have vision support
test_normalize_model_max_output_3_5 - Test that Claude 3.5 models have 8192 max output
test_normalize_model_max_output_3_0 - Test that Claude 3.0 models have 4096 max output
test_normalize_model_missing_id - Test that models without ID return None
test_normalize_model_fallback_name - Test fallback to model ID when display_name is missing
References: src/services.py, src/services/auth_rate_limiting.py
Module: Tests for Authentication Rate Limiting Module
class TestAuthRateLimitConfig - Tests for AuthRateLimitConfig
test_default_config_values - Test default configuration values
test_custom_config - Test custom configuration values
class TestAuthRateLimiter - Tests for AuthRateLimiter
test_login_rate_limit_allows_under_limit - Test that requests under limit are allowed
test_login_rate_limit_blocks_over_limit - Test that requests over limit are blocked
test_register_rate_limit - Test registration rate limiting
test_password_reset_rate_limit - Test password reset rate limiting
test_api_key_create_rate_limit - Test API key creation rate limiting
test_different_ips_have_separate_limits - Test that different IPs are rate limited independently
test_different_limit_types_are_independent - Test that different rate limit types are tracked independently
test_get_remaining - Test getting remaining attempts
test_reset_clears_limit - Test that reset clears the rate limit
test_mask_identifier_ipv4 - Test IP address masking for IPv4
test_mask_identifier_short_string - Test identifier masking for short strings
test_mask_identifier_long_string - Test identifier masking for long strings
test_mask_identifier_empty - Test identifier masking for empty/None values
class TestGetClientIP - Tests for get_client_ip helper function
test_get_ip_from_x_real_ip - Test extracting IP from X-Real-IP header (highest priority)
test_get_ip_from_x_forwarded_for_rightmost - Test extracting rightmost IP from X-Forwarded-For header (proxy-added)
test_get_ip_from_x_forwarded_for_single - Test extracting IP from X-Forwarded-For header with single IP
test_get_ip_from_client_host - Test extracting IP from client.host when no X-Forwarded-For
test_get_ip_unknown_when_no_client - Test returning 'unknown' when no client info available
test_spoofed_x_forwarded_for_uses_proxy_added_ip - Test that spoofed X-Forwarded-For headers don't bypass rate limiting
class TestGlobalRateLimiter - Tests for global rate limiter functions
test_get_auth_rate_limiter_returns_singleton - Test that get_auth_rate_limiter returns the same instance
test_check_auth_rate_limit_convenience_function - Test the convenience function for checking rate limits
References: src/services/model_health_monitor.py, src/services/providers.py
Module: Tests for backend error fixes.
class TestModelHealthMonitorFixes - Tests for model_health_monitor.py error handling fixes.
test_latency_parsing_with_invalid_format - Test that invalid latency formats are handled gracefully without bare except.
test_latency_parsing_with_none_value - Test that None latency values are handled without crashing.
test_latency_parsing_with_valid_value - Test that valid latency values are parsed correctly.
class TestAuthorizationHeaderParsingFixes - Tests for authorization header parsing fixes in chat.py and messages.py.
test_malformed_authorization_header_without_token - Test that authorization header with just 'Bearer' doesn't crash.
test_malformed_authorization_header_empty_token - Test that authorization header with empty token is handled.
test_valid_authorization_header - Test that valid authorization header is parsed correctly.
test_authorization_header_with_extra_spaces - Test that authorization header with extra spaces is handled.
test_unified_responses_auth_header_parsing - Test that unified_responses endpoint (chat.py:2780) handles malformed headers.
class TestXForwardedForParsingFixes - Tests for X-Forwarded-For header parsing fixes in chat.py and staging_security.py.
test_empty_forwarded_for_header - Test that empty X-Forwarded-For header doesn't crash.
test_single_ip_forwarded_for - Test that single IP in X-Forwarded-For is parsed correctly.
test_multiple_ips_forwarded_for - Test that first IP from multiple IPs is extracted.
test_forwarded_for_with_spaces - Test that X-Forwarded-For with extra spaces is handled.
test_malformed_forwarded_for_ipv6 - Test that IPv6 addresses in X-Forwarded-For are handled.
test_middleware_forwarded_for_parsing - Test that middleware X-Forwarded-For parsing is defensive.
test_middleware_empty_forwarded_for - Test middleware handles empty X-Forwarded-For gracefully.
class TestUnsafeListAccessFixes - Tests for unsafe list access fixes in messages.py.
test_empty_choices_list_finish_reason - Test that empty choices list doesn't crash when accessing finish_reason.
test_none_choices_finish_reason - Test that None choices value doesn't crash.
test_valid_choices_finish_reason - Test that valid choices list works correctly.
test_empty_choices_list_assistant_content - Test that empty choices list doesn't crash when extracting assistant content.
test_none_choices_assistant_content - Test that None choices doesn't crash.
test_valid_choices_assistant_content - Test that valid choices extracts content correctly.
test_choices_with_empty_message - Test that choices with empty message dict doesn't crash.
class TestSilentExceptionSwallowingFixes - Tests for silent exception swallowing fixes in providers.py.
test_url_parsing_exception_is_logged - Test that URL parsing exceptions are now logged instead of silently ignored.
test_valid_url_parsing - Test that valid URLs are parsed correctly.
class TestEndToEndErrorHandling - Integration tests for combined error handling improvements.
test_messages_endpoint_resilience - Test that messages endpoint handles edge cases without crashing.
test_chat_endpoint_resilience - Test that chat endpoint handles edge cases without crashing.
References: src/services/background_tasks.py
Module: Tests for background tasks service
class TestBackgroundTasks - Test background task functionality
test_log_activity_async_success - log_activity_async should execute database logging
test_log_activity_async_error_handling - log_activity_async should handle errors gracefully
test_log_activity_background_no_event_loop - log_activity_background should work without event loop
test_log_activity_background_with_event_loop - log_activity_background should create task when event loop exists
test_log_activity_background_error_handling - log_activity_background should handle errors gracefully
test_get_pending_tasks_count - get_pending_tasks_count should return count of pending tasks
test_log_activity_background_with_all_parameters - log_activity_background should handle all parameters
test_log_activity_async_with_all_parameters - log_activity_async should handle all parameters
test_log_activity_background_with_minimal_parameters - log_activity_background should work with minimal parameters
test_log_activity_async_with_none_metadata - log_activity_async should handle None metadata
test_log_activity_background_multiple_calls - log_activity_background should handle multiple calls
References: src/services.py, src/services/braintrust_service.py
Module: Unit tests for the Braintrust service module.
class TestBraintrustService - Tests for the braintrust_service module.
test_initialize_without_api_key - Test initialization fails gracefully without API key.
test_initialize_with_invalid_api_key - Test initialization warns with invalid API key format.
test_is_available_before_init - Test is_available returns False before initialization.
test_create_span_returns_noop_when_unavailable - Test create_span returns NoopSpan when Braintrust is unavailable.
test_noop_span_log_method - Test NoopSpan.log() doesn't raise exceptions.
test_noop_span_end_method - Test NoopSpan.end() doesn't raise exceptions.
test_noop_span_context_manager - Test NoopSpan works as a context manager.
test_noop_span_set_attributes - Test NoopSpan.set_attributes() doesn't raise exceptions.
test_flush_when_not_initialized - Test flush() doesn't raise when not initialized.
test_get_logger_when_not_initialized - Test get_logger() returns None when not initialized.
test_check_braintrust_available_alias - Test check_braintrust_available is alias for is_available.
class TestBraintrustServiceWithMocking - Tests that mock the Braintrust SDK.
test_initialize_calls_init_logger_with_correct_params - Test that initialize calls init_logger with correct parameters.
test_create_span_uses_logger_start_span - Test that create_span uses logger.start_span() for proper project association.
test_flush_calls_logger_flush - Test that flush() calls logger.flush().
test_create_span_returns_noop_on_exception - Test that create_span returns NoopSpan if an exception occurs.
class TestBraintrustRealIntegration - Real integration tests that require a valid API key.
test_real_initialization - Test real Braintrust initialization.
test_real_span_creation_and_logging - Test real span creation and logging.
References: src/config.py, src/services/bug_fix_generator.py, src/services/error_monitor.py
Module: Unit tests for the bug fix generator service.
class TestBugFixGeneratorInitialization - Test bug fix generator initialization and configuration.
test_init_without_api_key - Test that initialization fails without ANTHROPIC_API_KEY.
test_init_with_api_key - Test successful initialization with API key.
test_init_with_invalid_key_format - Test warning when API key doesn't start with sk-ant-.
test_init_with_github_token - Test initialization with GitHub token.
test_init_with_custom_model - Test initialization with custom ANTHROPIC_MODEL.
class TestAPIKeyValidation - Test API key validation logic.
test_validate_api_key_success - Test successful API key validation.
test_validate_api_key_401_unauthorized - Test validation fails with 401 Unauthorized.
test_validate_api_key_400_bad_request - Test validation fails with 400 Bad Request.
test_validate_api_key_timeout - Test validation handles timeout gracefully.
class TestPromptSanitization - Test prompt sanitization and length validation.
test_sanitize_text_short_text - Test sanitizing normal-length text.
test_sanitize_text_long_text - Test sanitizing text exceeding max length.
test_sanitize_text_with_null_bytes - Test removing null bytes from text.
test_sanitize_text_empty - Test sanitizing empty text.
test_sanitize_text_none - Test sanitizing None.
test_prepare_prompt_normal_length - Test preparing normal-length prompt.
test_prepare_prompt_exceeds_max - Test preparing prompt that exceeds max length.
class TestErrorAnalysis - Test error analysis with mocked Claude API.
test_analyze_error_success - Test successful error analysis.
test_analyze_error_no_content - Test error analysis with no content in response.
test_analyze_error_400_bad_request - Test error analysis with 400 error.
test_analyze_error_sanitizes_long_messages - Test that error analysis sanitizes long error messages.
class TestFixGeneration - Test fix generation with mocked Claude API.
test_generate_fix_success - Test successful fix generation.
test_generate_fix_analysis_fails - Test fix generation skips when analysis fails.
test_generate_fix_invalid_json - Test fix generation handles invalid JSON response.
test_generate_fix_no_changes - Test fix generation with empty changes array.
class TestRetryLogic - Test retry logic with exponential backoff.
test_retry_on_timeout - Test that requests are retried on timeout.
test_retry_max_attempts - Test that retry stops after max attempts.
test_no_retry_on_400 - Test that 400 errors are not retried.
class TestRequestLogging - Test request/response logging with correlation IDs.
test_request_logging_success - Test that successful requests are logged.
test_request_logging_error - Test that errors are logged with details.
class TestBugFixDataClass - Test BugFix data class functionality.
test_bug_fix_to_dict - Test BugFix.to_dict() serialization.
class TestEdgeCases - Test edge cases and error scenarios.
test_close_generator - Test closing the generator properly.
test_multiple_fixes_tracking - Test that generated fixes are tracked.
References: src/services/butter_client.py
Module: Tests for Butter.dev LLM Response Caching Client
class TestButterCompatibleProviders - Test provider compatibility checks.
test_compatible_providers_return_true - Test that known compatible providers are identified correctly.
test_excluded_providers_return_false - Test that excluded providers are identified correctly.
test_unknown_provider_returns_false - Test that unknown providers default to not compatible.
test_case_insensitive_provider_matching - Test that provider matching is case-insensitive.
class TestShouldUseButterCache - Test the main eligibility check function.
test_system_disabled_returns_false - Test that disabled system returns False.
test_anonymous_user_returns_false - Test that anonymous users don't use caching.
test_user_preference_disabled_returns_false - Test that users with caching disabled return False.
test_incompatible_provider_returns_false - Test that incompatible providers return False.
test_all_conditions_met_returns_true - Test that all conditions met returns True.
class TestCacheHitDetection - Test cache hit detection heuristics.
test_fast_response_is_cache_hit - Test that fast responses are detected as cache hits.
test_slow_response_is_cache_miss - Test that slow responses are detected as cache misses.
test_custom_threshold - Test that custom threshold works.
class TestButterCacheTimer - Test the cache timer context manager.
test_timer_measures_elapsed_time - Test that timer correctly measures elapsed time.
test_timer_detects_cache_hit - Test that timer correctly detects cache hit based on latency.
test_timer_detects_cache_miss - Test that timer correctly detects cache miss based on latency.
test_timer_custom_threshold - Test that timer respects custom threshold.
class TestButterRequestMetadata - Test metadata generation for request tracking.
test_cache_hit_metadata - Test metadata generation for cache hit.
test_cache_miss_metadata - Test metadata generation for cache miss.
class TestGetUserCachePreference - Test user preference extraction.
test_returns_true_when_enabled - Test that True is returned when caching is enabled.
test_returns_false_when_disabled - Test that False is returned when caching is disabled.
test_returns_true_for_missing_preference - Test that True is returned when preference is not set (enabled by default).
test_returns_false_for_none_user - Test that False is returned for None user.
References: src/services.py, src/services/canopywave_client.py
Module: Comprehensive tests for Canopy Wave Client service
class TestCanopywaveClient - Test Canopy Wave Client service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
test_get_canopywave_client_raises_without_api_key - Test that get_canopywave_client raises ValueError when API key is not configured
test_get_canopywave_client_returns_pooled_client - Test that get_canopywave_client returns a pooled client when API key is configured
test_fetch_models_from_canopywave_success - Test successful model fetching from Canopy Wave API
test_fetch_models_from_canopywave_handles_embedding_models - Test that embedding models are properly handled
test_fetch_models_from_canopywave_http_error - Test that HTTP errors are handled gracefully
test_fetch_models_from_canopywave_network_error - Test that network errors are handled gracefully
test_fetch_models_from_canopywave_skips_models_without_id - Test that models without an ID are skipped
test_fetch_models_from_canopywave_handles_missing_pricing - Test that missing pricing information is handled with defaults
test_fetch_models_from_canopywave_handles_context_in_capabilities - Test that context_length in capabilities is properly extracted
test_make_canopywave_request_openai - Test making a request to Canopy Wave using OpenAI client
test_make_canopywave_request_openai_stream - Test making a streaming request to Canopy Wave
test_process_canopywave_response - Test processing a Canopy Wave response
test_is_canopywave_model_returns_true_for_valid_model - Test that is_canopywave_model returns True for a valid model
test_is_canopywave_model_returns_false_for_invalid_model - Test that is_canopywave_model returns False for an invalid model
test_is_canopywave_model_handles_errors - Test that is_canopywave_model handles errors gracefully
test_fetch_models_handles_function_calling_capability - Test that function_calling capability is mapped to tools feature
test_fetch_models_handles_web_search_capability - Test that web_search capability is properly extracted
References: src/services.py, src/services/cerebras_client.py
Module: Comprehensive tests for Cerebras Client service
class TestCerebrasClient - Test Cerebras Client service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
test_fetch_models_from_cerebras_normalizes_sdk_response - Ensure fetch_models_from_cerebras returns normalized entries when SDK responds.
test_fetch_models_from_cerebras_uses_fallback_on_error - Ensure fallback catalog is used when client initialization fails.
References: src/handlers/chat_handler.py, src/services/circuit_breaker.py, src/services/openrouter_client.py, src/utils/error_factory.py
Module: Test Circuit Breaker Improvements - Issue #1089
class TestCircuitBreakerHalfOpenBehavior - Test improvements to HALF_OPEN state recovery
test_half_open_allows_multiple_failures_before_reopening - Test that HALF_OPEN state allows 2 failures before reopening
test_half_open_closes_after_success - Test that HALF_OPEN transitions to CLOSED after success_threshold successes
class TestConsecutiveOpensTracking - Test tracking of consecutive circuit opens
test_consecutive_opens_increments_on_failed_recovery - Test that consecutive_opens increments when recovery fails (circuit reopens from HALF_OPEN)
test_consecutive_opens_resets_on_successful_recovery - Test that consecutive_opens resets to 0 after successful recovery
class TestCircuitBreakerErrorHandling - Test CircuitBreakerError exception and error responses
test_circuit_breaker_error_raised_when_open - Test that CircuitBreakerError is raised when circuit is OPEN
test_circuit_breaker_error_contains_retry_info - Test that CircuitBreakerError message contains retry information
class TestDetailedErrorFactory - Test the new provider_unavailable error factory method
test_provider_unavailable_creates_503_error - Test that provider_unavailable returns 503 status
test_provider_unavailable_contains_helpful_info - Test that error contains helpful information for users
class TestOpenRouterCircuitConfig - Test the optimized OpenRouter circuit breaker configuration
test_openrouter_config_has_optimized_values - Test that OPENROUTER_CIRCUIT_CONFIG has the optimized parameters
class TestRedisStatePersistence - Test Redis persistence for distributed deployments
test_consecutive_opens_persisted_to_redis - Test that consecutive_opens is saved to Redis
test_consecutive_opens_loaded_from_redis - Test that consecutive_opens is loaded from Redis
class TestCircuitBreakerIntegration - Integration tests for circuit breaker with chat handler
test_chat_handler_catches_circuit_breaker_error - Test that chat handler catches CircuitBreakerError and returns 503
References: src/services/cloudflare_workers_ai_client.py
Module: Tests for Cloudflare Workers AI client
class TestCloudflareWorkersAIClient - Test Cloudflare Workers AI client functionality
test_get_cloudflare_workers_ai_client - Test getting Cloudflare Workers AI client
test_get_cloudflare_workers_ai_client_no_token - Test getting Cloudflare Workers AI client without API token
test_get_cloudflare_workers_ai_client_no_account_id - Test getting Cloudflare Workers AI client without Account ID
test_make_cloudflare_workers_ai_request_openai - Test making request to Cloudflare Workers AI
test_make_cloudflare_workers_ai_request_openai_with_kwargs - Test making request to Cloudflare Workers AI with additional parameters
test_make_cloudflare_workers_ai_request_openai_error - Test handling errors from Cloudflare Workers AI
test_make_cloudflare_workers_ai_request_openai_stream - Test making streaming request to Cloudflare Workers AI
test_make_cloudflare_workers_ai_request_openai_stream_with_kwargs - Test making streaming request to Cloudflare Workers AI with additional parameters
test_make_cloudflare_workers_ai_request_openai_stream_error - Test handling streaming errors from Cloudflare Workers AI
test_process_cloudflare_workers_ai_response - Test processing Cloudflare Workers AI response
test_process_cloudflare_workers_ai_response_no_usage - Test processing Cloudflare Workers AI response without usage data
test_process_cloudflare_workers_ai_response_multiple_choices - Test processing Cloudflare Workers AI response with multiple choices
test_fetch_models_from_cloudflare_workers_ai - Test fetching models from Cloudflare Workers AI
test_default_cloudflare_workers_ai_models - Test the default Cloudflare Workers AI models catalog
test_default_cloudflare_workers_ai_models_have_source_gateway - Test that all default Cloudflare Workers AI models have source_gateway field
class TestFetchModelsFromCloudflareAPI - Test fetching models from Cloudflare API with various response formats
test_fetch_models_with_properties_as_list - Test parsing models when properties is a list of property objects
test_fetch_models_with_properties_as_dict - Test parsing models when properties is a dict (legacy format)
test_fetch_models_with_missing_properties - Test parsing models when properties field is missing
test_fetch_models_with_task_as_string - Test parsing models when task is a string instead of a dict
test_fetch_models_with_invalid_context_length_value - Test parsing models when max_total_tokens has an invalid value
test_fetch_models_without_api_token - Test that API returns empty list when token is missing
test_fetch_models_without_account_id - Test that API returns empty list when account ID is missing
test_fetch_models_with_null_task - Test parsing models when task is null
test_fetch_models_with_dict_properties_string_value - Test parsing models when properties dict has string value for max_total_tokens
test_fetch_models_with_dict_properties_invalid_value - Test parsing models when properties dict has invalid value for max_total_tokens
test_fetch_models_with_non_dict_items_in_result - Test parsing models when result array contains non-dict items (lists, strings, etc.)
References: src/services/code_classifier.py
Module: Tests for Code Task Classifier
class TestCodeTaskClassifier - Test suite for CodeTaskClassifier.
test_classify_simple_code - Test classification of simple code tasks.
test_classify_code_explanation - Test classification of code explanation tasks.
test_classify_code_generation - Test classification of code generation tasks.
test_classify_debugging - Test classification of debugging tasks.
test_classify_refactoring - Test classification of refactoring tasks.
test_classify_architecture - Test classification of architecture tasks.
test_classify_agentic - Test classification of agentic coding tasks.
test_high_confidence_classification - Test that clear prompts get high confidence.
test_low_confidence_classification - Test that ambiguous prompts get lower confidence.
test_default_category_for_ambiguous - Test that ambiguous prompts default to code_generation.
test_classification_time_under_threshold - Test that classification completes within target time.
test_context_multi_file - Test that multi-file context boosts architecture/agentic scores.
test_context_error_trace - Test that error trace context boosts debugging score.
test_context_long_conversation - Test that long conversation context boosts refactoring/architecture.
test_is_code_related_positive - Test that code-related prompts are detected.
test_is_code_related_negative - Test that non-code prompts are not detected as code-related.
test_extract_context_from_messages - Test context extraction from conversation messages.
test_extract_context_multi_part_messages - Test context extraction with multi-part messages.
class TestModuleFunctions - Test module-level convenience functions.
test_get_classifier_singleton - Test that get_classifier returns the same instance.
test_classify_code_task_function - Test the convenience classify_code_task function.
test_is_code_related_function - Test the convenience is_code_related function.
class TestEdgeCases - Test edge cases and error handling.
test_empty_prompt - Test classification of empty prompt.
test_very_long_prompt - Test classification of very long prompt.
test_unicode_prompt - Test classification with unicode characters.
test_none_context - Test classification with None context.
test_empty_context - Test classification with empty context.
References: src/routes/code_router.py, src/services/code_router.py
Module: Tests for Code-Optimized Prompt Router
class TestParseRouterModelString - Test router model string parsing.
test_parse_router_code_auto - Test parsing 'router:code' as auto mode.
test_parse_router_code_price - Test parsing 'router:code:price' mode.
test_parse_router_code_quality - Test parsing 'router:code:quality' mode.
test_parse_router_code_agentic - Test parsing 'router:code:agentic' mode.
test_parse_non_router_model - Test parsing regular model strings.
test_parse_router_auto - Test parsing general 'router:' prefix (not code router).
test_parse_case_insensitive - Test that parsing is case-insensitive.
test_parse_unknown_mode - Test parsing unknown mode falls back to auto.
class TestCodeRouter - Test suite for CodeRouter class.
test_route_simple_task - Test routing a simple code task.
test_route_debugging_task - Test routing a debugging task.
test_route_architecture_task - Test routing an architecture task.
test_route_agentic_task - Test routing an agentic task.
test_route_price_mode - Test routing in price optimization mode.
test_route_quality_mode - Test routing in quality optimization mode.
test_route_agentic_mode - Test routing in agentic mode.
test_route_auto_mode - Test routing in auto mode (default).
test_quality_gate_debugging - Test that debugging tasks respect quality gate.
test_quality_gate_architecture - Test that architecture tasks respect quality gate.
test_quality_gate_agentic_task - Test that agentic tasks respect quality gate.
test_routing_result_structure - Test that routing result has expected structure.
test_routing_latency - Test that routing completes within target time.
test_savings_estimate_structure - Test savings estimate structure.
test_route_with_context - Test routing with context.
test_route_with_user_default_model - Test routing with user's default model for comparison.
class TestModuleFunctions - Test module-level convenience functions.
test_get_router_singleton - Test that get_router returns the same instance.
test_route_code_prompt_function - Test the convenience route_code_prompt function.
test_get_routing_metadata - Test get_routing_metadata formats correctly.
test_get_model_tiers - Test get_model_tiers returns valid data.
test_get_fallback_model - Test get_fallback_model returns valid model.
test_get_baselines - Test get_baselines returns baseline configs.
class TestEdgeCases - Test edge cases and error handling.
test_empty_prompt - Test routing with empty prompt.
test_very_long_prompt - Test routing with very long prompt.
test_unicode_prompt - Test routing with unicode characters.
test_none_context - Test routing with None context.
test_invalid_mode_defaults_to_auto - Test that invalid mode is handled gracefully.
class TestTierSelection - Test tier selection logic in detail.
test_tier_1_for_architecture - Test that architecture always gets tier 1.
test_tier_1_for_agentic_mode - Test that agentic mode always selects tier 1.
test_tier_respected_in_price_mode - Test that quality gates are respected even in price mode.
test_quality_mode_bumps_tier - Test that quality mode bumps up the tier.
References: src/services.py, src/services/connection_pool.py
Module: Comprehensive tests for Connection Pool service
class TestConnectionPool - Test Connection Pool service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
test_reuses_client_when_api_key_unchanged - The same provider/base/api_key triple should reuse the cached client.
test_rebuilds_client_when_api_key_changes - Rotating API keys should evict the old client and close it.
test_get_groq_pooled_client - Test that Groq pooled client is configured correctly.
test_get_groq_pooled_client_no_key - Test that Groq pooled client raises error without API key.
References: src/db/rate_limits.py, src/db/trials.py, src/db/users.py, src/services/credit_handler.py, src/services/pricing.py, src/services/prometheus_metrics.py, src/utils/sentry_context.py
Module: Tests for the unified credit handler service.
class TestRecordCreditMetrics - Test Prometheus metrics recording for credit deductions.
test_records_success_metrics - Test that success metrics are recorded correctly.
test_records_retry_metrics - Test that retry metrics are recorded for attempt > 1.
class TestRecordMissedDeduction - Test metrics recording for missed credit deductions.
test_records_missed_deduction - Test that missed deduction is recorded.
test_skips_zero_cost - Test that zero cost doesn't record metric.
class TestRecordBackgroundTaskFailure - Test metrics recording for background task failures.
test_records_failure - Test that background task failure is recorded.
class TestSendCriticalBillingAlert - Test Sentry alerting for billing failures.
test_sends_alert_for_significant_cost - Test that Sentry alert is sent for costs >= $0.01.
test_skips_message_for_small_cost - Test that Sentry breadcrumb and alert are sent for all costs.
class TestHandleCreditsAndUsage - Test the main credit handling function.
test_trial_user_no_deduction - Test that trial users don't get credits deducted.
test_paid_user_deduction_success - Test that paid users get credits deducted successfully.
test_retry_on_transient_failure - Test that credit deduction retries on transient failure.
test_no_retry_on_validation_error - Test that ValueError (insufficient credits) doesn't retry.
test_all_retries_exhausted - Test behavior when all retries are exhausted.
test_trial_override_for_paid_user_with_stale_flag - Test that paid users with stale is_trial flag still get charged.
test_no_duplicate_deduction_when_usage_logging_fails - Test that deduct_credits is only called once even if record_usage fails.
test_no_duplicate_deduction_when_rate_limit_update_fails - Test that deduct_credits is only called once even if rate limit update fails.
class TestHandleCreditsAndUsageWithFallback - Test the fallback wrapper for streaming requests.
test_returns_success_on_normal_operation - Test successful operation returns (cost, True).
test_returns_failure_and_logs_on_error - Test that failures return (cost, False) and log for reconciliation.
test_uses_fallback_cost_if_pricing_fails - Test that fallback cost estimation is used if pricing lookup fails.
class TestRetryConfiguration - Test that retry configuration is correct.
test_max_retries_is_reasonable - Test that max retries is between 1 and 5.
test_retry_delays_are_exponential - Test that retry delays increase.
test_retry_delays_match_max_retries - Test that we have enough delay values for all retries.
References: src/services/credit_precheck.py
Module: Tests for credit pre-flight check service.
class TestGetModelMaxTokens - Test getting maximum output tokens for different models.
test_exact_match_gpt4 - Test exact match for GPT-4.
test_exact_match_claude - Test exact match for Claude models.
test_partial_match_gpt4o - Test partial match for versioned GPT-4o.
test_unknown_model_defaults - Test unknown model returns default.
test_llama_models - Test Llama model detection.
class TestCalculateMaximumCost - Test maximum cost calculation.
test_with_explicit_max_tokens - Test cost calculation with explicit max_tokens.
test_without_max_tokens_uses_model_default - Test cost calculation without max_tokens uses model default.
test_zero_max_tokens_uses_model_default - Test cost calculation with zero max_tokens uses model default.
class TestCheckCreditSufficiency - Test credit sufficiency checking.
test_sufficient_credits - Test when user has sufficient credits.
test_insufficient_credits - Test when user has insufficient credits.
test_trial_user_always_allowed - Test that trial users are always allowed (no credit check).
test_exact_credits_match - Test when user has exactly enough credits.
class TestEstimateAndCheckCredits - Test the main entry point for credit pre-checks.
test_complete_flow_with_sufficient_credits - Test complete pre-check flow with sufficient credits.
test_complete_flow_with_insufficient_credits - Test complete pre-check flow with insufficient credits.
test_trial_user_bypasses_check - Test that trial users bypass credit checks entirely.
test_uses_model_default_when_no_max_tokens - Test that model defaults are used when max_tokens not specified.
test_expensive_model_high_max_tokens - Test expensive scenario: GPT-4 with high max_tokens.
class TestIntegrationScenarios - Test real-world integration scenarios.
test_scenario_user_starts_expensive_request - Scenario: User with $0.01 tries to start a GPT-4 request
test_scenario_user_reduces_max_tokens - Scenario: User sees error, reduces max_tokens to lower cost.
test_scenario_streaming_vs_non_streaming - Scenario: Same credit check for streaming and non-streaming.
References: src/config/usage_limits.py, src/services/daily_usage_limiter.py
Module: Tests for Daily Usage Limiter Service
class TestDailyResetTime - Test daily reset time calculation
test_get_daily_reset_time_future - Test that reset time is always in the future
test_get_daily_reset_time_tomorrow_if_past_midnight - Test that reset time is tomorrow if we're past midnight
class TestDailyUsageTracking - Test daily usage tracking
test_get_daily_usage_no_transactions - Test get_daily_usage when user has no transactions
test_get_daily_usage_with_transactions - Test get_daily_usage with multiple transactions
test_get_daily_usage_error_handling - Test that errors are handled gracefully
class TestDailyUsageLimitCheck - Test daily usage limit checking
test_check_daily_usage_limit_within_limit - Test check when usage is within limit
test_check_daily_usage_limit_would_exceed - Test check when request would exceed limit
test_check_daily_usage_limit_warning_threshold - Test warning level at 80% usage
test_check_daily_usage_limit_critical_threshold - Test warning level at 95% usage
test_check_daily_usage_limit_exceeded - Test when limit is already exceeded
class TestEnforceDailyUsageLimit - Test daily usage limit enforcement
test_enforce_daily_usage_limit_allowed - Test that enforcement passes when within limit
test_enforce_daily_usage_limit_exceeded_raises_exception - Test that enforcement raises exception when limit exceeded
class TestDailyUsageLimitIntegration - Integration tests for daily usage limiter
test_full_workflow_within_limit - Test full workflow when staying within limit
test_full_workflow_exceeds_limit - Test full workflow when exceeding limit
class TestDisabledLimits - Test behavior when limits are disabled
test_check_with_limits_disabled - Test that checks pass when limits are disabled
References: src/db/models_catalog_db.py, src/services/model_catalog_cache.py, src/services/model_catalog_sync.py, src/services/models.py
Module: Tests for DB-first catalog architecture
test_get_cached_full_catalog_returns_cache_when_available - Test cache hit returns cached data without DB query
test_get_cached_full_catalog_fetches_from_db_on_miss - Test cache miss fetches from database
test_get_cached_provider_catalog_fetches_from_db_on_miss - Test provider catalog cache miss fetches from database
test_get_cached_models_all_gateway - Test get_cached_models with 'all' gateway
test_get_cached_models_single_provider - Test get_cached_models with single provider
test_get_cached_models_handles_errors - Test get_cached_models handles errors gracefully
test_cache_invalidation_exists - Verify that cache invalidation logic exists.
test_full_flow_db_first - Integration test: Full flow with DB-first architecture.
test_get_cached_models_empty_gateway - Test handling of empty gateway parameter
test_get_cached_models_none_gateway - Test handling of None gateway parameter
References: src/services/models.py
Module: Tests for DeepInfra model normalization
class TestDeepInfraNormalization - Test DeepInfra model normalization
test_normalize_text_to_image_model - Test normalizing an image generation model
test_normalize_text_generation_model - Test normalizing a text generation model
test_normalize_model_without_type - Test normalizing a model without explicit type (defaults to text->text)
test_normalize_model_with_id_field - Test normalizing a model that uses 'id' instead of 'model_name'
test_normalize_model_without_pricing - Test normalizing a model without pricing information
test_normalize_deprecated_model_without_replacement - Test normalizing a deprecated model without a replacement
test_normalize_model_with_multimodal_type - Test normalizing a multimodal model
test_normalize_tts_model - Test normalizing a text-to-speech model
test_normalize_stt_model - Test normalizing a speech-to-text model
References: src/services/email_verification.py
Module: Tests for Email Verification Service using Emailable API.
class TestEmailVerificationResult - Tests for EmailVerificationResult dataclass.
test_disposable_email_is_bot - Disposable emails should be marked as bot.
test_risky_low_score_is_bot - Risky emails with low score should be marked as bot.
test_risky_high_score_not_bot - Risky emails with high score should not be marked as bot.
test_deliverable_email_not_bot - Deliverable emails should not be marked as bot.
test_unknown_email_not_bot - Unknown emails should not be marked as bot (benefit of doubt).
test_undeliverable_should_block - Undeliverable emails should be blocked.
test_invalid_domain_should_block - Invalid domain emails should be blocked.
test_deliverable_should_not_block - Deliverable emails should not be blocked.
class TestEmailVerificationService - Tests for EmailVerificationService.
test_service_disabled_by_default - Service should be disabled by default.
test_service_enabled_with_api_key - Service should be enabled when API key and enabled flag are set.
test_service_disabled_without_api_key - Service should be disabled if enabled but no API key.
test_verify_returns_unknown_when_disabled - When disabled, verify should return unknown result.
test_verify_parses_deliverable_response - Service should correctly parse deliverable response.
test_verify_parses_disposable_response - Service should correctly identify disposable emails.
test_verify_handles_rate_limit - Service should handle rate limit gracefully.
test_verify_handles_insufficient_credits - Service should handle insufficient credits gracefully.
test_verify_handles_timeout - Service should handle timeout gracefully.
class TestEmailVerificationEdgeCases - Tests for edge cases and special scenarios.
test_empty_email - Service should handle empty email gracefully.
test_email_without_at_symbol - Service should handle malformed email gracefully.
test_email_case_normalization - Service should normalize email to lowercase.
test_email_whitespace_trimming - Service should trim whitespace from email.
test_did_you_mean_suggestion - Result should include did_you_mean suggestion if provided.
class TestEmailVerificationCaching - Tests for email verification caching functionality.
test_result_to_cache_dict - Result should correctly convert to cache dict.
test_result_from_cache_dict - Result should correctly restore from cache dict.
test_cache_roundtrip - Result should survive roundtrip through cache dict.
test_verify_caches_successful_result - Successful verification should be cached.
test_verify_returns_cached_result - Verification should return cached result if available.
test_verify_does_not_cache_api_errors - API errors should not be cached.
test_verify_does_not_cache_timeouts - Timeouts should not be cached.
test_verify_works_when_redis_unavailable - Verification should work when Redis is unavailable.
References: src/enhanced_notification_service.py
Module: Tests for EnhancedNotificationService email validation.
class TestEmailValidation - Test email validation logic for EnhancedNotificationService
test_valid_email_passes_validation - Test that valid emails pass validation
test_privy_fallback_email_rejected - Test that Privy fallback emails are rejected
test_empty_email_rejected - Test that empty/None emails are rejected
test_invalid_email_format_rejected - Test that invalid email formats are rejected
test_send_welcome_email_skips_privy_fallback - Test that welcome email is skipped for Privy fallback addresses
test_send_email_notification_skips_privy_fallback - Test that email notification is skipped for Privy fallback addresses
test_send_email_notification_sends_valid_email - Test that valid emails are sent normally
class TestEdgeCases - Test edge cases for email validation
test_email_ending_with_privy_user_substring - Test emails that contain but don't end with @privy.user
test_similar_but_valid_domain - Test emails with similar-looking but valid domains
test_password_reset_validates_email_before_token_creation - Test that password reset validates email BEFORE creating token
References: src/services/error_monitor.py
Module: Tests for error monitor Loki logging improvements.
class TestLokiErrorLogging - Test Loki error logging behavior.
test_loki_empty_error_not_logged - Test that empty Loki errors are suppressed and not logged at ERROR level.
test_loki_none_error_not_logged - Test that 'None' string errors from Loki are suppressed.
test_loki_real_error_still_logged - Test that real Loki errors are still logged at ERROR level.
test_loki_successful_fetch - Test that successful Loki fetches work correctly.
test_loki_disabled_no_error - Test that disabled Loki doesn't trigger errors.
test_loki_whitespace_error_suppressed - Test that whitespace-only errors are suppressed.
References: src/services/fal_image_client.py
class TestFalModelsCatalog - Tests for Fal.ai models catalog functionality
test_load_fal_models_catalog - Test loading Fal.ai models catalog
test_get_fal_models - Test getting all Fal.ai models
test_get_fal_models_by_type_text_to_image - Test filtering models by text-to-image type
test_get_fal_models_by_type_image_to_video - Test filtering models by image-to-video type
test_get_fal_models_by_type_text_to_video - Test filtering models by text-to-video type
test_get_fal_models_by_type_empty - Test filtering with non-existent type returns empty list
test_validate_fal_model_valid - Test validating existing model IDs
test_validate_fal_model_invalid - Test validating non-existent model IDs
test_model_types_coverage - Test that catalog covers all expected model types
test_catalog_caching - Test that catalog is cached after first load
test_model_structure - Test that all models have consistent structure
References: src/services/featherless_client.py
Module: Unit tests for Featherless client
class TestGetFeatherlessClient - Test get_featherless_client function
test_get_featherless_client_success - Test successful client initialization
test_get_featherless_client_missing_key - Test client initialization with missing API key
class TestMakeFeatherlessRequest - Test make_featherless_request_openai function
test_make_featherless_request_openai_forwards_args - Test that request forwards all arguments correctly
test_make_featherless_request_openai_error - Test request error handling
class TestMakeFeatherlessRequestStream - Test make_featherless_request_openai_stream function
test_make_featherless_request_openai_stream - Test streaming request
test_make_featherless_request_openai_stream_error - Test streaming request error handling
class TestProcessFeatherlessResponse - Test process_featherless_response function
test_process_featherless_response_happy - Test processing a normal response
test_process_featherless_response_no_usage - Test processing response without usage data
test_process_featherless_response_error - Test processing error handling
class TestSanitizeMessagesForFeatherless - Test _sanitize_messages_for_featherless function
test_removes_null_tool_calls_from_dict - Test that null tool_calls are removed from dict messages
test_preserves_valid_tool_calls - Test that valid tool_calls arrays are preserved
test_preserves_messages_without_tool_calls - Test that messages without tool_calls field are unchanged
test_does_not_mutate_original_messages - Test that original messages are not modified
test_removes_invalid_tool_calls_type - Test that non-array tool_calls are removed
test_handles_empty_list - Test that empty message list is handled
References: src/services/fireworks_client.py
Module: Tests for Fireworks client
class TestFireworksClient - Test Fireworks client functionality
test_get_fireworks_client - Test getting Fireworks client
test_get_fireworks_client_no_key - Test getting Fireworks client without API key
test_make_fireworks_request_openai - Test making request to Fireworks
test_make_fireworks_request_openai_stream - Test making streaming request to Fireworks
test_process_fireworks_response - Test processing Fireworks response
References: src/routes/general_router.py, src/schemas/general_router.py, src/services/general_router.py
Module: Tests for general router service.
class TestNormalizeModelString - Test model string normalization.
test_normalize_general_router_hyphenated - Test normalizing gatewayz-general aliases.
test_normalize_code_router_hyphenated - Test normalizing gatewayz-code aliases.
test_normalize_already_normalized - Test that already normalized strings pass through unchanged.
test_normalize_regular_model - Test that regular model IDs pass through unchanged.
test_normalize_case_insensitive - Test case insensitivity.
class TestParseRouterModelString - Test router model string parsing.
test_parse_router_general_balanced - Test parsing router:general (balanced mode).
test_parse_router_general_quality - Test parsing router:general:quality.
test_parse_router_general_cost - Test parsing router:general:cost.
test_parse_router_general_latency - Test parsing router:general:latency.
test_parse_invalid_mode - Test parsing invalid mode falls back to balanced.
test_parse_non_router_string - Test parsing non-router strings.
test_parse_code_router_string - Test that code router strings are not matched.
test_parse_case_insensitive - Test case insensitivity.
class TestGeneralRouterRoute - Test GeneralRouter.route method.
test_route_with_notdiamond_enabled - Test routing when NotDiamond is enabled and working.
test_route_with_notdiamond_disabled - Test routing when NotDiamond is disabled.
test_route_with_unavailable_model - Test routing when NotDiamond recommends unavailable model.
test_route_with_notdiamond_exception - Test routing when NotDiamond throws exception.
class TestRoutingMetadata - Test routing metadata formatting.
test_get_routing_metadata_success - Test metadata formatting for successful routing.
test_get_routing_metadata_fallback - Test metadata formatting for fallback routing.
test_route_general_prompt_convenience_function - Test the convenience function route_general_prompt.
References: src/services.py, src/services/google_models_config.py
Module: Comprehensive tests for Google Models Config service
class TestGoogleModelsConfig - Test Google Models Config service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/services/google_oauth2_jwt.py
Module: Tests for Google OAuth2 JWT exchange service
test_base64url_encode_simple - Test base64url encoding of simple strings
test_base64url_encode_json - Test base64url encoding of JSON
test_base64url_encode_decode_roundtrip - Test that base64url encoding can be decoded back
test_build_jwt_assertion_structure - Test JWT assertion has correct structure
test_build_jwt_assertion_with_custom_subject - Test JWT assertion with custom subject claim
test_build_jwt_assertion_expiry - Test JWT assertion expiry calculation
test_build_jwt_assertion_invalid_key - Test JWT assertion building with invalid private key
test_exchange_jwt_for_access_token_success - Test successful JWT exchange for access token
test_exchange_jwt_for_access_token_401_error - Test JWT exchange with 401 unauthorized error
test_exchange_jwt_for_access_token_500_error - Test JWT exchange with 500 server error
test_exchange_jwt_for_access_token_network_error - Test JWT exchange with network error
test_get_access_token_from_service_account_success - Test getting access token from service account JSON
test_get_access_token_from_service_account_invalid_json - Test with invalid service account JSON
test_get_access_token_from_service_account_missing_email - Test with service account JSON missing client_email
test_get_access_token_from_service_account_missing_key - Test with service account JSON missing private_key
test_get_access_token_from_service_account_custom_scope - Test with custom OAuth2 scope
test_full_oauth2_jwt_flow_integration - Test the complete OAuth2 JWT flow end-to-end
References: src/services/google_oauth2_jwt.py
Module: Test Google OAuth2 JWT id_token fallback functionality
class TestIdTokenFallback - Test cases for id_token fallback in OAuth2 exchange
test_exchange_jwt_returns_access_token_when_present - Test normal flow: access_token is present in response
test_exchange_jwt_falls_back_to_id_token - Test fallback: id_token is used when access_token is missing
test_exchange_jwt_prefers_access_token_over_id_token - Test that access_token is preferred when both are present
test_exchange_jwt_raises_when_no_token_present - Test that error is raised when neither token is present
test_exchange_jwt_handles_http_error - Test that HTTP errors are properly handled
test_exchange_jwt_sends_correct_request - Test that the correct request is sent to Google's token endpoint
test_id_token_with_real_google_response_structure - Test with a realistic Google OAuth2 response containing id_token
References: src/cache.py, src/config.py, src/services/google_vertex_client.py, src/services/model_transformations.py
Module: Tests for Google Vertex AI client
class TestTransformGoogleVertexModelId - Tests for model ID transformation
test_transform_simple_model_id - Test transforming a simple model ID returns the model name
test_transform_full_resource_name - Test that full resource names are extracted to simple model name
test_transform_google_prefix - Test that google/ prefix is stripped from model IDs
test_transform_various_models - Test transforming various model IDs
class TestBuildVertexContent - Tests for content building
test_build_simple_text_content - Test building content from simple text messages
test_build_multimodal_content - Test building multimodal content with images
test_build_base64_image_content_strips_data_url_prefix - Test that base64 image data URLs are properly parsed to extract raw base64 data.
test_build_base64_image_content_jpeg - Test that JPEG base64 images are correctly parsed with proper MIME type.
test_build_base64_image_content_webp - Test that WebP base64 images are correctly parsed with proper MIME type.
test_build_system_message - Test that system messages are mapped to model role
class TestProcessGoogleVertexResponse - Tests for response processing
test_process_successful_rest_response - Test processing a successful REST API response
test_process_multiple_content_parts - Test processing response with multiple content parts
test_process_gemini_flash_lite_response - Test processing response from gemini-2.5-flash-lite-preview-09-2025
class TestMakeGoogleVertexRequest - Tests for making requests to Google Vertex
test_make_request_with_parameters - Test making a request with various parameters
test_make_streaming_request - Test making a streaming request
test_make_request_gemini_flash_lite - Test making a request to gemini-2.5-flash-lite (maps to preview version)
class TestGoogleVertexRestTransport - Tests for the REST fallback transport.
test_rest_request_success - Ensure REST transport returns normalized response.
test_rest_request_http_error - REST transport should raise ValueError on HTTP error.
class TestGoogleVertexModelIntegration - Integration tests for model detection and transformation
test_gemini_model_detection - Test that gemini models are properly detected when credentials are available
test_model_id_transformation_consistency - Test that model IDs are transformed consistently
class TestFetchModelsFromGoogleVertex - Tests for fetch_models_from_google_vertex static model catalog
test_fetch_models_returns_list - Test that fetch_models_from_google_vertex returns a list of models
test_fetch_models_includes_gemini_models - Test that returned models include known Gemini models
test_fetch_models_normalized_format - Test that returned models have the expected normalized format
test_fetch_models_updates_cache - Test that fetch_models_from_google_vertex updates the cache
test_fetch_models_includes_provider_model_id - Test that returned models include provider_model_id for database sync.
test_fetch_models_raw_google_vertex_includes_provider_model_id - Test that raw_google_vertex metadata includes provider_model_id.
class TestModelLocationRouting - Tests for region-specific model routing
test_gemini_3_uses_global_endpoint - Test that Gemini 3 models use the global endpoint
test_gemini_2_uses_regional_endpoint - Test that Gemini 2.x models use the configured regional endpoint
test_gemini_1_uses_regional_endpoint - Test that Gemini 1.x models use the configured regional endpoint
test_gemma_models_use_regional_endpoint - Test that Gemma models use the configured regional endpoint
test_rest_request_uses_global_for_gemini_3 - Ensure REST transport constructs URL with global endpoint for Gemini 3
test_rest_request_uses_regional_for_gemini_2 - Ensure REST transport constructs URL with regional endpoint for Gemini 2
test_rest_request_strips_google_prefix_from_model_id - Ensure google/ prefix is stripped from model ID to prevent 404 errors.
test_sdk_request_uses_global_for_gemini_3 - Ensure SDK transport initializes with global location for Gemini 3
class TestMaxOutputTokensValidation - Tests for maxOutputTokens validation and clamping.
test_max_tokens_capped_at_65536 - Test that max_tokens > 65536 is clamped to 65536 for Vertex AI.
test_max_tokens_minimum_is_16 - Test that max_tokens < 16 is raised to 16 for Vertex AI.
test_valid_max_tokens_unchanged - Test that valid max_tokens values are not modified.
class TestNoCandidatesErrorHandling - Tests for handling 'no candidates' responses from Vertex AI
test_no_candidates_without_prompt_feedback - Test that 'no candidates' error includes model version when no promptFeedback is present
test_no_candidates_with_block_reason - Test that 'no candidates' error includes block reason when promptFeedback is present
test_no_candidates_with_safety_ratings - Test that safety rating details are included in error when blocked
References: src/services/google_vertex_client.py
Module: Tests for Google Vertex AI multi-endpoint fallback
class TestGoogleVertexMultiEndpoint - Test Google Vertex AI model fetching with multi-endpoint fallback
test_first_endpoint_success - Test successful fetch from first endpoint (regional with project)
test_fallback_to_second_endpoint - Test fallback to second endpoint when first fails with 404
test_fallback_to_global_endpoint - Test fallback to global endpoint when regional endpoints fail
test_all_endpoints_fail_returns_none - Test that None is returned when all endpoints fail
test_exception_handling_falls_through - Test that exceptions during fetch are handled and fallback occurs
test_correct_endpoint_urls - Test that the correct endpoint URLs are constructed
References: src/config.py, src/services/google_vertex_client.py
Module: Tests for Google Vertex client tools extraction and translation
class TestOpenAIToVertexToolsTranslation - Test OpenAI tools format to Vertex AI functionDeclarations translation
test_translate_single_tool - Test translating a single OpenAI tool to Vertex format
test_translate_multiple_tools - Test translating multiple OpenAI tools to Vertex format
test_translate_tool_without_description - Test translating a tool without description
test_translate_tool_without_parameters - Test translating a tool without parameters
test_translate_empty_tools_list - Test translating an empty tools list
test_translate_none_tools - Test translating None tools
test_skip_non_function_tools - Test that non-function tools are skipped with warning
test_skip_tool_without_name - Test that tools without name are skipped
test_skip_tool_without_function_definition - Test that tools without function definition are skipped
class TestToolChoiceTranslation - Test OpenAI tool_choice to Vertex AI toolConfig translation
test_translate_tool_choice_none - Test translating tool_choice='none'
test_translate_tool_choice_auto - Test translating tool_choice='auto'
test_translate_tool_choice_required - Test translating tool_choice='required'
test_translate_tool_choice_specific_function - Test translating tool_choice with specific function
test_translate_tool_choice_null - Test translating tool_choice=None
test_translate_unknown_tool_choice_string - Test translating unknown tool_choice string defaults to AUTO
test_translate_tool_choice_function_missing_name - Test translating tool_choice function without name
test_translate_tool_choice_function_is_none - Test translating tool_choice when function value is None
class TestGoogleVertexToolsSupport - Test that Google Vertex client extracts tools parameter
test_tools_extracted_from_kwargs - Test that tools are extracted from kwargs
test_tools_request_completes_successfully - Test that requests with tools complete successfully
References: src/services/groq_client.py
Module: Tests for Groq client
class TestGroqClient - Test Groq client functionality
test_get_groq_client - Test getting Groq client
test_get_groq_client_no_key - Test getting Groq client without API key
test_make_groq_request_openai - Test making request to Groq
test_make_groq_request_openai_with_kwargs - Test making request to Groq with additional parameters
test_make_groq_request_openai_error - Test handling errors from Groq
test_make_groq_request_openai_stream - Test making streaming request to Groq
test_make_groq_request_openai_stream_with_kwargs - Test making streaming request to Groq with additional parameters
test_make_groq_request_openai_stream_error - Test handling streaming errors from Groq
test_process_groq_response - Test processing Groq response
test_process_groq_response_no_usage - Test processing Groq response without usage data
test_process_groq_response_multiple_choices - Test processing Groq response with multiple choices
References: src/services.py, src/services/helicone_client.py, src/services/models.py, src/services/pricing.py
Module: Comprehensive tests for Helicone Client service
class TestHeliconeClient - Test Helicone Client service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
class TestGetHeliconeClient - Test get_helicone_client function
test_get_helicone_client - Test getting Helicone client with valid API key
test_get_helicone_client_no_key - Test getting Helicone client without API key raises error
test_get_helicone_client_empty_key - Test getting Helicone client with empty API key raises error
class TestMakeHeliconeRequest - Test make_helicone_request_openai function
test_make_helicone_request - Test making request to Helicone AI Gateway
test_make_helicone_request_with_kwargs - Test making request with additional parameters
test_make_helicone_request_error - Test handling errors from Helicone
class TestMakeHeliconeRequestStream - Test make_helicone_request_openai_stream function
test_make_helicone_request_stream - Test making streaming request to Helicone
test_make_helicone_request_stream_with_kwargs - Test making streaming request with additional parameters
test_make_helicone_request_stream_error - Test handling errors during streaming
class TestProcessHeliconeResponse - Test process_helicone_response function
test_process_helicone_response - Test processing Helicone response
test_process_helicone_response_no_usage - Test processing response without usage data
test_process_helicone_response_multiple_choices - Test processing response with multiple choices
class TestFetchModelPricing - Test fetch_model_pricing_from_helicone function
test_fetch_pricing_no_api_key - Test fetching pricing without API key returns None
test_fetch_pricing_placeholder_key - Test fetching pricing with placeholder key returns None
test_fetch_pricing_during_catalog_build - Test fetching pricing during catalog build returns None
test_fetch_pricing_from_api - Test fetching pricing from Helicone API
test_fetch_pricing_fallback_to_provider - Test fetching pricing falls back to provider lookup
class TestGetProviderPricing - Test get_provider_pricing_for_helicone_model function
test_get_provider_pricing_during_catalog_build - Test getting provider pricing during catalog build returns None
test_get_provider_pricing_found - Test getting provider pricing when found
test_get_provider_pricing_not_found - Test getting provider pricing when not found
test_get_provider_pricing_with_prefix - Test getting provider pricing with provider prefix
class TestHeliconeTimeout - Test Helicone timeout configuration
test_helicone_timeout_values - Test that timeout values are correctly configured
References: src/services/huggingface_client.py
Module: Tests for HuggingFace 400 error handling improvements
class TestHuggingFace400ErrorHandling - Test improved error handling for HuggingFace 400 Bad Request errors
test_400_error_with_detailed_logging - Test that 400 errors are logged with detailed information including payload
test_400_error_streaming_with_detailed_logging - Test that streaming 400 errors are logged with detailed information
test_non_400_error_standard_logging - Test that non-400 errors use standard error logging
test_400_error_without_text_attribute - Test 400 error handling when response has no text attribute
References: src/services/huggingface_client.py
Module: Tests for Hugging Face Inference API client
class TestHuggingFaceClient - Test Hugging Face Inference API client functionality
test_get_huggingface_client - Test getting Hugging Face client
test_get_huggingface_client_no_key - Test getting Hugging Face client without API key
test_make_huggingface_request_openai - Test making request to Hugging Face
test_make_huggingface_request_openai_stream - Test making streaming request to Hugging Face
test_make_huggingface_request_openai_foreign_namespace - Models scoped to other providers should not be routed through Hugging Face
test_make_huggingface_request_openai_stream_foreign_namespace - Streaming helper should also reject foreign provider models
test_process_huggingface_response - Test processing Hugging Face response
class TestModelIdCaseResolution - Test model ID case resolution for HuggingFace Router API
test_resolve_model_id_case_with_cached_model - Test that lowercase model IDs are resolved to correct case from cache
test_resolve_model_id_case_preserves_suffix - Test that :hf-inference suffix is preserved after case resolution
test_resolve_model_id_case_not_in_cache - Test that model IDs not in cache are returned as-is
test_resolve_model_id_case_empty_cache - Test that model IDs are returned as-is when cache is empty
test_resolve_model_id_case_already_correct - Test that correctly-cased model IDs are returned unchanged
test_prepare_model_resolves_case_and_adds_suffix - Test that _prepare_model resolves case and adds suffix
test_prepare_model_does_not_double_suffix - Test that _prepare_model doesn't add suffix if already present
test_request_uses_correct_case - Test that requests use the correctly-cased model ID
References: src/services.py, src/services/huggingface_models.py
Module: Comprehensive tests for Huggingface Models service
class TestHuggingfaceModels - Test Huggingface Models service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/services/huggingface_client.py
Module: Tests for HuggingFace client tools support
class TestHuggingFaceToolsSupport - Test that HuggingFace client supports tools parameter
test_tools_in_allowed_params - Test that 'tools' is in ALLOWED_PARAMS
test_build_payload_includes_tools - Test that _build_payload includes tools when provided
test_build_payload_without_tools - Test that _build_payload works without tools
test_build_payload_filters_other_params - Test that _build_payload only includes allowed params
References: src/services.py, src/services/image_generation_client.py
Module: Comprehensive tests for Image Generation Client service
class TestImageGenerationClient - Test Image Generation Client service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/services/huggingface_models.py, src/services/models.py, src/services/providers.py
Module: Tests for robust JSON parsing error handling across service modules.
class TestHuggingFaceModelsJSONParsing - Test JSON parsing error handling in huggingface_models.py
test_fetch_models_with_invalid_json_response - Test that fetch_huggingface_models handles invalid JSON gracefully
test_search_models_with_invalid_json - Test that search_huggingface_models handles malformed JSON
test_get_model_info_with_invalid_json - Test that get_huggingface_model_info handles JSON errors
test_fetch_models_with_html_error_response - Test handling when API returns HTML error page instead of JSON
class TestProvidersJSONParsing - Test JSON parsing error handling in providers.py
test_fetch_providers_with_invalid_json - Test that fetch_providers handles malformed JSON response
test_fetch_providers_with_empty_response - Test handling of empty response body
class TestModelsJSONParsing - Test JSON parsing error handling in models.py
test_fetch_openrouter_models_with_invalid_json - Test that fetch_openrouter_models handles JSON parse errors
test_fetch_models_with_corrupted_json_response - Test handling of corrupted/truncated JSON
class TestJSONParsingWithDifferentContentTypes - Test that services handle unexpected Content-Type headers
test_xml_content_type_returned_instead_of_json - Test when API incorrectly returns XML but claims JSON
test_plain_text_error_message_as_json - Test handling when API returns plain text error
class TestJSONParsingErrorLogging - Verify that JSON parsing errors are logged appropriately
test_json_error_is_logged_with_context - Verify JSON parse errors are logged with useful context
test_providers_json_error_logging - Verify provider JSON errors include response details
class TestJSONParsingEdgeCases - Test edge cases in JSON parsing
test_null_json_response - Test when response.json() returns None
test_json_with_unexpected_structure - Test when JSON structure differs from expected
References: src/services.py, src/services/model_availability.py
Module: Comprehensive tests for Model Availability service
class TestModelAvailability - Test Model Availability service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
class TestIsModelAvailable - Tests for the is_model_available method
test_unknown_model_returns_available - Test that unknown models (not in availability cache) return True.
test_available_model_returns_available - Test that models with AVAILABLE status return True
test_model_with_open_circuit_returns_unavailable - Test that models with OPEN circuit breaker return False
test_model_in_maintenance_returns_unavailable - Test that models in maintenance mode return False
test_unavailable_status_returns_unavailable - Test that models with UNAVAILABLE status return False
References: src/services/model_availability.py
Module: Comprehensive tests for Model Availability Service and Circuit Breaker
class TestCircuitBreakerStateTransitions - Test circuit breaker state machine
test_initial_state_is_closed - Test circuit breaker starts in CLOSED state
test_can_execute_when_closed - Test requests can execute when CLOSED
test_transitions_to_open_after_failures - Test CLOSED -> OPEN after failure threshold
test_cannot_execute_when_open - Test requests blocked when OPEN
test_transitions_to_half_open_after_recovery_timeout - Test OPEN -> HALF_OPEN after recovery timeout
test_half_open_to_closed_after_successes - Test HALF_OPEN -> CLOSED after success threshold
test_half_open_to_open_on_failure - Test HALF_OPEN -> OPEN on failure
test_success_decrements_failure_count_when_closed - Test success reduces failure count in CLOSED state
class TestAvailabilityConfig - Test availability configuration
test_default_config_values - Test default configuration values
test_custom_config_values - Test custom configuration values
class TestModelAvailabilityDataclass - Test ModelAvailability dataclass
test_create_model_availability - Test creating ModelAvailability instance
class TestModelAvailabilityService - Test ModelAvailabilityService functionality
test_service_initialization - Test service initializes correctly
test_fallback_mappings_loaded - Test fallback mappings are loaded correctly
test_get_fallback_models - Test getting fallback models
test_get_model_availability_not_found - Test getting availability for unknown model
test_get_model_availability_with_gateway - Test getting availability with specific gateway
test_get_available_models_empty - Test getting available models when cache is empty
test_get_available_models_filtered - Test getting available models with filters
test_is_model_available - Test model availability check
test_is_model_available_with_open_circuit - Test availability check with open circuit breaker
test_is_model_available_during_maintenance - Test availability check during maintenance
test_get_best_available_model_preferred_available - Test best model selection when preferred is available
test_get_best_available_model_with_fallback - Test best model selection falls back when preferred unavailable
test_get_availability_summary_empty - Test availability summary with empty cache
test_get_availability_summary - Test availability summary with data
test_set_maintenance_mode - Test setting maintenance mode
test_clear_maintenance_mode - Test clearing maintenance mode
class TestModelAvailabilityServiceAsync - Test async functionality of ModelAvailabilityService
test_start_monitoring - Test starting availability monitoring
test_start_monitoring_idempotent - Test starting monitoring multiple times is idempotent
test_stop_monitoring_when_not_started - Test stopping monitoring when not started
class TestAvailabilityStatus - Test AvailabilityStatus enum
test_available_status - Test AVAILABLE status
test_unavailable_status - Test UNAVAILABLE status
test_degraded_status - Test DEGRADED status
test_maintenance_status - Test MAINTENANCE status
test_unknown_status - Test UNKNOWN status
class TestGlobalAvailabilityService - Test the global availability_service instance
test_global_service_exists - Test global service is initialized
test_global_service_has_fallback_mappings - Test global service has fallback mappings loaded
References: src/services/google_vertex_client.py, src/services/model_catalog_sync.py
Module: Tests for model catalog sync functionality.
class TestTransformNormalizedModelToDbSchema - Tests for transform_normalized_model_to_db_schema function.
test_uses_explicit_provider_model_id - Test that explicit provider_model_id is used when available.
test_falls_back_to_model_id_when_provider_model_id_missing - Test that model_id is used when provider_model_id is not specified.
test_handles_none_provider_model_id - Test that None provider_model_id falls back to model_id.
class TestGoogleVertexProviderModelId - Tests specifically for Google Vertex model ID handling.
test_gemini_3_flash_provider_model_id - Test that Gemini 3 Flash models have correct provider_model_id.
test_gemini_3_pro_provider_model_id - Test that Gemini 3 Pro models have correct provider_model_id.
test_gemini_2_models_same_provider_model_id - Test that Gemini 2.x models have same model_id and provider_model_id.
class TestExtractModality - Tests for extract_modality function.
test_extract_from_architecture - Test modality extraction from architecture field.
test_extract_from_top_level - Test modality extraction from top-level field.
test_default_modality - Test default modality when not specified.
class TestExtractPricing - Tests for extract_pricing function.
test_extract_valid_pricing - Test pricing extraction with valid values.
test_extract_missing_pricing - Test pricing extraction with missing pricing.
class TestExtractCapabilities - Tests for extract_capabilities function.
test_extract_vision_support - Test vision support extraction.
test_no_vision_support - Test no vision support.
class TestProviderFetchFunctionsRegistry - Tests for PROVIDER_FETCH_FUNCTIONS registry completeness.
test_openai_registered - Test that OpenAI provider is registered.
test_anthropic_registered - Test that Anthropic provider is registered.
test_clarifai_registered - Test that Clarifai provider is registered.
test_simplismart_registered - Test that SimpliSmart provider is registered.
test_onerouter_registered - Test that OneRouter provider is registered.
test_cloudflare_workers_ai_registered - Test that Cloudflare Workers AI provider is registered.
test_modelz_registered - Test that Modelz provider is registered.
test_all_core_providers_registered - Test that all core providers have fetch functions registered.
test_provider_count - Test that we have the expected number of providers registered.
References: src/services/model_catalog_validator.py
Module: Tests for model catalog validation service.
class TestValidateModelAvailability - Test model availability validation
test_validate_cerebras_model_available - Test validating available Cerebras model
test_validate_cerebras_model_unavailable - Test validating unavailable Cerebras model
test_validate_huggingface_model_available - Test validating available HuggingFace model
test_validate_huggingface_model_not_on_hub - Test model not found on HuggingFace Hub
test_validate_huggingface_model_not_on_inference_router - Test model exists on Hub but not available on Inference Router
test_validate_openrouter_model_available - Test validating available OpenRouter model
test_validate_openrouter_model_unavailable - Test validating unavailable OpenRouter model
test_validate_unsupported_provider - Test validation for provider without validation implementation
test_validation_caching - Test that validation results are cached
class TestValidateModelsBatch - Test batch model validation
test_validate_batch_all_available - Test validating batch where all models are available
test_validate_batch_some_unavailable - Test validating batch where some models are unavailable
test_validate_batch_empty_list - Test validating empty batch
test_validate_batch_with_exceptions - Test batch validation handles exceptions gracefully
class TestValidationCacheManagement - Test validation cache management
test_clear_specific_model_cache - Test clearing cache for specific model
test_clear_all_validation_cache - Test clearing all validation cache
References: src/services/model_health_monitor.py, src/utils/sentry_context.py
Module: Tests for Model Health Monitor Service
class TestModelHealthTracking - Test model health tracking functionality using ModelHealthMonitor methods
test_get_model_health_returns_none_for_unknown - Get health for unknown model returns None
test_get_all_models_health_returns_list - Get all models health returns a list
test_get_health_summary_returns_dict - Get health summary returns a dictionary
test_get_system_health_initially_none - System health is None before monitoring starts
class TestHealthStatusManagement - Test health status management via ModelHealthMonitor
test_get_provider_health_returns_none_for_unknown - Get provider health for unknown provider returns None
test_get_all_providers_health_returns_list - Get all providers health returns a list
test_health_data_structure - Health data internal structure is initialized
class TestHealthScoreCalculation - Test health score calculation
test_calculate_health_score_all_success - Calculate health score with all successful requests
test_calculate_health_score_mixed - Calculate health score with mixed results
test_calculate_health_score_all_failures - Calculate health score with all failures
class TestHealthMetrics - Test health metrics collection
test_model_health_metrics_dataclass - Test ModelHealthMetrics dataclass structure
test_model_health_metrics_unhealthy - Test ModelHealthMetrics with unhealthy status
class TestFailureDetection - Test failure detection and thresholds
test_detect_high_error_rate - Detect when error rate exceeds threshold
test_detect_slow_response - Detect when latency exceeds threshold
test_consecutive_failures - Detect consecutive failures
class TestRecoveryDetection - Test recovery detection
test_health_status_enum_values - Test HealthStatus enum has expected values
test_health_status_transitions - Test that health status can transition between states
class TestHealthAlerts - Test health monitoring alerts
test_alert_on_model_failure - Alert should trigger when model fails
test_alert_cooldown - Alerts should have cooldown period
class TestHealthMonitoringEdgeCases - Test edge cases
test_get_model_health_with_none_model_id - Handle None model ID gracefully
test_get_model_health_with_empty_string - Handle empty string model ID
test_get_provider_health_with_none_values - Handle None provider/gateway gracefully
class TestHealthMonitorPersistence - Test health data persistence
test_health_data_dict_initialized - Health data dictionary is initialized on monitor creation
test_model_health_metrics_has_timestamp - ModelHealthMetrics includes last_checked timestamp
class TestHealthMonitorIntegration - Test integration with other systems
test_unhealthy_models_excluded_from_routing - Unhealthy models should be excluded from routing
test_health_check_integration - Health monitor should integrate with health check endpoint
class TestPerformanceMetrics - Test performance metrics calculation
test_calculate_success_rate - Calculate success rate correctly
test_calculate_average_latency - Calculate average latency
test_calculate_error_rate - Calculate error rate
class TestSentryErrorCapture - Test Sentry error capture for model failures
test_sentry_capture_on_model_failure - Test that Sentry captures errors when models fail health checks
test_sentry_capture_on_exception - Test that Sentry captures exceptions during health checks
test_sentry_not_captured_on_success - Test that Sentry does not capture errors when models are healthy
class TestApiKeyForGateway - Test API key retrieval for different gateways
test_get_api_key_for_known_gateway - Test that API key is returned for known gateways
test_get_api_key_for_unknown_gateway - Test that None is returned for unknown gateways
class TestPerformModelRequestAuthentication - Test authentication in _perform_model_request
test_returns_error_when_no_api_key_configured - Test that request fails when no API key is configured for gateway
test_returns_error_for_unknown_gateway - Test that request fails for unknown gateway (no URL configured)
class TestShouldCaptureError - Test _should_capture_error method for intelligent error filtering
test_should_capture_unknown_errors - Test that unknown errors (no status code) are captured
test_should_not_capture_rate_limits - Test that rate limit errors (429) are NOT captured
test_should_not_capture_data_policy_errors - Test that data policy restriction errors (404) are NOT captured
test_should_capture_other_404_errors - Test that other 404 errors (not data policy) ARE captured
test_should_not_capture_service_unavailable - Test that service unavailable errors (503) are NOT captured
test_should_capture_other_503_errors - Test that other 503 errors ARE captured
test_should_not_capture_max_output_tokens_validation - Test that max_output_tokens validation errors (400) are NOT captured
test_should_not_capture_audio_modality_errors - Test that audio modality requirement errors (400) are NOT captured
test_should_capture_other_400_errors - Test that other 400 errors ARE captured
test_should_not_capture_auth_key_errors - Test that authentication key errors (403) are NOT captured
test_should_capture_other_403_errors - Test that other 403 errors ARE captured
test_should_capture_server_errors - Test that server errors (500, 502, etc.) ARE captured
test_should_capture_client_errors - Test that most client errors ARE captured
test_error_filtering_in_check_model_health_rate_limit - Test that rate limit errors are not sent to Sentry during health checks
test_error_filtering_in_check_model_health_server_error - Test that server errors ARE sent to Sentry during health checks
References: src/services/model_transformations.py
test_openrouter_prefixed_model_keeps_nested_provider
test_openrouter_gpt51_hyphen_alias_transforms
test_detect_provider_gpt51_alias_without_org - Test that gpt-5-1 alias routes to native OpenAI provider.
test_bare_openai_model_names_alias_to_canonical - Test that bare OpenAI model names (without openai/ prefix) are aliased correctly.
test_bare_openai_model_names_detect_as_native_openai - Test that bare OpenAI model names are detected as native OpenAI provider.
test_bare_anthropic_model_names_alias_to_canonical - Test that bare Anthropic/Claude model names (without anthropic/ prefix) are aliased correctly.
test_bare_anthropic_model_names_detect_as_native_anthropic - Test that bare Anthropic/Claude model names are detected as native Anthropic provider.
test_openrouter_auto_preserves_prefix
test_openrouter_auto_transforms_for_huggingface
test_openrouter_auto_transforms_for_cerebras
test_detect_provider_from_model_id_fal_ai - Test that fal-ai models are detected as 'fal' provider
test_detect_provider_from_model_id_fal_orgs - Test that various Fal-related orgs are detected as 'fal' provider
test_detect_provider_from_model_id_existing_providers - Test that existing provider detection still works.
test_detect_provider_google_vertex_models - Test that Google Vertex AI models are correctly detected when credentials are available
test_detect_provider_at_prefix_models - Test that models with @ prefix are routed to OpenRouter after Portkey removal
test_z_ai_glm_with_exacto_suffix - Test that z-ai/glm-4.6:exacto is correctly detected as OpenRouter
test_z_ai_glm_prefix_detected_as_openrouter - Test that z-ai/ prefixed models are detected as OpenRouter provider
test_z_ai_glm_47_alias_transforms - Test that z-ai/glm-4.7 (non-existent model) is aliased to existing model
test_openrouter_colon_suffix_variants - Test that OpenRouter models with colon suffixes are correctly detected
test_detect_provider_groq_models - Test that Groq models are correctly detected as 'groq' provider
test_transform_groq_model_strips_prefix - Test that groq/ prefix is stripped when transforming for Groq provider
test_transform_groq_model_without_prefix - Test that Groq models without prefix pass through correctly
test_detect_provider_onerouter_prefixed_models - Test that onerouter/ prefixed models are detected as 'onerouter' provider
test_detect_provider_onerouter_versioned_models - Test that OneRouter models with @ version suffix are detected correctly
test_transform_onerouter_strips_prefix - Test that onerouter/ prefix is stripped when transforming for OneRouter provider
test_transform_onerouter_passthrough_versioned - Test that versioned OneRouter models pass through correctly
test_transform_onerouter_simple_names - Test that simple model names get @ version suffix added
test_transform_onerouter_unknown_model_passthrough - Test that unknown models pass through as lowercase
test_onerouter_model_id_mapping_exists - Test that OneRouter has model ID mappings defined
test_fireworks_unknown_model_does_not_construct_invalid_id - Test that unknown models are NOT naively transformed into invalid Fireworks IDs.
test_fireworks_known_model_still_transforms - Test that known Fireworks models are still properly transformed.
test_fireworks_unknown_model_without_slash_passthrough - Test that unknown models without org prefix pass through as-is.
test_fireworks_nonexistent_variant_passthrough - Test various nonexistent model variants pass through without naive construction.
test_fireworks_fuzzy_match_still_works - Test that fuzzy matching still works for models with slight variations.
References: src/services/model_transformations.py
Module: Comprehensive tests for Model Transformations
class TestModelAliasResolution - Test model alias resolution
test_apply_model_alias_gpt_variants - Test GPT-5.1 alias variants
test_apply_model_alias_xai_deprecated - Test XAI deprecated model aliases - now mapped to canonical x-ai/ prefix
test_apply_model_alias_case_insensitive - Test that alias lookup is case insensitive
test_apply_model_alias_no_match - Test non-aliased models pass through
test_apply_model_alias_none_input - Test None input returns None
test_apply_model_alias_empty_string - Test empty string returns empty string
class TestFireworksTransformations - Test Fireworks AI model transformations
test_transform_deepseek_v3 - Test DeepSeek-V3 transformation
test_transform_deepseek_v3_alt_org - Test DeepSeek-V3 with alternative org prefix
test_transform_llama_3_3_70b - Test Llama 3.3 70B transformation
test_transform_llama_3_1_70b - Test Llama 3.1 70B transformation
test_transform_qwen_models - Test Qwen model transformations
test_transform_already_in_fireworks_format - Test model already in Fireworks format passes through
test_transform_unknown_model_passes_through - Test unknown model passes through as-is (Fireworks API will reject if invalid)
test_transform_deepseek_r1 - Test DeepSeek-R1 transformation
class TestOpenRouterTransformations - Test OpenRouter model transformations
test_transform_openrouter_prefix_stripped - Test openrouter/ prefix is stripped
test_transform_openrouter_auto_preserved - Test openrouter/auto is preserved
test_transform_openrouter_bodybuilder_preserved - Test openrouter/bodybuilder is preserved
test_transform_openrouter_meta_models_preserved - Test all OpenRouter meta-models are preserved
test_transform_claude_sonnet_variants - Test Claude Sonnet 4.5 transformation variants
test_transform_cerebras_to_openrouter - Test Cerebras model routed through OpenRouter
class TestOpenRouterAutoFallbacks - Test OpenRouter auto fallback behavior
test_openrouter_auto_fallback_to_cerebras - Test openrouter/auto fallback to Cerebras
test_openrouter_auto_fallback_to_huggingface - Test openrouter/auto fallback to HuggingFace
test_openrouter_auto_fallback_to_google_vertex - Test openrouter/auto fallback to Google Vertex
test_openrouter_auto_fallback_to_alibaba - Test openrouter/auto fallback to Alibaba Cloud
class TestGroqTransformations - Test Groq model transformations
test_transform_groq_prefix_stripped - Test groq/ prefix is stripped
test_transform_groq_mixtral - Test Groq Mixtral transformation
test_transform_groq_gemma - Test Groq Gemma transformation
test_transform_groq_without_prefix - Test Groq model without prefix
class TestGoogleVertexTransformations - Test Google Vertex AI model transformations
test_transform_gemini_2_5_flash - Test Gemini 2.5 Flash transformation
test_transform_gemini_with_google_prefix - Test Gemini with google/ prefix
test_transform_gemini_with_at_prefix - Test Gemini with @google/models/ prefix
test_transform_gemini_1_5_pro - Test Gemini 1.5 Pro transformation
test_transform_gemini_2_0_flash - Test Gemini 2.0 Flash transformation
class TestHuggingFaceTransformations - Test HuggingFace model transformations
test_transform_llama_models - Test Llama model transformations
test_transform_deepseek_models - Test DeepSeek model transformations
test_transform_qwen_models - Test Qwen model transformations
test_transform_mistral_models - Test Mistral model transformations
test_hug_alias_same_as_huggingface - Test 'hug' alias behaves same as 'huggingface'
class TestNearTransformations - Test Near AI model transformations
test_transform_near_prefix_stripped - Test near/ prefix is stripped
class TestAIMOTransformations - Test AIMO model transformations
test_transform_aimo_prefix_stripped - Test aimo/ prefix is stripped
class TestMorpheusTransformations - Test Morpheus model transformations
test_transform_morpheus_prefix_stripped - Test morpheus/ prefix is stripped
class TestProviderDetection - Test provider detection from model ID
test_detect_fal_ai_provider - Test Fal.ai provider detection
test_detect_fal_related_orgs - Test Fal-related org detection
test_detect_openrouter_colon_suffix - Test OpenRouter models with colon suffix
test_detect_groq_models - Test Groq model detection
test_detect_google_vertex_models - Test Google Vertex AI model detection
test_detect_at_prefix_models - Test @ prefix model detection
class TestModelIdNormalization - Test model ID normalization
test_transform_normalizes_to_lowercase - Test that output is normalized to lowercase
test_transform_empty_string - Test empty string input
test_transform_none_input - Test None input
class TestGetModelIdMapping - Test model ID mapping retrieval
test_get_fireworks_mapping - Test getting Fireworks mapping
test_get_openrouter_mapping - Test getting OpenRouter mapping
test_get_groq_mapping - Test getting Groq mapping
test_get_google_vertex_mapping - Test getting Google Vertex AI mapping
test_get_unknown_provider_mapping - Test getting mapping for unknown provider returns empty dict
class TestModelProviderOverrides - Test model provider overrides
test_katanemo_override - Test Katanemo model override
test_zai_override - Test Z-AI model override
class TestOpenRouterAutoFallbackConfig - Test OpenRouter auto fallback configuration
test_fallback_providers_defined - Test fallback providers are defined
test_fallback_models_valid - Test fallback models are valid model IDs
class TestEdgeCases - Test edge cases and special scenarios
test_transform_with_version_variations - Test transformation handles version variations
test_transform_preserves_special_characters - Test transformation preserves necessary special characters
test_transform_handles_mixed_case - Test transformation handles mixed case input
test_transform_with_instruct_suffix - Test transformation handles instruct suffix
class TestNormalizeModelName - Test model name normalization function
test_normalize_basic - Test basic normalization
test_normalize_removes_version_separators - Test normalization handles version separators
References: src/services/models.py
Module: Tests for models service functions
class TestSanitizePricing - Test pricing sanitization
test_sanitize_none - Test with None input
test_sanitize_empty_dict - Test with empty dict
test_sanitize_non_dict - Test with non-dict input
test_sanitize_positive_values - Test that positive values are preserved
test_sanitize_negative_values_returns_none - Test that negative values (dynamic pricing) return None to filter model
test_sanitize_mixed_values_with_negative_returns_none - Test with mix of positive and negative values returns None
test_sanitize_numeric_negative_values_returns_none - Test with numeric negative values returns None
test_sanitize_invalid_values - Test with invalid numeric values
test_sanitize_zero_values - Test that zero values are preserved
test_sanitize_with_negative_prompt_returns_none - Test that negative pricing in prompt returns None (filters model)
test_sanitize_preserves_other_fields_with_valid_pricing - Test that non-pricing fields are preserved when pricing is valid
class TestLoadFeatherlessCatalogExport - Test Featherless catalog loading
test_load_when_no_file_exists - Test when CSV file doesn't exist
test_load_file_not_found - Test when file is not found
class TestGetCachedModels - Test model caching functionality
test_get_cached_models_openrouter - Test getting OpenRouter models from cache
test_get_cached_models_featherless - Test getting Featherless models
test_get_cached_models_groq - Test getting Groq models
test_get_cached_models_hug - Test getting Hugging Face models
test_get_cached_models_huggingface_alias - Test huggingface alias
test_get_cached_models_invalid_gateway - Test with invalid gateway name
test_get_cached_models_case_insensitive - Test that gateway names are case-insensitive
class TestEnhanceModelWithProviderInfo - Test model enhancement with provider info
test_enhance_empty_model - Test enhancing empty model
test_enhance_model_basic - Test basic model enhancement
test_enhance_model_no_providers - Test when no providers available
test_enhance_preserves_existing_fields - Test that existing fields are preserved
class TestEnhanceModelWithHuggingFaceData - Test Hugging Face data enhancement
test_enhance_with_hf_data_success - Test successful HF enhancement
test_enhance_with_hf_data_none - Test when HF returns None
test_enhance_with_hf_data_no_model - Test with empty model
class TestGetModelCountByProvider - Test model counting by provider
test_count_empty_list - Test counting with empty model list
test_count_no_providers - Test with no providers
test_count_basic - Test basic counting
test_count_with_none_inputs - Test with None inputs
class TestFetchSpecificModel - Test fetching specific model
test_fetch_model_not_found - Test fetching non-existent model
test_fetch_model_invalid_format - Test with invalid model format
test_fetch_model_none - Test with None input
test_fetch_openrouter_auto - Test fetching openrouter/auto model specifically
class TestFalAiIntegration - Test Fal.ai model integration
test_fetch_specific_model_from_fal_success - Test successful fetch from Fal.ai cache
test_fetch_specific_model_from_fal_case_insensitive - Test case-insensitive matching
test_fetch_specific_model_from_fal_not_found - Test model not found in cache
test_fetch_specific_model_from_fal_empty_cache - Test with empty cache
test_fetch_specific_model_from_fal_exception - Test exception handling
test_detect_model_gateway_fal - Test that Fal models are detected in gateway detection
test_detect_model_gateway_fallback_to_openrouter - Test fallback to openrouter when model not found
class TestOpenRouterIsFreeField - Test that OpenRouter models correctly set the is_free field
test_openrouter_free_model_sets_is_free_true - OpenRouter models with :free suffix should have is_free=True
test_openrouter_paid_model_sets_is_free_false - OpenRouter models without :free suffix should have is_free=False
test_openrouter_mixed_models_correct_is_free - Test that mixed models (free and paid) are correctly marked
test_openrouter_exacto_and_extended_suffixes_not_free - Models with :exacto or :extended suffixes should NOT be marked as free
test_openrouter_null_model_id_does_not_crash - Models with null id should not crash and should have is_free=False
References: src/config.py, src/db/models_catalog_db.py, src/routes/catalog.py, src/services/models.py
Module: Test that get_all_models can be imported and works correctly.
class TestGetAllModelsImport - Test suite for get_all_models import compatibility.
test_get_all_models_can_be_imported - Verify get_all_models can be imported from models module.
test_get_all_models_is_alias_of_parallel - Verify get_all_models is an alias of get_all_models_parallel.
test_get_all_models_returns_models_including_simplismart - Verify get_all_models returns models including SimpliSmart.
test_startup_can_import_get_all_models - Verify the startup module can successfully import get_all_models.
test_all_provider_gateways_included - Verify all key providers are supported by get_all_models_parallel.
test_gateway_registry_consistency - Verify all gateways from GATEWAY_REGISTRY are included in parallel fetch.
test_morpheus_gateway_included - Verify morpheus gateway is included in parallel fetch.
test_vercel_ai_gateway_included - Verify vercel-ai-gateway is included in parallel fetch.
test_sybil_gateway_included - Verify sybil gateway is included in parallel fetch.
test_near_ai_fallback_models_are_correct - Verify Near AI fallback models match the actual models available on the platform.
test_get_fallback_models_from_db_function_exists - Verify get_fallback_models_from_db function is available.
test_get_fallback_models_from_db_handles_missing_provider - Verify get_fallback_models_from_db returns None for unknown provider.
test_get_fallback_models_from_db_converts_models_correctly - Verify get_fallback_models_from_db converts database models to raw format.
test_get_fallback_models_from_db_near_pricing_format - Verify Near AI models get correct pricing format with amount and scale.
References: src/services.py, src/services/modelz_client.py
Module: Comprehensive tests for Modelz Client service
class TestModelzClient - Test Modelz Client service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/cache.py, src/config.py, src/routes/catalog.py, src/services.py, src/services/connection_pool.py, src/services/model_transformations.py, src/services/morpheus_client.py
Module: Comprehensive tests for Morpheus AI Gateway Client service
class TestMorpheusClient - Test Morpheus Client service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
test_morpheus_base_url - Test Morpheus base URL is correctly set
test_get_morpheus_client_raises_without_api_key - Test that get_morpheus_client raises error without API key
test_get_morpheus_client_returns_pooled_client - Test that get_morpheus_client returns pooled client
test_make_morpheus_request_openai - Test making a request through Morpheus
test_make_morpheus_request_openai_stream - Test making a streaming request through Morpheus
test_process_morpheus_response - Test processing Morpheus response
test_process_morpheus_response_with_tool_calls - Test processing Morpheus response with tool calls
test_fetch_models_from_morpheus_success - Test fetching models from Morpheus API
test_fetch_models_from_morpheus_updates_cache_on_success - Test that cache timestamp is updated after successful fetch
test_fetch_models_from_morpheus_updates_cache_on_http_error - Test that cache timestamp is updated even when API fails (prevents repeated calls)
test_fetch_models_from_morpheus_no_api_key - Test fetch_models returns empty list without API key
test_fetch_models_from_morpheus_updates_cache_when_no_api_key - Test that cache is updated even when API key is missing
test_fetch_models_from_morpheus_skips_empty_ids - Test that models with empty or missing IDs are skipped
class TestMorpheusModelTransformations - Test model ID transformations for Morpheus
test_morpheus_provider_detection - Test that morpheus/ prefix models are detected correctly
test_morpheus_prefix_stripping - Test that morpheus/ prefix is stripped during transformation
test_morpheus_direct_model_passthrough - Test that direct model names pass through
class TestMorpheusConfig - Test Morpheus configuration
test_config_has_morpheus_api_key - Test that Config class has MORPHEUS_API_KEY attribute
class TestMorpheusCacheIntegration - Test Morpheus cache integration
test_morpheus_cache_exists - Test that Morpheus cache is defined in cache module
test_morpheus_cache_in_get_models_cache - Test that Morpheus is included in get_models_cache mapping
test_morpheus_cache_clearable - Test that Morpheus cache can be cleared
class TestMorpheusGatewayRegistry - Test Morpheus gateway registry integration
test_morpheus_in_gateway_registry - Test that Morpheus is in the GATEWAY_REGISTRY
class TestMorpheusConnectionPool - Test Morpheus connection pool integration
test_morpheus_pooled_client_function_exists - Test that get_morpheus_pooled_client function exists
test_morpheus_pooled_client_raises_without_key - Test that get_morpheus_pooled_client raises without API key
References: src/services.py, src/services/multi_provider_registry.py
Module: Comprehensive tests for Multi Provider Registry service
class TestMultiProviderRegistry - Test Multi Provider Registry service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/services.py, src/services/near_client.py
Module: Comprehensive tests for Near Client service
class TestNearClient - Test Near Client service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/services/nosana_client.py
Module: Tests for Nosana GPU Computing Network client
class TestNosanaClient - Test Nosana client functionality
test_get_nosana_client - Test getting Nosana client
test_get_nosana_client_no_key - Test getting Nosana client without API key
test_make_nosana_request_openai - Test making request to Nosana
test_make_nosana_request_openai_stream - Test making streaming request to Nosana
test_process_nosana_response - Test processing Nosana response
test_process_nosana_response_no_usage - Test processing Nosana response without usage data
test_make_nosana_request_with_kwargs - Test making request to Nosana with additional parameters
class TestNosanaJobDefinitionBuilders - Test Nosana job definition builders
test_build_container_job_definition_basic - Test building a basic container job definition
test_build_container_job_definition_with_all_options - Test building a container job definition with all options
test_build_llm_inference_job_definition_vllm - Test building vLLM inference job definition
test_build_llm_inference_job_definition_ollama - Test building Ollama inference job definition
test_build_llm_inference_job_definition_lmdeploy - Test building LMDeploy inference job definition
test_build_llm_inference_job_definition_invalid_framework - Test building LLM inference with invalid framework
test_build_stable_diffusion_job_definition - Test building Stable Diffusion job definition
test_build_whisper_job_definition - Test building Whisper job definition
class TestNosanaDeploymentAPI - Test Nosana deployment API functions
test_get_credits_balance - Test getting credit balance
test_list_deployments - Test listing deployments
test_create_deployment - Test creating a deployment
test_create_deployment_invalid_strategy - Test creating deployment with invalid strategy
test_create_deployment_invalid_timeout - Test creating deployment with invalid timeout
test_create_deployment_invalid_replicas - Test creating deployment with invalid replicas
class TestNosanaJobsAPI - Test Nosana jobs API functions
test_create_job - Test creating a job
test_create_job_invalid_timeout - Test creating job with invalid timeout
test_extend_job_invalid_timeout - Test extending job with invalid timeout
class TestNosanaMarketsAPI - Test Nosana markets API functions
test_list_markets - Test listing markets
test_list_markets_with_filter - Test listing markets with type filter
test_list_markets_invalid_type - Test listing markets with invalid type
class TestNosanaConstants - Test Nosana constants
test_deployment_strategies - Test deployment strategies are defined
test_deployment_statuses - Test deployment statuses are defined
References: src/schemas/notification.py, src/services/notification.py
Module: Comprehensive tests for notification service
class TestUserPreferences - Test notification preference management
test_get_user_preferences_exists - Test getting existing user preferences
test_get_user_preferences_not_found - Test getting preferences for user without preferences
test_get_user_preferences_error_handling - Test error handling when getting preferences
test_create_user_preferences_success - Test creating default preferences
test_create_user_preferences_error - Test error handling when creating preferences
test_update_user_preferences_success - Test updating user preferences
test_update_user_preferences_not_found - Test updating non-existent preferences
test_update_user_preferences_error - Test error handling when updating preferences
class TestLowBalanceAlerts - Test low balance alert detection
test_check_low_balance_alert_triggered - Test low balance alert is triggered when credits < $5
test_check_low_balance_alert_not_triggered - Test low balance alert not triggered when credits > $5
test_check_low_balance_alert_trial_user - Test low balance alert for trial user includes trial info
test_check_low_balance_alert_user_not_found - Test low balance alert when user not found
test_check_low_balance_alert_notifications_disabled - Test low balance alert when notifications are disabled
test_check_low_balance_alert_recent_notification - Test low balance alert skipped if recent notification sent
class TestTrialExpiryAlerts - Test trial expiry alert detection
test_check_trial_expiry_alert_triggered - Test trial expiry alert triggered 1 day before expiry
test_check_trial_expiry_alert_not_trial_user - Test no alert for non-trial users
test_check_trial_expiry_alert_too_early - Test no alert when trial has >1 day remaining
class TestRecentNotificationTracking - Test recent notification tracking
test_has_recent_notification_found - Test detecting recent notification
test_has_recent_notification_not_found - Test no recent notification
test_has_recent_notification_error - Test error handling in recent notification check
class TestNotificationServiceInit - Test NotificationService initialization
test_init_with_environment_variables - Test initialization with environment variables
test_init_with_defaults - Test initialization with default values
class TestNotificationIntegration - Test notification service integration scenarios
test_low_balance_alert_with_plan_info - Test low balance alert includes plan information for paid users
class TestEmailValidation - Test email address validation for notification service
test_valid_email_addresses - Test that valid email addresses pass validation
test_privy_placeholder_emails_rejected - Test that Privy placeholder emails are rejected
test_did_prefix_emails_rejected - Test that DID-prefixed emails are rejected
test_malformed_emails_rejected - Test that malformed emails are rejected
test_send_email_skips_invalid_addresses - Test that send_email_notification skips invalid email addresses
test_send_email_proceeds_for_valid_addresses - Test that send_email_notification proceeds for valid email addresses
References: src/services.py, src/services/novita_client.py
Module: Comprehensive tests for src/services/novita_client.py
class TestNovitaSDKImport - Test SDK import and availability detection
test_sdk_availability_flag_exists - Test that NOVITA_SDK_AVAILABLE flag is defined
test_module_imports_successfully - Test that module can be imported without errors
class TestGetNovitaSDKClient - Test get_novita_sdk_client function
test_returns_none_when_sdk_not_available - Test that function returns None when SDK is not installed
test_raises_error_when_api_key_missing - Test that function raises ValueError when API key is not configured
test_returns_client_when_sdk_available_and_key_configured - Test that function returns NovitaClient instance when everything is configured
class TestFetchModelsFromNovita - Test fetch_models_from_novita function (LLM models)
test_returns_fallback_when_api_key_missing - Test that function returns fallback models when API key is not configured
test_fetches_models_from_openai_api - Test that function fetches models from OpenAI-compatible API
test_returns_fallback_on_api_error - Test that function returns fallback models when API call fails
class TestFetchImageModelsFromNovitaSDK - Test fetch_image_models_from_novita_sdk function
test_returns_none_when_sdk_not_available - Test that function returns None when SDK is not installed
test_returns_none_when_client_initialization_fails - Test that function returns None when client initialization fails
test_fetches_models_using_sdk - Test that function fetches models using SDK's models_v3 method
test_returns_none_on_sdk_error - Test that function returns None when SDK call fails
class TestGenerateImageWithNovitaSDK - Test generate_image_with_novita_sdk function
test_raises_import_error_when_sdk_not_available - Test that function raises ImportError when SDK is not installed
test_raises_value_error_when_client_initialization_fails - Test that function raises ValueError when client initialization fails
test_generates_image_with_default_parameters - Test that function generates image with default parameters
test_generates_image_with_custom_parameters - Test that function generates image with custom parameters
test_raises_error_on_generation_failure - Test that function raises error when image generation fails
class TestDefaultModels - Test default Novita models configuration
test_default_models_are_defined - Test that default models list is defined
test_default_models_have_required_fields - Test that default models have required fields
class TestHelperFunctions - Test helper functions
test_cleanup_model_id_removes_prefixes - Test that _cleanup_model_id removes various prefixes
test_normalize_pricing_handles_none - Test that _normalize_pricing handles None input
test_normalize_pricing_extracts_values - Test that _normalize_pricing extracts pricing values
class TestModuleDocumentation - Test module and function documentation
test_module_has_docstring - Test that module has proper docstring
test_functions_have_docstrings - Test that all main functions have docstrings
References: src/cache.py, src/services/onerouter_client.py
Module: Unit tests for OneRouter client
class TestGetOneRouterClient - Test get_onerouter_client function
test_get_onerouter_client_success - Test successful client initialization
test_get_onerouter_client_missing_key - Test client initialization with missing API key
class TestMakeOneRouterRequest - Test make_onerouter_request_openai function
test_make_onerouter_request_openai_forwards_args - Test that request forwards all arguments correctly
test_make_onerouter_request_openai_error - Test request error handling
class TestMakeOneRouterRequestStream - Test make_onerouter_request_openai_stream function
test_make_onerouter_request_openai_stream - Test streaming request
test_make_onerouter_request_openai_stream_error - Test streaming request error handling
class TestProcessOneRouterResponse - Test process_onerouter_response function
test_process_onerouter_response_happy - Test processing a normal response
test_process_onerouter_response_no_usage - Test processing response without usage data
test_process_onerouter_response_error - Test processing error handling
class TestParseTokenLimit - Test _parse_token_limit helper function
test_parse_token_limit_int - Test parsing integer value
test_parse_token_limit_string - Test parsing string value
test_parse_token_limit_string_with_commas - Test parsing string value with commas
test_parse_token_limit_none - Test parsing None value
test_parse_token_limit_invalid_string - Test parsing invalid string values returns default
test_parse_token_limit_float - Test parsing float values
class TestParsePricing - Test _parse_pricing helper function
test_parse_pricing_with_dollar_sign - Test parsing pricing with $ sign
test_parse_pricing_without_dollar_sign - Test parsing pricing without $ sign
test_parse_pricing_zero - Test parsing zero pricing
test_parse_pricing_none - Test parsing None value
test_parse_pricing_with_commas - Test parsing pricing with commas
class TestFetchModelsFromOneRouter - Test fetch_models_from_onerouter function with caching
test_fetch_models_success_with_caching_and_pricing - Test successful model fetch with pricing enrichment from display_models
test_fetch_models_context_length_priority - Test that context_length is prioritized over context_window
test_fetch_models_skip_empty_model_id - Test that models without id are skipped
test_fetch_models_http_error_with_caching - Test HTTP error handling and verify cache is still updated
test_fetch_models_generic_error_with_caching - Test generic error handling and verify cache is updated
test_fetch_models_uses_correct_endpoints - Test that fetch_models_from_onerouter calls both /v1/models and display_models
test_fetch_models_missing_api_key - Test that fetch returns empty list when API key is not configured
References: src/services/openai_client.py
Module: Tests for OpenAI client
class TestOpenAIClient - Test OpenAI client functionality
test_get_openai_client - Test getting OpenAI client
test_get_openai_client_no_key - Test getting OpenAI client without API key
test_make_openai_request - Test making request to OpenAI
test_make_openai_request_with_kwargs - Test making request to OpenAI with additional parameters
test_make_openai_request_error - Test handling errors from OpenAI
test_make_openai_request_stream - Test making streaming request to OpenAI
test_make_openai_request_stream_with_kwargs - Test making streaming request to OpenAI with additional parameters
test_make_openai_request_stream_error - Test handling streaming errors from OpenAI
test_process_openai_response - Test processing OpenAI response
test_process_openai_response_no_usage - Test processing OpenAI response without usage data
test_process_openai_response_multiple_choices - Test processing OpenAI response with multiple choices
References: src/services/openrouter_client.py
test_get_openrouter_client_success
test_get_openrouter_client_missing_key_raises
test_make_openrouter_request_openai_forwards_args
test_process_openrouter_response_happy
test_process_openrouter_response_no_usage
class TestMergeExtraBody - Tests for the _merge_extra_body helper that ensures data_collection: allow is set.
test_merge_empty_kwargs - Empty kwargs should get extra_body with provider settings.
test_merge_preserves_other_kwargs - Other kwargs like temperature should be preserved.
test_merge_preserves_existing_extra_body - Existing extra_body values should be preserved.
test_merge_preserves_existing_provider_settings - Existing provider settings should be preserved and merged.
test_user_can_override_data_collection - User-provided data_collection should override the default.
class TestNormalizeMessageRoles - Tests for the _normalize_message_roles helper that transforms developer role to system.
test_normalize_empty_messages - Empty messages list should return empty list.
test_normalize_preserves_standard_roles - Standard roles (user, assistant, system, tool) should be preserved.
test_normalize_transforms_developer_to_system - Developer role should be transformed to system role.
test_normalize_preserves_other_message_fields - Other message fields should be preserved when transforming developer role.
test_normalize_multiple_developer_messages - Multiple developer messages should all be transformed.
test_normalize_handles_non_dict_messages - Non-dict messages should be passed through unchanged.
test_make_openrouter_request_normalizes_developer_role - Verify that make_openrouter_request_openai normalizes developer role to system.
test_make_openrouter_request_stream_normalizes_developer_role - Verify that make_openrouter_request_openai_stream normalizes developer role to system.
test_make_openrouter_request_stream_async_normalizes_developer_role - Verify that make_openrouter_request_openai_stream_async normalizes developer role to system.
References: src/services/partner_trial_service.py
Module: Tests for Partner Trial Service
class TestPartnerTrialService - Tests for PartnerTrialService class
test_get_partner_config_valid_partner - Test fetching valid partner configuration
test_get_partner_config_case_insensitive - Test that partner code lookup is case insensitive
test_get_partner_config_not_found - Test fetching non-existent partner
test_get_partner_config_caching - Test that partner configs are cached
test_is_partner_code_true - Test is_partner_code returns True for valid partner
test_is_partner_code_false - Test is_partner_code returns False for invalid partner
test_is_partner_code_empty - Test is_partner_code returns False for empty string
test_start_partner_trial_success - Test starting a partner trial successfully
test_start_partner_trial_invalid_partner - Test starting trial with invalid partner code raises error
test_get_user_daily_limit_partner_user - Test getting daily limit for partner trial user
test_get_user_daily_limit_standard_user - Test getting daily limit for standard trial user
test_get_user_daily_limit_paid_subscriber - Test getting daily limit for paid subscriber (unlimited)
test_get_partner_trial_status - Test getting partner trial status
test_get_partner_trial_status_no_trial - Test getting status when user has no partner trial
test_get_partner_analytics - Test getting partner analytics
test_cache_invalidation - Test cache invalidation
test_cache_invalidation_all - Test invalidating entire cache
References: src/config/supabase_config.py, src/services/payments.py
Module: Tests for payment failure tier downgrade functionality
class TestPaymentFailureDowngrade - Test tier downgrade on payment failure
test_invoice_payment_failed_downgrades_tier - Test that invoice payment failure downgrades user to basic tier
test_invoice_payment_failed_for_max_tier - Test that MAX tier is also downgraded on payment failure
test_invoice_payment_failed_non_subscription - Test that non-subscription invoices are skipped
test_payment_failure_updates_both_tables - Test that payment failure updates both users and api_keys_new tables
test_payment_failure_logs_warning - Test that payment failure logs appropriate warning
test_payment_failure_handles_database_error - Test that database errors during payment failure are handled
References: src/config/supabase_config.py, src/schemas/payments.py, src/services/payments.py
Module: Comprehensive payment processing tests - CRITICAL
class TestStripeServiceInitialization - Test StripeService initialization
test_init_success - Test successful initialization
test_init_missing_api_key - Test initialization fails without API key
class TestCheckoutSession - Test checkout session creation
test_create_checkout_session_success - Test successful checkout session creation
test_create_checkout_session_with_discounted_credit_value - Test checkout session with discounted credit_value (e.g., $9 for $10 credits)
test_create_checkout_session_large_discounted_package - Test checkout session with larger discounted package ($75 for $100 credits)
test_create_checkout_session_persists_payment_intent - Ensure checkout session stores payment_intent when provided by Stripe.
test_create_checkout_session_user_not_found - Test checkout session fails for non-existent user
test_create_checkout_session_stripe_error - Test checkout session handles Stripe errors
test_create_checkout_session_with_privy_did - Test checkout session handles Privy DID emails
class TestPaymentIntents - Test payment intent creation
test_create_payment_intent_success - Test successful payment intent creation
test_create_payment_intent_with_specific_payment_methods - Test payment intent with specific payment methods
class TestWebhooks - Test webhook processing
test_handle_checkout_completed_webhook - Test checkout.session.completed webhook
test_handle_webhook_invalid_signature - Test webhook with invalid signature
test_checkout_completed_adds_credits - Test checkout completed webhook adds credits to user
test_checkout_completed_refetches_metadata_when_missing - Ensure checkout handler refetches the session when metadata is missing
test_checkout_completed_recovers_metadata_from_payment_intent - Verify fallback metadata retrieval pulls from the related payment intent.
test_checkout_completed_hydrates_metadata_from_payment_intent - Ensure metadata can be recovered from the PaymentIntent when absent on the session.
test_checkout_completed_raises_when_metadata_and_id_missing - Ensure handler fails fast when both metadata and session id are missing
test_checkout_completed_recovers_missing_metadata - Ensure webhook handler falls back to Supabase when metadata is absent.
test_checkout_completed_recovers_via_session_id_when_intent_lookup_fails - Ensure fallback lookup retries with the checkout session ID.
test_payment_intent_succeeded_webhook - Test payment_intent.succeeded webhook
test_payment_intent_failed_webhook - Test payment_intent.payment_failed webhook
class TestCreditPackages - Test credit package listings
test_get_credit_packages - Test get credit packages
class TestRefunds - Test refund processing
test_create_refund_success - Test successful refund creation
test_create_refund_stripe_error - Test refund handles Stripe errors
class TestSessionRetrieval - Test session and intent retrieval
test_retrieve_checkout_session - Test retrieve checkout session
test_retrieve_payment_intent - Test retrieve payment intent
class TestPaymentIntegration - Integration tests for complete payment flows
test_complete_payment_flow - Test complete payment flow: create session → webhook → credits added
test_checkout_completed_missing_credits_uses_amount_total - Ensure handler falls back to session amount when credits metadata is missing
test_checkout_completed_missing_ids_uses_payment_lookup - Ensure handler can recover user/payment IDs from Supabase when metadata is incomplete
test_checkout_completed_creates_fallback_payment_when_missing_metadata - Ensure handler creates a fallback payment record when payment_id cannot be recovered.
class TestCheckoutCompletedSubscriptionStatus - Test that checkout completed sets subscription_status correctly
test_checkout_completed_sets_inactive_status_for_trial_user - Test that checkout completed sets subscription_status to 'inactive' for trial users.
test_checkout_completed_preserves_active_subscription_status - Test that checkout completed does NOT change subscription_status for users with active subscriptions.
test_checkout_completed_sets_inactive_for_expired_trial_user - Test that checkout completed sets subscription_status to 'inactive' for expired trial users.
References: src/services/payments.py
Module: Test decimal precision in payment credit calculations
test_decimal_credit_calculation - Test that Decimal provides precise credit calculations.
test_decimal_precision_edge_cases - Test edge cases where floating-point precision matters.
test_decimal_preserves_precision - Test that Decimal preserves precision better than float.
test_credit_value_conversion_matches_schema - Test that credit value conversion matches expected schema behavior.
test_large_credit_values - Test that large credit values are handled correctly.
test_fractional_cents_handled - Test that fractional cents are handled correctly.
References: src/services.py, src/services/posthog_service.py
Module: Comprehensive tests for Posthog Service service
class TestPosthogService - Test Posthog Service service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
test_initialize_with_exception_autocapture - Test that PostHog initializes with exception autocapture enabled
test_capture_exception_success - Test successful exception capture
test_capture_exception_default_distinct_id - Test exception capture with default distinct_id
test_capture_exception_when_not_initialized - Test that capture_exception handles uninitialized client gracefully
References: src/services/models.py, src/services/pricing.py
test_get_model_pricing_found_by_id
test_get_model_pricing_found_by_slug
test_get_model_pricing_model_not_found_uses_default
test_get_model_pricing_empty_cache_uses_default
test_get_model_pricing_handles_missing_prices
test_get_model_pricing_exception_returns_default
test_get_model_pricing_normalizes_hf_suffix - Test that HuggingFace :hf-inference suffix is stripped for pricing lookup
test_get_model_pricing_handles_multiple_provider_suffixes - Test that various provider suffixes are normalized
test_calculate_cost_happy
test_calculate_cost_zero_tokens
test_calculate_cost_uses_fallback_on_exception
test_calculate_cost_free_model_returns_zero - Test that models ending with :free return $0 cost
test_calculate_cost_free_model_openrouter_format - Test that OpenRouter free models (ending with :free) return $0 cost
test_calculate_cost_free_model_with_zero_tokens - Test that free models with zero tokens return $0
test_calculate_cost_non_free_model_normal_pricing - Test that non-free models are charged normally
test_calculate_cost_free_model_fallback_on_exception - Test that free models return $0 even in fallback/exception case
test_calculate_cost_free_suffix_case_sensitive - Test that :free suffix detection is case sensitive (lowercase only)
test_calculate_cost_multiple_free_models - Test various free model formats
References: src/services/pricing.py, src/services/pricing_lookup.py, src/services/pricing_normalization.py
Module: Comprehensive Pricing Accuracy Tests
class TestProviderAPIFormats - Test that we correctly understand provider API pricing formats
test_openrouter_api_returns_per_token_pricing - CRITICAL TEST: Verify OpenRouter returns per-token pricing, not per-1M
test_manual_pricing_format - Test that manual_pricing.json uses per-1M format
class TestPricingNormalization - Test pricing normalization functions
test_normalize_per_1m_to_per_token - Test normalization from per-1M tokens to per-token
test_normalize_per_1k_to_per_token - Test normalization from per-1K tokens to per-token
test_normalize_already_per_token - Test that per-token pricing passes through unchanged
test_normalize_pricing_dict - Test normalizing a full pricing dictionary
test_validate_normalized_price - Test price validation
class TestProviderFormatMapping - Test that provider format mappings are correct
test_openrouter_format_should_be_per_token - CRITICAL: OpenRouter format should be PER_TOKEN, not PER_1M_TOKENS
test_deepinfra_format_mapping - DeepInfra uses per-1M tokens pricing
test_aihubmix_format_mapping - AiHubMix uses per-1K tokens pricing
class TestCreditCalculations - Test that credit deductions are calculated correctly
test_calculate_cost_with_correct_pricing - Test credit calculation with known pricing
test_calculate_cost_with_wrong_normalization - Test showing the impact of wrong normalization
class TestEndToEndPricing - Test complete pricing flow from API to credit deduction
test_gpt4o_mini_pricing_end_to_end - Integration test: Verify GPT-4o-mini pricing end-to-end
class TestPricingConsistency - Test that pricing is consistent across different code paths
test_openrouter_pricing_not_double_normalized - Ensure OpenRouter pricing isn't normalized twice
class TestKnownModelPricing - Test pricing for known models against public pricing pages
test_known_model_pricing - Test that known models have correct pricing (within 20% tolerance)
References: src/services/models.py, src/services/pricing.py
Module: Pricing Coverage Validation Tests
class TestGetPricingCoverageReport - Tests for the get_pricing_coverage_report() utility function.
test_empty_model_list - An empty list should return 100% coverage with zero counts.
test_all_models_covered - When every model has real pricing, coverage should be 100%.
test_no_models_covered - When no model has real pricing, coverage should be 0%.
test_partial_coverage - When some models are covered and some are not, report reflects both.
test_uncovered_models_sorted - Uncovered models list should be sorted alphabetically.
test_high_value_model_without_pricing_counted_as_uncovered - High-value models (GPT-4, Claude, etc.) that raise ValueError when
class TestCatalogPricingCoverage - Warning-level tests that check pricing coverage for a representative
test_sample_catalog_coverage_report - Run the coverage report over the sample catalog and warn about gaps.
test_high_value_models_have_pricing - High-value models (OpenAI, Anthropic, Google) should always have
test_individual_model_pricing_not_default - Per-model check: warns if an individual model falls back to default pricing.
References: src/config/supabase_config.py, src/services/models.py, src/services/pricing.py
Module: Unit tests for database-based pricing lookup system
class TestDatabasePricingLookup - Test database pricing queries
test_get_pricing_from_database_success - Test successful database query returns pricing
test_get_pricing_from_database_not_found - Test database query returns None when model not found
test_get_pricing_from_database_handles_error - Test database query handles exceptions gracefully
class TestPricingCache - Test pricing cache functionality
test_cache_stores_and_retrieves_pricing - Test cache stores and retrieves pricing correctly
test_cache_expiration - Test cache expires after TTL
test_clear_pricing_cache - Test clearing pricing cache
test_get_pricing_cache_stats - Test cache statistics
class TestFallbackMechanism - Test fallback to provider API cache
test_fallback_to_cache_on_database_failure - Test fallback to provider API cache when database fails
test_fallback_to_default_when_all_fail - Test fallback to default pricing when both database and cache fail
test_database_takes_priority_over_cache - Test database is queried before falling back to cache
class TestModelIDNormalization - Test model ID normalization and alias resolution
test_handles_provider_suffixes - Test stripping provider-specific suffixes
test_free_model_detection - Test free models return $0 cost
References: src/services.py, src/services/pricing_lookup.py
Module: Comprehensive tests for Pricing Lookup service
class TestPricingLookup - Test Pricing Lookup service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
class TestEnrichModelWithPricing - Test enrich_model_with_pricing function
test_enriches_model_with_zero_pricing - Models with zero pricing should be enriched with manual pricing
test_skips_enrichment_for_non_zero_pricing - Models with non-zero pricing should not be enriched
test_handles_scientific_notation_pricing - Models with scientific notation pricing should not be enriched
test_handles_float_zero_pricing - Models with float 0.0 pricing should be enriched
test_handles_string_zero_variations - Models with various zero string formats should be enriched
test_handles_no_pricing_field - Models without pricing field should be enriched
test_handles_empty_pricing - Models with empty pricing dict should be enriched
test_handles_none_pricing_values - Models with None pricing values should be enriched
test_handles_empty_string_pricing - Models with empty string pricing values should be enriched
test_handles_missing_model_id - Models without id should be returned unchanged
test_handles_no_manual_pricing_available - Models without manual pricing should keep original pricing
test_handles_mixed_zero_and_nonzero_pricing - Models with some non-zero values should not be enriched
test_handles_invalid_pricing_values - Models with invalid pricing values should be treated as zero
class TestGetModelPricing - Test get_model_pricing function
test_returns_pricing_for_existing_model - Should return pricing for model in manual pricing data
test_returns_none_for_missing_gateway - Should return None for unknown gateway
test_returns_none_for_missing_model - Should return None for unknown model
test_case_insensitive_gateway_match - Should match gateway case-insensitively
test_case_insensitive_model_match - Should match model case-insensitively
test_returns_none_for_empty_pricing_data - Should return None when no pricing data loaded
class TestGatewayProviders - Test GATEWAY_PROVIDERS constant
test_gateway_providers_contains_expected_providers - Test that GATEWAY_PROVIDERS contains all expected gateway providers
test_gateway_providers_is_set - Test that GATEWAY_PROVIDERS is a set for O(1) lookup
class TestCrossReferencePricing - Test cross-reference pricing functionality
test_cross_reference_pricing_returns_none_when_catalog_building - Test that cross-reference returns None when catalog is building
test_cross_reference_pricing_returns_none_when_no_openrouter_models - Test that cross-reference returns None when OpenRouter cache is empty
test_cross_reference_pricing_finds_matching_model - Test that cross-reference finds pricing for matching model
test_cross_reference_pricing_handles_provider_prefix - Test that cross-reference works with provider/model format
test_cross_reference_pricing_returns_none_for_unknown_model - Test that cross-reference returns None for models not in OpenRouter
test_cross_reference_pricing_handles_versioned_model_ids - Test that cross-reference matches versioned OpenRouter model IDs
test_cross_reference_pricing_does_not_match_different_model_variants - Test that cross-reference does NOT match different model variants incorrectly
test_cross_reference_pricing_matches_correct_model_with_variants - Test that cross-reference matches the correct model when variants exist
class TestEnrichModelWithPricingGatewayProviders - Test enrich_model_with_pricing for gateway providers
test_gateway_provider_uses_cross_reference_pricing - Gateway provider should use cross-reference pricing when no manual pricing
test_gateway_provider_returns_none_when_no_pricing_and_not_building - Gateway provider should return None when no pricing found and not building catalog
test_gateway_provider_keeps_model_during_catalog_build - Gateway provider should keep model with zero pricing during catalog build
test_gateway_provider_prefers_manual_pricing - Gateway provider should prefer manual pricing over cross-reference
test_non_gateway_provider_returns_model_without_pricing - Non-gateway provider should return model unchanged when no pricing found
test_gateway_provider_with_existing_non_zero_pricing - Gateway provider with non-zero pricing should not be enriched
test_gateway_provider_filters_on_exception - Gateway provider should return None on exception to prevent appearing free
test_non_gateway_provider_returns_model_on_exception - Non-gateway provider should return model data on exception
class TestGatewayProviderZeroPricingFiltering - Test that gateway providers filter out models with zero cross-reference pricing
test_gateway_provider_filters_zero_cross_reference_pricing - Gateway provider should filter out models with zero cross-reference pricing
test_gateway_provider_accepts_nonzero_cross_reference_pricing - Gateway provider should accept models with non-zero cross-reference pricing
test_gateway_provider_filters_zero_string_variants - Gateway provider should filter zero pricing in various formats
test_gateway_provider_accepts_partial_nonzero_pricing - Gateway provider should accept models with at least one non-zero prompt/completion
test_clarifai_in_gateway_providers - Test that clarifai is now in GATEWAY_PROVIDERS
test_onerouter_in_gateway_providers - Test that onerouter is now in GATEWAY_PROVIDERS
test_deepinfra_in_gateway_providers - Test that deepinfra is now in GATEWAY_PROVIDERS
test_featherless_in_gateway_providers - Test that featherless is now in GATEWAY_PROVIDERS
test_deepinfra_filters_models_without_pricing - Test that deepinfra filters out models without valid pricing
test_featherless_filters_models_without_pricing - Test that featherless filters out models without valid pricing
test_deepinfra_accepts_models_with_manual_pricing - Test that deepinfra accepts models with manual pricing
test_featherless_accepts_models_with_manual_pricing - Test that featherless accepts models with manual pricing
test_groq_in_gateway_providers - Test that groq is now in GATEWAY_PROVIDERS
test_fireworks_in_gateway_providers - Test that fireworks is now in GATEWAY_PROVIDERS
test_together_in_gateway_providers - Test that together is now in GATEWAY_PROVIDERS
test_akash_in_gateway_providers - Test that akash is now in GATEWAY_PROVIDERS
test_cloudflare_workers_ai_in_gateway_providers - Test that cloudflare-workers-ai is now in GATEWAY_PROVIDERS
test_groq_filters_models_without_pricing - Test that groq filters out models without valid pricing
test_fireworks_filters_models_without_pricing - Test that fireworks filters out models without valid pricing
test_together_filters_models_without_pricing - Test that together filters out models without valid pricing
test_groq_accepts_models_with_manual_pricing - Test that groq accepts models with manual pricing
test_fireworks_accepts_models_with_manual_pricing - Test that fireworks accepts models with manual pricing
class TestCrossReferencePricingNullHandling - Test null/None value handling in cross-reference pricing
test_cross_reference_handles_none_pricing_values - Cross-reference should handle None pricing values correctly
test_cross_reference_handles_empty_string_pricing - Cross-reference should handle empty string pricing values correctly
class TestIsFreeField - Test is_free field is set correctly for models
test_non_openrouter_gateway_sets_is_free_false - Non-OpenRouter gateways should have is_free set to False
test_deepinfra_sets_is_free_false - DeepInfra models should have is_free set to False
test_featherless_sets_is_free_false - Featherless models should have is_free set to False
test_together_sets_is_free_false - Together models should have is_free set to False
test_fireworks_sets_is_free_false - Fireworks models should have is_free set to False
test_openrouter_does_not_set_is_free - OpenRouter models should not have is_free set by enrich_model_with_pricing
test_aihubmix_sets_is_free_false - AiHubMix gateway models should have is_free set to False
test_helicone_sets_is_free_false - Helicone gateway models should have is_free set to False
References: src/services/pricing_normalization.py
Module: Tests for pricing normalization utilities
class TestNormalizeToPerToken - Test normalize_to_per_token function
test_normalize_per_1m_to_per_token - Test normalization from per-1M format
test_normalize_per_1k_to_per_token - Test normalization from per-1K format
test_normalize_already_per_token - Test normalization when already per-token
test_normalize_negative_price - Test that negative prices (dynamic) return None
test_normalize_zero_price - Test that zero prices are handled
test_normalize_none_price - Test that None returns None
test_normalize_empty_string - Test that empty string returns None
test_normalize_string_price - Test normalization with string input
test_normalize_decimal_price - Test normalization with Decimal input
test_normalize_large_price - Test normalization with large price (expensive model)
test_normalize_very_small_price - Test normalization with very small price (cheap model)
class TestNormalizePricingDict - Test normalize_pricing_dict function
test_normalize_full_pricing_dict - Test normalizing full pricing dictionary
test_normalize_partial_pricing_dict - Test normalizing dict with missing fields
test_normalize_empty_pricing_dict - Test normalizing empty dict
test_normalize_none_pricing_dict - Test normalizing None (should handle gracefully)
class TestProviderFormats - Test provider format mappings
test_get_openrouter_format - OpenRouter uses per-1M format
test_get_deepinfra_format - DeepInfra uses per-1M format
test_get_aihubmix_format - AiHubMix uses per-1K format
test_get_unknown_provider_format - Unknown providers default to per-1M
test_provider_format_case_insensitive - Provider format lookup is case-insensitive
class TestAutoDetectFormat - Test auto-detection of pricing format
test_detect_per_token - Detect per-token format (very small values)
test_detect_per_1k - Detect per-1K format (medium values)
test_detect_per_1m - Detect per-1M format (large values)
test_detect_boundary_values - Test detection at boundary values
class TestConvertBetweenFormats - Test conversion between different formats
test_convert_1m_to_token - Convert from per-1M to per-token
test_convert_1k_to_token - Convert from per-1K to per-token
test_convert_token_to_1m - Convert from per-token to per-1M
test_convert_1k_to_1m - Convert from per-1K to per-1M
class TestValidateNormalizedPrice - Test price validation
test_validate_correct_per_token - Valid per-token prices should pass
test_validate_incorrect_per_token - Prices that are too large should fail
class TestNormalizePriceFromProvider - Test convenience function for provider-specific normalization
test_normalize_from_deepinfra - DeepInfra uses per-1M format
test_normalize_from_aihubmix - AiHubMix uses per-1K format
test_normalize_from_openrouter - OpenRouter uses per-1M format
class TestCostCalculations - Test that cost calculations are accurate
test_llama_3_1_8b_cost - Test cost calculation for Llama-3.1-8B (known pricing)
test_gpt4_cost - Test cost calculation for GPT-4 (known pricing)
test_mixed_input_output_cost - Test cost with different input/output pricing
test_large_request_cost - Test cost calculation for large request
class TestEdgeCases - Test edge cases and error handling
test_invalid_price_string - Invalid price strings should return None
test_scientific_notation - Test handling of scientific notation
test_very_large_number - Test handling of very large numbers
test_very_small_number - Test handling of very small numbers
class TestRealWorldScenarios - Test real-world pricing scenarios
test_typical_openrouter_model - Test typical OpenRouter model pricing
test_typical_aihubmix_model - Test typical AiHubMix model pricing
References: src/services/google_models_config.py, src/services/pricing_normalization.py, src/services/pricing_validation.py
Module: Pricing Validation Tests
class TestGoogleModelsPricing - Test Google models pricing configuration.
test_all_models_have_pricing - Test that all Google models have pricing configured.
test_pricing_format_not_per_million - Test that pricing is in per-1K format, not per-1M format.
test_google_pricing_matches_official - Test that Google models match official pricing.
test_gemma_models_are_free - Test that Gemma models are free.
test_output_price_higher_than_input - Test that output pricing is higher than input pricing.
test_pricing_reasonable_range - Test that pricing is in a reasonable range.
test_per_token_format_conversion - Test that when converted to per-token format, prices are very small.
class TestPricingNormalization - Test pricing normalization logic.
test_per_1k_to_per_token_conversion - Test conversion from per-1K to per-token format.
test_per_1m_to_per_token_conversion - Test conversion from per-1M to per-token format.
test_google_pricing_conversion_examples - Test specific Google pricing conversion examples.
class TestPriceBoundsValidation - Test price bounds validation from Issue #1038
test_valid_price_within_bounds - Test that valid prices pass validation
test_price_below_minimum_rejected - Test that prices below minimum are rejected
test_price_above_maximum_rejected - Test that prices above maximum are rejected
test_zero_price_valid_with_warning - Test that zero pricing is valid but triggers warning
test_unusually_low_price_warning - Test that unusually low prices trigger warnings
class TestPriceSpikeDetection - Test price spike detection from Issue #1038
test_small_price_change_valid - Test that small price changes are valid
test_large_price_spike_rejected - Test that large price spikes are rejected
test_spike_detection_with_zero_old_price - Test spike detection when old price is zero
class TestComprehensivePricingValidation - Test comprehensive pricing update validation from Issue #1038
test_valid_pricing_update - Test validation of a valid pricing update
test_pricing_update_with_bounds_violation - Test pricing update with bounds violations
test_pricing_update_with_spike - Test pricing update with price spike
class TestValidationWithRealPricing - Test validation with real-world pricing examples
test_openai_gpt4o_pricing - Test validation with actual GPT-4o pricing
test_anthropic_claude_opus_pricing - Test validation with actual Claude Opus pricing
test_llama_8b_pricing - Test validation with actual Llama 3.1 8B pricing
References: src/services.py, src/services/professional_email_templates.py
Module: Comprehensive tests for Professional Email Templates service
class TestProfessionalEmailTemplates - Test Professional Email Templates service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/services.py, src/services/prometheus_pb2.py, src/services/prometheus_remote_write.py
Module: Comprehensive tests for Prometheus Remote Write service
class TestPrometheusProtobuf - Test Prometheus protobuf message implementations
test_label_serialization - Test Label message serialization
test_label_empty_values - Test Label with empty values
test_sample_serialization - Test Sample message serialization
test_sample_zero_value - Test Sample with zero value but non-zero timestamp
test_timeseries_serialization - Test TimeSeries message serialization
test_write_request_serialization - Test WriteRequest message serialization
test_write_request_multiple_timeseries - Test WriteRequest with multiple timeseries
class TestPrometheusRemoteWrite - Test Prometheus Remote Write service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
test_prometheus_remote_writer_initialization - Test PrometheusRemoteWriter initialization
test_push_metrics_when_disabled - Test push_metrics returns False when disabled
test_push_metrics_when_no_client - Test push_metrics returns False when client is None
test_push_metrics_success - Test successful metrics push with protobuf
test_push_metrics_http_error - Test metrics push with HTTP error
test_get_stats - Test get_stats returns correct statistics
test_get_stats_no_pushes - Test get_stats with no pushes returns 0 success rate
test_circuit_breaker_initial_state - Test circuit breaker is initially closed
test_circuit_breaker_opens_after_threshold - Test circuit breaker opens after consecutive failures
test_circuit_breaker_resets_on_success - Test circuit breaker resets after successful push
test_circuit_breaker_closes_after_timeout - Test circuit breaker allows retry after timeout
test_push_metrics_skipped_when_circuit_open - Test push_metrics is skipped when circuit is open
test_circuit_breaker_opens_on_connection_errors - Test circuit breaker opens after repeated connection errors
test_circuit_breaker_opens_on_timeout_errors - Test circuit breaker opens after repeated timeout errors
test_init_prometheus_remote_write_disabled - Test init_prometheus_remote_write when Prometheus is disabled
test_init_prometheus_remote_write_enabled - Test init_prometheus_remote_write creates and starts writer
test_shutdown_prometheus_remote_write_with_writer - Test shutdown_prometheus_remote_write with active writer
test_shutdown_prometheus_remote_write_no_writer - Test shutdown_prometheus_remote_write with no writer
test_get_prometheus_writer - Test get_prometheus_writer returns the global instance
test_serialize_metrics_success - Test successful metrics serialization with new protobuf implementation
test_serialize_metrics_empty_registry - Test serialization with empty registry
test_get_instance_labels - Test _get_instance_labels returns expected labels
class TestVarintEncoding - Test varint encoding utility
test_encode_small_value - Test encoding small values (< 128)
test_encode_medium_value - Test encoding medium values (128-16383)
test_encode_large_value - Test encoding large values (timestamps)
test_encode_negative_value_raises - Test that encoding negative values raises ValueError
class TestLabelSorting - Test that labels are sorted lexicographically
test_labels_are_sorted - Test that labels in TimeSeries are sorted by name
test_serialize_metrics_sorts_labels - Test that _serialize_metrics_to_protobuf sorts labels correctly
References: src/schemas/router.py, src/services/capability_gating.py, src/services/fallback_chain.py, src/services/model_selector.py, src/services/prompt_classifier_rules.py, src/services/prompt_router.py
Module: Tests for Prompt Router components.
class TestCapabilityGating - Tests for capability extraction and filtering.
test_extract_capabilities_basic - Test basic capability extraction from messages.
test_extract_capabilities_with_tools - Test capability extraction with tools.
test_extract_capabilities_with_json_mode - Test capability extraction with JSON response format.
test_extract_capabilities_with_vision - Test capability extraction with image content.
test_filter_by_capabilities_tools - Test filtering models by tool capability.
test_filter_by_capabilities_context - Test filtering models by context length.
test_filter_by_capabilities_cost - Test filtering models by cost limit.
class TestPromptClassifier - Tests for rule-based prompt classification.
test_classify_simple_question - Test classification of simple Q&A.
test_classify_code_generation - Test classification of code generation request.
test_classify_code_with_block - Test classification of message with code block.
test_classify_math - Test classification of math calculation.
test_classify_reasoning - Test classification of complex reasoning.
test_classify_summarization - Test classification of summarization request.
test_classify_translation - Test classification of translation request.
test_classify_multimodal_content - Test classification handles multimodal content.
test_classification_performance - Test classification completes within target time.
class TestModelSelector - Tests for model selection with stability.
test_select_model_basic - Test basic model selection.
test_select_model_price_optimization - Test model selection with price optimization.
test_select_model_quality_optimization - Test model selection with quality optimization.
test_select_model_stable_with_conversation_id - Test that same conversation ID produces stable selection.
test_select_model_excludes_models - Test that excluded models are not selected.
test_select_model_no_candidates - Test selection with no candidates returns default.
class TestFallbackChain - Tests for fallback chain building.
test_build_fallback_chain_basic - Test basic fallback chain building.
test_build_fallback_chain_provider_diversity - Test fallback chain prefers different providers.
test_build_fallback_chain_respects_capabilities - Test fallback chain only includes capable models.
test_build_fallback_chain_max_limit - Test fallback chain respects tier limit.
class TestPromptRouter - Tests for main router with fail-open behavior.
test_router_basic_routing - Test basic routing without auto mode.
test_router_fail_open_on_no_candidates - Test router fails open when no candidates available.
test_router_timeout_protection - Test router returns default on timeout.
test_router_with_tools - Test routing with tools requirement.
test_router_with_user_preferences - Test routing with user preferences.
class TestAutoRouteHelpers - Tests for auto-route helper functions.
test_is_auto_route_request - Test detection of auto-route requests.
test_parse_auto_route_options - Test parsing of auto-route options.
class TestPerformanceBenchmarks - Performance benchmarks for router components.
test_capability_extraction_performance - Benchmark capability extraction.
test_classification_performance - Benchmark prompt classification.
test_full_router_performance - Benchmark full router pipeline.
References: src/services.py, src/services/cerebras_client.py, src/services/google_vertex_client.py, src/services/groq_client.py
Module: Comprehensive tests for provider client modules.
class TestCerebrasClient - Test Cerebras client functionality
test_module_imports - Test that module imports successfully
test_default_models_defined - Test that default models are defined
test_default_models_have_required_fields - Test that default models have required fields
test_default_supported_parameters - Test default supported parameters
test_get_cerebras_client_no_api_key - Test client raises ValueError without API key
test_get_cerebras_client_fallback_info - Test client fallback mechanism is documented
test_normalize_cerebras_model - Test model normalization
test_normalize_cerebras_model_missing_id - Test normalization uses name as fallback ID when id is missing
test_cleanup_model_id - Test model ID cleanup
test_normalize_pricing - Test pricing normalization
test_extract_supported_parameters - Test parameter extraction
test_coerce_sequence - Test sequence coercion
test_fallback_cerebras_models - Test fallback to static models
class TestGroqClient - Test Groq client functionality
test_module_imports - Test that module imports successfully
test_get_groq_client_no_api_key - Test client fails without API key
test_get_groq_client_success - Test successful client initialization
test_make_groq_request_openai - Test Groq request making
test_make_groq_request_stream - Test Groq streaming request
test_process_groq_response - Test Groq response processing
class TestGoogleVertexClient - Test Google Vertex AI client functionality
test_module_imports - Test that module imports successfully
test_sanitize_system_content_string - Test system content sanitization with string
test_sanitize_system_content_list - Test system content sanitization with list
test_sanitize_system_content_other - Test system content sanitization with other types
test_prepare_vertex_environment_missing_project - Test environment preparation fails without project ID
class TestOpenRouterClient - Test OpenRouter client functionality
test_module_imports - Test that module imports successfully
class TestDeepInfraClient - Test DeepInfra client functionality
test_module_imports - Test that module imports successfully
class TestFireworksClient - Test Fireworks AI client functionality
test_module_imports - Test that module imports successfully
class TestTogetherClient - Test Together AI client functionality
test_module_imports - Test that module imports successfully
class TestXAIClient - Test XAI (Grok) client functionality
test_module_imports - Test that module imports successfully
class TestHuggingFaceClient - Test HuggingFace client functionality
test_module_imports - Test that module imports successfully
class TestNovitaClient - Test Novita client functionality
test_module_imports - Test that module imports successfully
class TestNebiusClient - Test Nebius client functionality
test_module_imports - Test that module imports successfully
class TestMorpheusClient - Test Morpheus client functionality
test_module_imports - Test that module imports successfully
class TestCloudflareWorkersAIClient - Test Cloudflare Workers AI client functionality
test_module_imports - Test that module imports successfully
class TestFeatherlessClient - Test Featherless client functionality
test_module_imports - Test that module imports successfully
class TestChutesClient - Test Chutes client functionality
test_module_imports - Test that module imports successfully
class TestAimoClient - Test AIMO client functionality
test_module_imports - Test that module imports successfully
class TestNearClient - Test Near AI client functionality
test_module_imports - Test that module imports successfully
class TestFalImageClient - Test Fal.ai image client functionality
test_module_imports - Test that module imports successfully
class TestImageGenerationClient - Test image generation router client
test_module_imports - Test that module imports successfully
class TestHeliconeClient - Test Helicone client functionality
test_module_imports - Test that module imports successfully
class TestAiHubMixClient - Test AiHubMix client functionality
test_module_imports - Test that module imports successfully
class TestAnannasClient - Test Anannas client functionality
test_module_imports - Test that module imports successfully
class TestVercelAIGatewayClient - Test Vercel AI Gateway client functionality
test_module_imports - Test that module imports successfully
class TestAlpacaNetworkClient - Test Alpaca Network client functionality
test_module_imports - Test that module imports successfully
class TestOneRouterClient - Test OneRouter client functionality
test_module_imports - Test that module imports successfully
class TestAkashClient - Test Akash client functionality
test_module_imports - Test that module imports successfully
class TestClarifaiClient - Test Clarifai client functionality
test_module_imports - Test that module imports successfully
class TestModelzClient - Test Modelz client functionality
test_module_imports - Test that module imports successfully
class TestAISDKClient - Test AI SDK client functionality
test_module_imports - Test that module imports successfully
class TestAlibabaCloudClient - Test Alibaba Cloud client functionality
test_module_imports - Test that module imports successfully
References: src/services/provider_credit_monitor.py
Module: Tests for provider credit monitoring service.
class TestCreditStatusDetermination - Test credit status determination logic
test_healthy_balance - Test that high balance returns healthy status
test_info_balance - Test that moderate balance returns info status
test_warning_balance - Test that low balance returns warning status
test_critical_balance - Test that very low balance returns critical status
class TestOpenRouterCreditCheck - Test OpenRouter credit balance checking
test_successful_credit_check - Test successful credit balance check
test_credit_check_with_warning_balance - Test credit check with low balance triggering warning
test_credit_check_with_critical_balance - Test credit check with very low balance triggering critical
test_credit_check_without_api_key - Test credit check fails gracefully without API key
test_credit_check_http_error - Test credit check handles HTTP errors
test_credit_check_caching - Test that credit checks are cached
class TestCheckAllProviderCredits - Test checking all provider credits
test_check_all_providers - Test checking all monitored providers
class TestLowCreditAlerts - Test low credit alerting system
test_send_critical_alert - Test sending critical credit alert
test_send_warning_alert - Test sending warning credit alert
class TestCreditCacheManagement - Test credit cache management
test_clear_specific_provider_cache - Test clearing cache for specific provider
test_clear_all_provider_cache - Test clearing all provider caches
References: src/services/provider_failover.py
Module: Comprehensive tests for provider failover logic - CRITICAL for reliability
class TestBuildProviderFailoverChain - Test provider failover chain construction
test_chain_with_huggingface_first - Test chain starting with huggingface
test_chain_with_openrouter_first - Test chain starting with openrouter
test_chain_with_featherless_first - Test chain starting with featherless
test_chain_with_none_provider - Test chain with None provider defaults to onerouter
test_chain_with_empty_string - Test chain with empty string defaults to onerouter
test_chain_with_unknown_provider - Test chain with unknown provider (not in fallback list)
test_chain_with_removed_provider - Test chain with removed provider (not eligible for fallback)
test_chain_case_insensitive - Test chain building is case insensitive
test_chain_no_duplicates - Test chain has no duplicate providers
test_fallback_provider_priority_constants - Test fallback provider constants are defined correctly
class TestEnforceModelFailoverRules - Test model-specific failover filtering
test_openrouter_prefix_locked
test_openrouter_suffix_locked
test_openai_prefix_routes_to_native_then_openrouter - Test that openai/* models route to native OpenAI first, then OpenRouter fallback.
test_openai_prefix_without_native_falls_back_to_openrouter - Test that openai/* models fall back to OpenRouter if native OpenAI not in chain.
test_anthropic_prefix_routes_to_native_then_openrouter - Test that anthropic/* models route to native Anthropic first, then OpenRouter fallback.
test_anthropic_prefix_without_native_falls_back_to_openrouter - Test that anthropic/* models fall back to OpenRouter if native Anthropic not in chain.
test_non_locked_model_noop
test_payment_failover_does_not_bypass_anthropic_restriction - Test that allow_payment_failover=True does NOT bypass anthropic/ restriction.
test_payment_failover_does_not_bypass_openai_restriction - Test that allow_payment_failover=True does NOT bypass openai/ restriction.
test_payment_failover_does_not_bypass_suffix_lock - Test that allow_payment_failover=True does NOT bypass suffix-based provider lock.
test_payment_failover_no_effect_on_unlocked_models - Test that allow_payment_failover has no effect on models without provider lock
test_bare_openai_model_names_route_to_native_first - Test that bare OpenAI model names route to native OpenAI first, then OpenRouter.
test_bare_openai_model_names_fallback_to_openrouter - Test that bare OpenAI model names fall back to OpenRouter if native not in chain.
test_bare_openai_model_names_keep_restriction_with_payment_failover - Test that bare OpenAI model names keep their restriction even with payment_failover=True.
test_bare_anthropic_model_names_route_to_native_first - Test that bare Anthropic/Claude model names route to native Anthropic first, then OpenRouter.
test_bare_anthropic_model_names_fallback_to_openrouter - Test that bare Anthropic model names fall back to OpenRouter if native not in chain.
test_bare_anthropic_model_names_keep_restriction_with_payment_failover - Test that bare Anthropic model names keep restriction with payment_failover=True.
class TestShouldFailover - Test failover eligibility detection
test_should_failover_401 - Test 401 Unauthorized triggers failover
test_should_failover_403 - Test 403 Forbidden triggers failover
test_should_failover_404 - Test 404 Not Found triggers failover
test_should_failover_429 - Test 429 Rate Limit does NOT trigger failover - client should retry
test_should_failover_502 - Test 502 Bad Gateway triggers failover
test_should_failover_503 - Test 503 Service Unavailable triggers failover
test_should_failover_504 - Test 504 Gateway Timeout triggers failover
test_should_not_failover_200 - Test 200 OK does not trigger failover
test_should_not_failover_400 - Test 400 Bad Request does not trigger failover
test_should_not_failover_500 - Test 500 Internal Server Error does not trigger failover
test_failover_status_codes_constant - Test FAILOVER_STATUS_CODES contains expected codes
test_should_failover_402 - Test 402 Payment Required triggers failover (e.g., provider credits exhausted)
class TestMapProviderErrorHTTPException - Test mapping existing HTTPException instances
test_map_http_exception_passthrough - Test HTTPException is passed through unchanged
test_map_value_error - Test ValueError is mapped to 400
class TestMapProviderErrorHTTPX - Test mapping httpx exceptions
test_map_httpx_timeout_exception - Test httpx.TimeoutException maps to 504
test_map_asyncio_timeout_error - Test asyncio.TimeoutError maps to 504
test_map_httpx_request_error - Test httpx.RequestError maps to 503
test_map_httpx_status_error_429_with_retry_after - Test httpx 429 error preserves Retry-After header
test_map_httpx_status_error_429_without_retry_after - Test httpx 429 error without Retry-After header
test_map_httpx_status_error_401 - Test httpx 401 error maps to 500 (internal auth issue)
test_map_httpx_status_error_404 - Test httpx 404 error indicates model not found
test_map_httpx_status_error_4xx - Test httpx 4xx errors map to 400
test_map_httpx_status_error_5xx - Test httpx 5xx errors map to 502
class TestMapProviderErrorOpenAI - Test mapping OpenAI SDK exceptions
test_map_api_connection_error - Test APIConnectionError maps to 503
test_map_api_timeout_error - Test APITimeoutError maps to 504
test_map_rate_limit_error_with_retry_after_header - Test RateLimitError with Retry-After in response headers
test_map_rate_limit_error_with_retry_after_body - Test RateLimitError with retry_after in body
test_map_authentication_error - Test AuthenticationError maps to 401
test_map_permission_denied_error - Test PermissionDeniedError maps to 401
test_map_not_found_error - Test NotFoundError indicates model not available
test_map_bad_request_error - Test BadRequestError maps to 400
test_map_generic_openai_error - Test generic OpenAIError maps to 502
test_map_api_status_error_with_custom_status - Test APIStatusError with custom status code
test_map_api_status_error_invalid_status - Test APIStatusError with invalid status code defaults to 500
test_map_api_status_error_404_generic - Test generic APIStatusError with 404 status provides proper error message
test_map_api_status_error_403_maps_to_401 - Test generic APIStatusError with 403 maps to 401 for consistency
class TestMapProviderErrorCerebras - Test mapping Cerebras SDK exceptions
test_map_cerebras_api_connection_error - Test Cerebras APIConnectionError maps to 503
test_map_cerebras_rate_limit_error_with_retry_after_header - Test Cerebras RateLimitError with Retry-After in response headers
test_map_cerebras_rate_limit_error_with_retry_after_body - Test Cerebras RateLimitError with retry_after in body
test_map_cerebras_authentication_error - Test Cerebras AuthenticationError maps to 401
test_map_cerebras_permission_denied_error - Test Cerebras PermissionDeniedError maps to 401
test_map_cerebras_not_found_error - Test Cerebras NotFoundError indicates model not available
test_map_cerebras_bad_request_error - Test Cerebras BadRequestError maps to 400
test_map_cerebras_api_status_error_with_custom_status - Test Cerebras APIStatusError with custom status code
test_map_cerebras_api_status_error_invalid_status - Test Cerebras APIStatusError with invalid status code defaults to 500
test_map_cerebras_api_status_error_5xx - Test Cerebras APIStatusError with 5xx status maps to service error
test_map_cerebras_api_status_error_403_generic - Test generic Cerebras APIStatusError with 403 maps to 401
test_map_cerebras_api_status_error_404_generic - Test generic Cerebras APIStatusError with 404 provides proper error message
class TestMapProviderErrorCerebras - Test mapping Cerebras SDK exceptions
test_map_cerebras_api_connection_error - Test Cerebras APIConnectionError maps to 503
test_map_cerebras_rate_limit_error_with_retry_after_header - Test Cerebras RateLimitError with Retry-After in response headers
test_map_cerebras_rate_limit_error_with_retry_after_body - Test Cerebras RateLimitError with retry_after in body
test_map_cerebras_authentication_error - Test Cerebras AuthenticationError maps to 401
test_map_cerebras_permission_denied_error - Test Cerebras PermissionDeniedError maps to 401
test_map_cerebras_not_found_error - Test Cerebras NotFoundError indicates model not available
test_map_cerebras_bad_request_error - Test Cerebras BadRequestError maps to 400
test_map_cerebras_api_status_error_with_custom_status - Test Cerebras APIStatusError with custom status code
test_map_cerebras_api_status_error_invalid_status - Test Cerebras APIStatusError with invalid status code defaults to 500
test_map_cerebras_api_status_error_5xx - Test Cerebras APIStatusError with 5xx status maps to service error
test_map_cerebras_api_status_error_403_generic - Test generic Cerebras APIStatusError with 403 maps to 401
test_map_cerebras_api_status_error_404_generic - Test generic Cerebras APIStatusError with 404 provides proper error message
class TestMapProviderErrorGeneric - Test mapping generic exceptions
test_map_generic_exception - Test generic Exception maps to 502
test_map_runtime_error - Test RuntimeError maps to 502
test_map_key_error - Test KeyError maps to 502
class TestProviderFailoverIntegration - Test provider failover integration scenarios
test_complete_failover_chain_all_providers - Test complete failover chain with all providers
test_failover_decision_matrix - Test failover decisions for various error scenarios
test_error_mapping_preserves_provider_context - Test error messages include provider and model context
class TestMapProviderErrorNoCandidates - Test that 'no candidates' errors from Vertex AI trigger failover
test_no_candidates_error_triggers_failover - Test 'no candidates' ValueError maps to 503 for failover
test_block_reason_error_triggers_failover - Test that block reason errors map to 503
test_prompt_feedback_error_triggers_failover - Test that promptFeedback errors map to 503
test_regular_value_error_does_not_trigger_failover - Test that regular ValueErrors map to 400 (no failover)
test_503_is_in_failover_status_codes - Verify that 503 is in the set of status codes that trigger failover
test_no_candidates_from_other_provider - Test 'no candidates' detection works for any provider, not just google-vertex
References: src/services.py, src/services/provider_selector.py
Module: Comprehensive tests for Provider Selector service
class TestProviderSelector - Test Provider Selector service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/schemas/providers.py, src/services.py, src/services/providers.py
Module: Comprehensive tests for Providers service
class TestProviders - Test Providers service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
References: src/services/query_classifier.py
Module: Tests for the Query Classifier Service.
class TestNormalizeText - Tests for text normalization.
test_lowercase - Test that text is lowercased.
test_strip_whitespace - Test that whitespace is stripped.
test_combined - Test combined normalization.
class TestContainsKeywords - Tests for keyword detection.
test_single_keyword_found - Test detection of single keyword.
test_multiple_keywords_found - Test detection of multiple keywords.
test_no_keywords_found - Test when no keywords are found.
test_case_insensitive - Test case-insensitive matching.
class TestMatchesPatterns - Tests for pattern matching.
test_factual_question_pattern - Test factual question pattern detection.
test_what_is_the_best_pattern - Test 'what is the best' pattern.
test_code_pattern_function - Test code pattern detection for functions.
test_code_pattern_import - Test code pattern detection for imports.
test_no_pattern_match - Test when no patterns match.
class TestIsCodeQuery - Tests for code query detection.
test_code_block - Test detection of code block.
test_python_function - Test detection of Python function.
test_javascript_function - Test detection of JavaScript function.
test_import_statement - Test detection of import statement.
test_not_code - Test non-code query.
class TestExtractUserQuery - Tests for extracting user query from messages.
test_single_user_message - Test extraction from single user message.
test_multiple_messages - Test extraction from conversation with multiple messages.
test_multimodal_content - Test extraction from multimodal content.
test_no_user_message - Test when there's no user message.
test_empty_messages - Test with empty messages list.
class TestClassifyQuery - Tests for query classification.
test_wifi_travel_query_el_salvador - Test classification of wifi/travel query for El Salvador.
test_wifi_travel_query_thailand - Test classification of wifi/travel query for Thailand.
test_cost_of_living_query - Test classification of cost of living query.
test_current_events_query - Test classification of current events query.
test_price_query - Test classification of price query.
test_weather_query - Test classification of weather query.
test_code_query_no_search - Test that code queries don't trigger search.
test_import_statement_no_search - Test that import statements don't trigger search.
test_simple_greeting_no_search - Test that simple greetings don't trigger search.
test_general_knowledge_low_confidence - Test general knowledge questions have lower confidence.
test_empty_messages - Test handling of empty messages.
test_threshold_adjustment - Test that threshold affects search decision.
class TestShouldAutoSearch - Tests for the should_auto_search function.
test_enabled_returns_result - Test that enabled=True processes the query.
test_disabled_returns_false - Test that enabled=False always returns False.
test_remote_work_query - Test remote work related query triggers search.
class TestRealWorldQueries - Integration tests with real-world query examples.
test_query_classification - Test classification of various real-world queries.
class TestKeywordSets - Tests to verify keyword sets are properly defined.
test_current_info_keywords_not_empty - Test that current info keywords set is not empty.
test_location_keywords_not_empty - Test that location keywords set is not empty.
test_travel_destinations_not_empty - Test that travel destinations set is not empty.
test_all_keywords_lowercase - Test that all keywords are lowercase.
class TestEdgeCases - Tests for edge cases and boundary conditions.
test_very_long_query - Test handling of very long queries.
test_unicode_characters - Test handling of unicode characters.
test_mixed_content_messages - Test handling of mixed content in conversation.
test_none_content - Test handling of None content.
test_empty_string_content - Test handling of empty string content.
References: src/services/rate_limiting.py, src/services/user_lookup_cache.py, src/utils/security_validators.py
test_check_rate_limit_happy_path_local
test_concurrency_limit_exceeded - Test concurrency limit - currently disabled in source code (see rate_limiting.py lines 94-104)
test_burst_limit_local
test_minute_request_limit_local
test_minute_token_limit_local
test_fail_open_on_fallback_exception
test_manager_get_key_config_from_db
test_manager_get_key_config_default_on_error
test_manager_update_key_config_calls_db
test_get_rate_limiter_singleton
test_get_rate_limit_manager_singleton
test_concurrency_increment_decrement_helpers
test_severe_rate_limit_configs_exist - Test that SEVERE_RATE_LIMIT_CONFIG and BLOCKED_ACCOUNT_CONFIG exist and have correct values
test_severe_rate_limit_for_temporary_email - Test that temporary email domains get SEVERE rate limiting
test_blocked_rate_limit_for_blocked_email_domain - Test that blocked email domains get BLOCKED rate limiting (most restrictive)
test_no_severe_rate_limit_for_normal_user - Test that normal users don't get severe rate limiting
test_severe_rate_limit_graceful_on_missing_user - Test that severe rate limit check handles missing user gracefully
test_severe_rate_limit_graceful_on_exception - Test that severe rate limit check handles exceptions gracefully
test_blocked_email_domain_check_priority - Test that blocked domains take priority over temporary domains
References: src/config/supabase_config.py, src/routes/catalog.py, src/services/local_memory_cache.py, src/services/model_catalog_cache.py
Module: Tests for rebuild_full_catalog_from_providers()
class TestRebuildFullCatalogFromProviders - Tests for rebuild_full_catalog_from_providers().
test_assembles_all_providers - All provider models are merged into a single catalog.
test_caches_result_in_redis_and_local - Assembled catalog is cached in both Redis and local memory.
test_partial_provider_failure - If some providers fail, others still contribute to the catalog.
test_all_providers_empty_returns_empty - If every provider returns empty, result is empty and nothing is cached.
test_empty_provider_list_returns_empty - If no provider slugs are discovered, return empty immediately.
test_db_discovery_failure_falls_back_to_registry - If DB slug query fails, falls back to GATEWAY_REGISTRY keys.
test_providers_with_zero_models_handled_gracefully - Providers returning 0 models don't break the assembly.
test_does_not_use_provider_slugs_with_overrides - Fallback uses GATEWAY_REGISTRY.keys() not PROVIDER_SLUGS
class TestRebuildIntegrationWithGetCachedFullCatalog - Test that get_cached_full_catalog() correctly calls rebuild on cache miss.
test_cache_hit_does_not_trigger_rebuild - Redis cache hit returns immediately without rebuilding.
test_cache_miss_triggers_rebuild - Cache miss triggers rebuild_full_catalog_from_providers.
References: src/config/redis_config.py, src/services/auth_cache.py, src/services/db_cache.py, src/services/model_catalog_cache.py
Module: Basic integration tests for Redis caching functionality.
class TestAuthCacheBasic - Basic tests for authentication caching
test_cache_and_retrieve_user_by_api_key - Test caching user data by API key
test_cache_miss - Test cache miss returns None
test_cache_invalidation - Test cache invalidation works
test_cache_user_by_id - Test caching user by ID
class TestDBCacheBasic - Basic tests for database query caching
test_db_cache_set_and_get - Test basic cache set and get operations
test_db_cache_invalidate - Test cache invalidation
test_convenience_functions - Test convenience functions work
test_cache_stats - Test cache statistics work
class TestModelCatalogCache - Basic tests for model catalog caching
test_cache_full_catalog - Test caching full model catalog
test_cache_provider_catalog - Test caching provider-specific catalog
test_invalidate_full_catalog - Test invalidating full catalog
class TestCacheWithRedisUnavailable - Tests for graceful degradation when Redis is unavailable
test_cache_fails_gracefully_when_redis_unavailable - Test that cache operations don't crash when Redis is down
test_db_cache_fails_gracefully - Test DB cache fails gracefully without Redis
class TestCacheIntegrationWithDB - Integration tests with actual database operations
test_user_caching_integration - Test full user caching flow with DB
test_cache_invalidation_on_credit_update - Test cache invalidates when credits are updated
test_redis_availability_check - Test that we can check Redis availability
References: src/services/redis_metrics.py
Module: Tests for Redis metrics service.
class TestRecordRequest - Test request recording functionality
test_record_successful_request - Test recording a successful request
test_record_failed_request - Test recording a failed request with error
test_record_request_disabled - Test recording when Redis is disabled
test_record_request_exception_handling - Test exception handling during recording
class TestProviderHealth - Test provider health tracking
test_get_provider_health - Test getting provider health score
test_get_provider_health_no_data - Test getting health score when no data exists
test_get_all_provider_health - Test getting all provider health scores
class TestRecentErrors - Test error tracking
test_get_recent_errors - Test getting recent errors
test_get_recent_errors_disabled - Test getting errors when Redis is disabled
class TestHourlyStats - Test hourly statistics
test_get_hourly_stats - Test getting hourly statistics
test_get_hourly_stats_disabled - Test getting stats when Redis is disabled
class TestLatencyPercentiles - Test latency percentile calculations
test_get_latency_percentiles - Test calculating latency percentiles
test_get_latency_percentiles_no_data - Test percentiles when no data exists
test_get_latency_percentiles_disabled - Test percentiles when Redis is disabled
class TestCircuitBreaker - Test circuit breaker state management
test_update_circuit_breaker - Test updating circuit breaker state
test_update_circuit_breaker_disabled - Test updating circuit breaker when Redis is disabled
class TestCleanup - Test old data cleanup
test_cleanup_old_data - Test cleaning up old Redis data
test_cleanup_old_data_disabled - Test cleanup when Redis is disabled
class TestRequestMetricsDataclass - Test RequestMetrics dataclass
test_request_metrics_creation - Test creating RequestMetrics instance
test_request_metrics_with_error - Test creating RequestMetrics with error
References: src/services.py, src/services/request_prioritization.py
Module: Comprehensive tests for Request Prioritization service
class TestRequestPrioritization - Test Request Prioritization service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
class TestLowLatencyModels - Tests for low-latency model configuration and routing
test_ultra_low_latency_models_defined - Test that ultra-low-latency models are defined
test_low_latency_models_defined - Test that low-latency models are defined
test_ultra_low_latency_subset_of_low_latency - Test that ultra-low-latency models are included in low-latency set
test_is_low_latency_model - Test is_low_latency_model function
test_is_ultra_low_latency_model - Test is_ultra_low_latency_model function
test_get_low_latency_models - Test get_low_latency_models returns sorted list
test_get_ultra_low_latency_models - Test get_ultra_low_latency_models returns sorted list
class TestProviderLatencyTiers - Tests for provider latency tier configuration
test_provider_latency_tiers_defined - Test that provider latency tiers are defined
test_groq_is_tier_1 - Test that Groq is classified as tier 1 (fastest)
test_cerebras_is_tier_1 - Test that Cerebras is classified as tier 1 (fastest)
test_fireworks_is_tier_2 - Test that Fireworks is classified as tier 2
test_get_provider_latency_tier - Test get_provider_latency_tier function
test_get_fastest_providers - Test get_fastest_providers returns sorted list
class TestLowLatencyAlternatives - Tests for low-latency alternative suggestions
test_suggest_low_latency_alternative_for_claude - Test suggesting alternatives for Claude models
test_suggest_low_latency_alternative_for_gpt - Test suggesting alternatives for GPT models
test_suggest_low_latency_alternative_for_reasoning_model - Test suggesting alternatives for reasoning models
test_suggest_low_latency_alternative_for_unknown - Test suggesting alternatives for unknown models
class TestProviderSelection - Tests for provider selection based on priority
test_high_priority_gets_fastest_first - Test that high priority requests get fastest providers first
test_low_priority_gets_slower_first - Test that low priority requests get slower providers first
test_all_providers_included - Test that all available providers are included in result
test_empty_providers_list - Test handling of empty providers list
References: src/services/response_cache.py
Module: Tests for Response Cache Service
class TestCacheKeyGeneration - Test cache key generation
test_generate_cache_key_from_request - Generate cache key from request parameters
test_same_request_same_key - Same request should generate same cache key
test_different_request_different_key - Different requests should generate different keys
test_cache_key_ignores_irrelevant_params - Cache key should ignore irrelevant parameters
class TestCacheOperations - Test cache operations
test_cache_miss - Cache miss returns None
test_cache_hit - Cache hit returns cached response
test_cache_set_and_get - Set and get cache entry
class TestCacheTTL - Test cache TTL (Time-To-Live)
test_cache_expires_after_ttl - Cache entry should expire after TTL
test_cache_valid_within_ttl - Cache entry should be valid within TTL
test_custom_ttl - Support custom TTL values
class TestCacheInvalidation - Test cache invalidation
test_invalidate_single_entry - Invalidate single cache entry
test_clear_all_cache - Clear all cache entries
class TestCachingStrategies - Test different caching strategies
test_streaming_responses_not_cached - Streaming responses should not be cached
test_error_responses_not_cached - Error responses should not be cached
test_successful_responses_cached - Successful responses should be cached
test_cache_respects_max_tokens - Different max_tokens should create different cache keys
test_cache_respects_temperature - Different temperature should create different cache keys
class TestCachePerformance - Test cache performance characteristics
test_cache_reduces_latency - Cache hit should be faster than API call
test_cache_size_limited - Cache size should be limited to prevent memory issues
test_cache_eviction_lru - Least Recently Used (LRU) eviction strategy
class TestCacheMonitoring - Test cache monitoring and metrics
test_cache_hit_ratio_calculation - Calculate cache hit ratio
test_cache_memory_usage - Track cache memory usage
test_cache_metrics_exposed - Cache metrics should be exposed via API
class TestCacheEdgeCases - Test edge cases
test_cache_null_response - Cache should handle null responses
test_cache_large_response - Cache should handle large responses
test_cache_key_collision_handling - Handle potential cache key collisions
class TestCacheConfiguration - Test cache configuration
test_default_ttl_configuration - Default TTL should be configurable
test_max_cache_size_configuration - Max cache size should be configurable
test_cache_enabled_configuration - Cache should be enable/disable via configuration
References: src/db/roles.py, src/routes/roles.py, src/services/roles.py
Module: Tests for role-based access control service
class TestRequireRole - Test role requirement function
test_require_role_admin_success - Test admin accessing admin resource
test_require_role_developer_success - Test developer accessing developer resource
test_require_role_user_success - Test user accessing user resource
test_require_role_hierarchy_admin_to_dev - Test admin can access developer resources
test_require_role_hierarchy_admin_to_user - Test admin can access user resources
test_require_role_hierarchy_dev_to_user - Test developer can access user resources
test_require_role_user_cannot_access_dev - Test user cannot access developer resources
test_require_role_user_cannot_access_admin - Test user cannot access admin resources
test_require_role_dev_cannot_access_admin - Test developer cannot access admin resources
test_require_role_default_to_user - Test missing role defaults to user level
test_require_role_unknown_role - Test unknown role is rejected
class TestRequireAdmin - Test admin requirement shortcut
test_require_admin_success - Test admin can access admin-only resource
test_require_admin_developer_rejected - Test developer cannot access admin-only resource
test_require_admin_user_rejected - Test user cannot access admin-only resource
class TestRequireDeveloper - Test developer requirement shortcut
test_require_developer_success - Test developer can access developer resource
test_require_developer_admin_success - Test admin can access developer resource
test_require_developer_user_rejected - Test user cannot access developer resource
class TestRequirePermission - Test permission requirement
test_require_permission_granted - Test permission granted
test_require_permission_denied - Test permission denied
test_require_permission_different_resources - Test permission checking for different resources
class TestCreatePermissionChecker - Test permission checker factory
test_create_permission_checker_success - Test permission checker factory
test_create_permission_checker_denied - Test permission checker factory with denied permission
test_create_permission_checker_multiple_resources - Test creating multiple permission checkers
References: src/services/simplismart_client.py
Module: Tests for Simplismart client
class TestSimplismartClient - Test Simplismart client functionality
test_get_simplismart_client - Test getting Simplismart client
test_get_simplismart_client_no_key - Test getting Simplismart client without API key
test_make_simplismart_request_openai - Test making request to Simplismart
test_make_simplismart_request_openai_stream - Test making streaming request to Simplismart
test_process_simplismart_response - Test processing Simplismart response
test_process_simplismart_response_no_usage - Test processing response when usage is not available
class TestSimplismartModelResolution - Test model ID resolution for Simplismart
test_resolve_alias_llama_3_1_8b - Test resolving llama-3.1-8b alias
test_resolve_alias_llama_3_1_70b - Test resolving llama-3.1-70b alias
test_resolve_alias_llama_3_3_70b - Test resolving llama-3.3-70b alias
test_resolve_alias_gemma - Test resolving gemma aliases
test_resolve_alias_qwen - Test resolving qwen aliases
test_resolve_alias_deepseek - Test resolving deepseek aliases
test_resolve_alias_mixtral - Test resolving mixtral aliases
test_resolve_alias_devstral - Test resolving devstral aliases
test_resolve_full_model_id - Test that full model IDs are returned as-is
test_resolve_unknown_model - Test that unknown models are returned as-is
test_resolve_case_insensitive - Test that alias resolution is case insensitive
class TestSimplismartModelCatalog - Test Simplismart model catalog functions
test_fetch_models_from_simplismart - Test fetching models from Simplismart catalog
test_fetch_models_has_llama_models - Test that catalog includes Llama models
test_fetch_models_has_gemma_models - Test that catalog includes Gemma models
test_fetch_models_has_qwen_models - Test that catalog includes Qwen models
test_fetch_models_has_deepseek_models - Test that catalog includes DeepSeek models
class TestIsSimplismartModel - Test is_simplismart_model function
test_is_simplismart_model_with_alias - Test checking models with aliases
test_is_simplismart_model_with_full_id - Test checking models with full IDs
test_is_simplismart_model_case_insensitive - Test case insensitivity
test_is_simplismart_model_unknown - Test that unknown models return False
class TestSimplismartConstants - Test Simplismart constants
test_base_url - Test that base URL is correct
test_model_catalog_not_empty - Test that model catalog is not empty
test_model_aliases_not_empty - Test that model aliases are not empty
test_all_aliases_resolve_to_catalog_models - Test that all aliases resolve to models in the catalog
test_models_have_required_metadata - Test that all models have required metadata
test_models_have_pricing - Test that all models have pricing information
class TestSimplismartPricing - Test Simplismart pricing functionality
test_fetch_models_includes_pricing - Test that fetched models include pricing data
test_pricing_values_match_source - Test specific pricing values from https://simplismart.ai/pricing
test_new_models_present - Test that new models from pricing page are present
test_new_aliases_work - Test that new model aliases resolve correctly
class TestSimplismartImageModels - Test Simplismart image/diffusion model functionality
test_fetch_models_has_flux_models - Test that catalog includes Flux image models
test_fetch_models_has_sdxl - Test that catalog includes SDXL
test_image_models_have_type - Test that image models have correct type
test_image_models_have_per_image_pricing - Test that image models have per_image pricing model
test_image_model_pricing_values - Test specific image model pricing from simplismart.ai/pricing
test_flux_aliases_resolve_correctly - Test Flux image model aliases
class TestSimplismartSpeechModels - Test Simplismart speech-to-text model functionality
test_fetch_models_has_whisper_models - Test that catalog includes Whisper models
test_whisper_models_have_type - Test that Whisper models have correct type
test_whisper_models_have_per_minute_pricing - Test that Whisper models have per_minute pricing model
test_whisper_model_pricing_values - Test specific Whisper model pricing from simplismart.ai/pricing
test_whisper_aliases_resolve_correctly - Test Whisper model aliases
class TestSimplismartIsModelWithNewTypes - Test is_simplismart_model function with new model types
test_is_simplismart_model_flux - Test checking Flux image models
test_is_simplismart_model_whisper - Test checking Whisper speech models
References: src/config/supabase_config.py, src/services.py, src/services/startup.py
Module: Comprehensive tests for Startup service
class TestStartup - Test Startup service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
test_lifespan_startup_success - Test successful lifespan startup
test_lifespan_startup_critical_env_vars_missing - Test lifespan startup fails with missing critical env vars
test_lifespan_startup_supabase_init_fails - Test lifespan startup continues in degraded mode when Supabase init fails
test_lifespan_startup_continues_if_fal_cache_fails - Test lifespan startup continues even if optional Fal cache fails
test_initialize_services - Test initialize_services function
test_shutdown_services - Test shutdown_services function
test_shutdown_services_handles_failure - Test shutdown_services handles failures gracefully
References: src/services.py, src/services/statsig_service.py
Module: Comprehensive tests for Statsig Service service
class TestStatsigService - Test Statsig Service service functionality
test_module_imports - Test that module imports successfully
test_module_has_expected_attributes - Test module exports
test_statsig_service_class_exists - Test StatsigService class is importable
test_statsig_service_singleton_exists - Test statsig_service singleton is created
test_statsig_service_has_required_methods - Test StatsigService has all required methods
test_log_event_returns_true_in_fallback_mode - Test log_event returns True in fallback mode (no server key)
test_flush_returns_true_when_not_enabled - Test flush returns True when service is not enabled
test_get_feature_flag_returns_default_when_not_enabled - Test get_feature_flag returns default value when not enabled
class TestStatsigServiceInitialization - Test Statsig Service initialization behavior
test_initialize_without_server_key - Test initialization gracefully handles missing server key
test_initialize_with_missing_sdk - Test initialization handles missing statsig_python_core package
class TestStatsigServiceShutdown - Test Statsig Service shutdown behavior
test_shutdown_when_not_enabled - Test shutdown works when service is not enabled
test_shutdown_calls_wait - Test shutdown calls .wait() on the statsig SDK
class TestStatsigServiceFlush - Test Statsig Service flush behavior
test_flush_when_not_enabled - Test flush returns True when service is not enabled
test_flush_calls_wait_when_flush_exists - Test flush calls .wait() on the SDK flush method
test_flush_handles_exception_gracefully - Test flush returns False when exception occurs
class TestStatsigServiceBatchingConfig - Test Statsig Service batching configuration
test_initialization_sets_batching_options - Test that initialization configures event batching options
class TestStatsigServiceSessionTracking - Test Statsig Service session tracking for DAU/WAU/MAU
test_log_session_start_in_fallback_mode - Test log_session_start returns True in fallback mode
test_log_session_start_includes_platform - Test log_session_start includes platform in metadata
test_log_session_end_in_fallback_mode - Test log_session_end returns True in fallback mode
test_log_session_end_includes_duration - Test log_session_end includes duration in metadata
References: src/services/stream_normalizer.py
class TestStreamNormalizer
test_initialization
test_openai_format
test_gemini_format
test_anthropic_format
test_reasoning_extraction
test_reasoning_details_array_extraction - Test Grok 4.1 reasoning_details array format
test_reasoning_details_string_array_extraction - Test reasoning_details as array of strings
test_reasoning_details_text_field - Test reasoning_details with 'text' field instead of 'content'
test_anthropic_thinking_delta - Test Anthropic extended thinking delta events (Claude Sonnet 4, etc.)
test_anthropic_text_delta - Test Anthropic text_delta events with type field
test_anthropic_signature_delta - Test Anthropic signature delta events for extended thinking verification
test_anthropic_content_block_start_thinking - Test Anthropic content_block_start event with thinking type
test_anthropic_content_block_start_text - Test Anthropic content_block_start event with text type
test_anthropic_message_delta_stop_reason - Test Anthropic message_delta event with stop_reason
test_anthropic_message_stop - Test Anthropic message_stop event
test_anthropic_tool_input_delta - Test Anthropic input_json_delta for tool use streaming
test_anthropic_ping_ignored - Test that Anthropic ping events are ignored
test_anthropic_message_start_ignored - Test that Anthropic message_start events are ignored
test_anthropic_content_block_stop_ignored - Test that Anthropic content_block_stop events are ignored
test_anthropic_full_extended_thinking_flow - Test a complete extended thinking flow with multiple events
test_finish_reason_normalization
test_object_chunk
References: src/config/supabase_config.py, src/db/credit_transactions.py, src/db/plans.py, src/db/subscription_products.py, src/db/users.py, src/schemas/payments.py, src/services/payments.py
Module: Tests for Subscription Management Service
class TestGetCurrentSubscription - Tests for get_current_subscription method
test_get_subscription_with_active_subscription - Test getting subscription for user with active subscription
test_get_subscription_without_subscription - Test getting subscription for user without subscription
test_get_subscription_user_not_found - Test getting subscription for non-existent user
class TestUpgradeSubscription - Tests for upgrade_subscription method
test_upgrade_invalid_product_id_returns_basic - Test upgrade fails when product ID resolves to basic tier
test_upgrade_invalid_product_id_returns_none - Test upgrade fails when product ID resolves to None
test_upgrade_allowance_reset_failure - Test upgrade fails when allowance reset fails
test_upgrade_logs_credit_transaction - Test that upgrade logs audit trail via credit transaction
test_upgrade_pro_to_max - Test upgrading from Pro to Max tier
test_upgrade_without_subscription - Test upgrading when user has no subscription
test_upgrade_inactive_subscription - Test upgrading when subscription is not active
class TestDowngradeSubscription - Tests for downgrade_subscription method
test_downgrade_invalid_product_id_returns_basic - Test downgrade fails when product ID resolves to basic tier
test_downgrade_invalid_product_id_returns_none - Test downgrade fails when product ID resolves to None
test_downgrade_allowance_reset_failure - Test downgrade fails when allowance reset fails
test_downgrade_logs_credit_transaction - Test that downgrade logs audit trail via credit transaction
test_downgrade_max_to_pro - Test downgrading from Max to Pro tier
test_downgrade_without_subscription - Test downgrading when user has no subscription
class TestCancelSubscription - Tests for cancel_subscription method
test_cancel_at_period_end - Test canceling subscription at end of billing period
test_cancel_immediately - Test canceling subscription immediately
test_cancel_without_subscription - Test canceling when user has no subscription
test_cancel_already_canceled_subscription - Test canceling already canceled subscription
class TestTransactionTypes - Tests for subscription-related transaction types
test_subscription_upgrade_transaction_type_exists - Test that SUBSCRIPTION_UPGRADE transaction type exists
test_subscription_downgrade_transaction_type_exists - Test that SUBSCRIPTION_DOWNGRADE transaction type exists
class TestSubscriptionManagementSchemas - Tests for subscription management request/response schemas
test_upgrade_request_validation - Test UpgradeSubscriptionRequest validation
test_downgrade_request_validation - Test DowngradeSubscriptionRequest validation
test_cancel_request_defaults - Test CancelSubscriptionRequest defaults
References: src/config/supabase_config.py, src/services/payments.py
Module: Test subscription usage credits allocation
class TestSubscriptionUsageCreditsAllocation - Test that users receive correct usage credits when subscribing
test_pro_subscription_receives_20_dollar_credits - Test that PRO subscription gets $20 credits per month
test_max_subscription_receives_150_dollar_credits - Test that MAX subscription gets $150 credits per month
test_pro_subscription_initial_payment - Test PRO subscription initial payment adds credits immediately
test_max_subscription_initial_payment - Test MAX subscription initial payment adds credits immediately
test_pro_subscription_monthly_renewal - Test PRO subscription renewal adds $20 credits each month
test_max_subscription_monthly_renewal - Test MAX subscription renewal adds $150 credits each month
test_non_subscription_invoice_skipped - Test that non-subscription invoices don't add credits
test_unknown_tier_no_credits - Test that unknown tiers don't receive credits
class TestSubscriptionCreditsIntegration - Integration tests for complete subscription flow with credits
test_pro_subscription_created_then_first_invoice_paid - Test complete flow: subscription created -> first invoice paid -> credits added
test_max_subscription_created_then_first_invoice_paid - Test complete flow: MAX subscription created -> first invoice paid -> credits added
test_multiple_users_different_tiers - Test multiple users with different subscription tiers receive correct credits
class TestSubscriptionCreditsEdgeCases - Test edge cases and error scenarios
test_missing_user_id_in_subscription_metadata - Test handling when user_id is missing from subscription metadata
test_tier_defaults_to_pro_when_missing - Test that tier defaults to 'pro' when not specified in metadata
test_case_sensitivity_of_tier - Test that tier matching is case-sensitive (lowercase expected)
test_subscription_retrieve_fails - Test handling when Stripe API fails to retrieve subscription
test_add_credits_fails_gracefully - Test that errors in add_credits_to_user are propagated
class TestTrialStatusClearedOnSubscription - Test that trial status is cleared when user subscribes
test_subscription_created_clears_trial_status - Test that creating a subscription clears is_trial flag on API keys
test_subscription_updated_to_active_clears_trial_status - Test that updating subscription to active clears is_trial flag
test_subscription_updated_to_canceled_does_not_clear_trial - Test that canceling subscription does not clear trial status
References: src/config/supabase_config.py, src/db/plans.py, src/db/users.py, src/services/payments.py
Module: Tests for Stripe subscription webhook handlers
class TestSubscriptionCreatedWebhook - Test customer.subscription.created webhook handler
test_subscription_created_creates_user_plan_entry - Test that subscription.created webhook creates user_plans entry
test_subscription_created_deactivates_old_plans - Test that subscription.created deactivates existing user plans
test_subscription_created_handles_missing_plan - Test that subscription.created handles tier with no matching plan gracefully
class TestSubscriptionUpdatedWebhook - Test customer.subscription.updated webhook handler
test_subscription_updated_active_creates_plan_entry - Test that subscription.updated with status=active creates/updates user_plans
test_subscription_updated_past_due_does_not_create_plan - Test that subscription.updated with status=past_due doesn't create plan and downgrades tier
test_subscription_updated_tier_change_pro_to_max - Test upgrading from Pro to Max tier
class TestTierResolutionFromSubscriptionItems - Test tier resolution from subscription items when metadata is missing/basic
test_resolve_tier_from_subscription_metadata_present - Test that tier from metadata is used when present
test_resolve_tier_from_subscription_metadata_missing_defaults_to_pro_for_active - Test that active subscriptions default to 'pro' when tier can't be determined
test_resolve_tier_from_subscription_metadata_basic_defaults_to_pro_for_active - Test that 'basic' metadata is upgraded to 'pro' for active subscriptions
test_resolve_tier_from_subscription_items_with_product_lookup - Test tier resolution from subscription items when product_id is mapped
test_subscription_created_with_missing_tier_defaults_to_pro - Test that subscription.created with missing tier metadata defaults to 'pro'
test_subscription_created_with_basic_tier_defaults_to_pro - Test that subscription.created with tier='basic' upgrades to 'pro'
class TestCheckoutCompletedTrialStatusClearing - Test that checkout completed webhook clears trial status
test_checkout_completed_clears_trial_status - Test that checkout.session.completed clears trial status for users
test_checkout_completed_clears_multiple_api_keys - Test that checkout.session.completed clears trial status for all user's API keys
test_checkout_completed_does_not_fail_if_no_api_keys - Test that checkout completed doesn't fail if user has no API keys
class TestCacheInvalidationOnSubscriptionUpdates - Test that cache is invalidated when subscription status changes
test_subscription_created_invali
Reading Path (start here, in order)
- Conceptual Model
- Stability Definition
- Conceptual Model Features
- Features
- Delta Report
- Features-Acceptance-Criteria
Testing
Security & Access
Billing
Monitoring
Features
Providers
Operations
Data References