Skip to content

Test Mapping

Jump to bottom Edit New page
github-actions[bot] edited this page Mar 17, 2026 · 22 revisions

Test Mapping

Auto-generated from test files in tests/. Do not edit manually.

Generated: 2026-03-17 | Source: scripts/wiki/generate_test_mapping.py

Summary

Metric Count
Test files 334
Test classes 1192
Test functions 5603
Test directories 28

By Directory

Directory Files Classes Functions
tests/benchmarks 4 21 70
tests/billing 1 0 16
tests/conceptual_model 18 70 196
tests/config 7 27 122
tests/db 32 80 533
tests/e2e 8 17 125
tests/health 1 5 19
tests/integration 34 96 326
tests/middleware 3 17 65
tests/routes 61 237 958
tests/schemas 10 16 113
tests/security 6 22 113
tests/services 113 442 2213
tests/smoke 1 9 20
tests/test_admin_stats_growth_fix.py 1 3 9
tests/test_auth_timeout_fixes.py 1 4 17
tests/test_cache.py 1 11 54
tests/test_health_alerting.py 1 0 21
tests/test_huggingface_hub_integration.py 1 9 31
tests/test_intelligent_health_monitor.py 1 0 31
tests/test_jwt_id_token_unit.py 1 1 3
tests/test_metrics_parser.py 1 1 10
tests/test_observability_middleware.py 1 8 34
tests/test_prometheus_metrics.py 1 5 23
tests/test_schemas.py 1 16 67
tests/test_status_page.py 1 0 19
tests/unit 1 5 17
tests/utils 22 70 378

tests/benchmarks/

4 files, 70 tests

tests/benchmarks/test_benchmark_config.py

19 tests

  • TestModelConfig (4 tests)
    • test_soundsgood_config_values — Test that Soundsgood GLM config has correct values.
    • test_api_key_from_env — Test that API key is read from environment.
    • test_api_key_empty_when_not_set — Test that API key is empty when env var not set.
    • test_custom_model_config — Test creating a custom ModelConfig.
  • TestBenchmarkConfig (4 tests)
    • test_default_values — Test that default values are set correctly.
    • test_category_weights_sum_to_one — Test that category weights sum to 1.0.
    • test_tier_thresholds — Test tier threshold values.
    • test_custom_run_id — Test setting custom run ID.
  • TestTestCategory (2 tests)
    • test_all_categories_defined — Test that all expected categories are defined.
    • test_category_values — Test category string values.
  • TestDifficulty (1 tests)
    • test_all_difficulties_defined — Test that all difficulty levels are defined.
  • TestTestCase (2 tests)
    • test_create_test_case — Test creating a test case.
    • test_test_case_optional_fields — Test that optional fields default correctly.
  • TestTestResult (2 tests)
    • test_tokens_per_second_calculation — Test TPS calculation.
    • test_tokens_per_second_zero_duration — Test TPS with zero duration returns 0.
  • TestBaselineModels (4 tests)
    • test_baseline_tiers_exist — Test that all tier levels are defined.
    • test_tier_1_models — Test tier 1 model definitions.
    • test_tier_2_models — Test tier 2 model definitions.
    • test_baseline_score_ordering — Test that tier 1 models generally score higher than tier 4.

tests/benchmarks/test_latency_tracker.py

16 tests

  • TestLatencyMetrics (4 tests)
    • test_tokens_per_second — Test TPS calculation.
    • test_total_output_tokens — Test total output tokens including reasoning.
    • test_effective_tps — Test effective TPS including reasoning tokens.
    • test_zero_duration_tps — Test TPS with zero duration returns 0.
  • TestLatencyTracker (7 tests)
    • test_record_metrics — Test recording metrics.
    • test_record_from_response — Test recording from response data.
    • test_get_stats — Test getting aggregated stats.
    • test_get_stats_empty_raises — Test that getting stats with no data raises error.
    • test_percentile_calculation — Test percentile calculations.
    • test_session_duration — Test session duration tracking.
    • test_to_dict — Test exporting to dictionary.
  • TestCategoryLatencyTracker (3 tests)
    • test_track_by_category — Test tracking metrics by category.
    • test_get_all_stats — Test getting stats for all categories.
    • test_get_overall_stats — Test getting combined stats across categories.
  • TestFormatLatencyReport (2 tests)
    • test_format_report — Test formatting a latency report.
    • test_format_report_without_ttfc — Test formatting report without TTFC data.

tests/benchmarks/test_quality_evaluator.py

19 tests

  • TestQualityScore (2 tests)
    • test_create_quality_score — Test creating a quality score.
    • test_quality_score_optional_execution_result — Test that execution_result is optional.
  • TestQualityEvaluator (11 tests)
    • test_init_defaults — Test default initialization.
    • test_init_custom_values — Test initialization with custom values.
    • test_extract_code_fenced_python — Test extracting Python code from fenced block.
    • test_extract_code_generic_fenced — Test extracting code from generic fenced block.
    • test_extract_code_no_fence — Test extracting code without fence markers.
    • test_extract_code_empty — Test extracting from response with no code.
    • test_assess_code_quality_with_docstring — Test code quality assessment with docstring.
    • test_assess_code_quality_minimal — Test code quality assessment with minimal code.
    • test_parse_json_response_clean — Test parsing clean JSON response.
    • test_parse_json_response_with_markdown — Test parsing JSON with markdown code block.
    • test_parse_json_response_embedded — Test parsing JSON embedded in text.
  • TestCodeExecutionEvaluation (3 tests)
    • test_run_code_tests_success — Test running code tests that pass.
    • test_run_code_tests_failure — Test running code tests that fail.
    • test_run_code_tests_syntax_error — Test running code with syntax error.
  • TestEvaluateCodeGeneration (2 tests)
    • test_evaluate_code_gen_passed — Test evaluating passing code generation.
    • test_evaluate_code_gen_no_code — Test evaluating response with no extractable code.
  • TestIntegration (1 tests)
    • test_evaluate_with_mock_judge — Test evaluation with mocked judge call.

tests/benchmarks/test_soundsgood_client.py

16 tests

  • TestCompletionResponse (2 tests)
    • test_create_response — Test creating a completion response.
    • test_response_without_reasoning — Test response with no reasoning field.
  • TestStreamingMetrics (1 tests)
    • test_create_metrics — Test creating streaming metrics.
  • TestSoundsgoodClient (7 tests)
    • test_init — Test client initialization.
    • test_init_custom_timeout — Test client initialization with custom timeout.
    • test_headers — Test headers property.
    • test_context_manager — Test async context manager.
    • test_parse_response — Test parsing non-streaming response.
    • test_parse_response_no_choices — Test parsing response with no choices raises error.
    • test_parse_response_missing_fields — Test parsing response with missing optional fields.
  • TestStreamingParsing (3 tests)
    • test_sse_line_parsing — Test that SSE lines are properly identified.
    • test_parse_streaming_chunk — Test parsing a single streaming chunk.
    • test_parse_final_streaming_chunk — Test parsing final streaming chunk with usage.
  • TestHealthCheck (3 tests)
    • test_health_check_success — Test successful health check.
    • test_health_check_failure — Test failed health check.
    • test_health_check_empty_response — Test health check with empty response.

tests/billing/

1 files, 16 tests

tests/billing/test_billing_integrity.py

16 tests

  • test_deduct_credits_uses_atomic_rpc — deduct_credits should try atomic_deduct_credits RPC as primary path.
  • test_legacy_path_rolls_back_on_log_failure — Legacy path must roll back balance when transaction log fails.
  • test_chat_imports_pricing_for_precheck — chat.py must import get_model_pricing_async for pre-inference validation.
  • test_pricing_precheck_before_provider_dispatch — Pricing pre-check must appear BEFORE provider dispatch in chat.py.
  • test_pricing_precheck_returns_422 — Pricing pre-check failure must return 422, not 500.
  • test_non_streaming_deduction_only_after_provider_success — Credit deduction in non-streaming path must only happen AFTER
  • test_streaming_background_task_only_after_done_event — In the streaming generator, create_task(_process_stream_completion_background)
  • test_reconciliation_table_exists — credit_deduction_failures table must exist for manual reconciliation.
  • test_refund_transaction_type_exists — TransactionType.REFUND must exist for manual admin refunds.
  • test_trial_duration_matches_config
  • test_trial_credits_is_5_dollars
  • test_start_trial_default_matches_config — start_trial_for_key() default trial_days must match TRIAL_DURATION_DAYS.
  • test_api_keys_uses_config_constants — api_keys.py should import from usage_limits, not hardcode values.
  • test_free_model_zero_cost — Free models (:free suffix) always cost $0.
  • test_zero_tokens_skip_logic_exists — deduct_credits must skip deduction for near-zero amounts.
  • test_atomic_rpc_migration_exists — The atomic_deduct_credits migration must exist.

tests/conceptual_model/

18 files, 196 tests

tests/conceptual_model/test_cm01_auth_api_key_security.py

19 tests

  • TestCM01ApiKeyEncryptedWithFernet (1 tests)
    • test_api_key_encrypted_with_fernet — encrypt_api_key output is a valid Fernet token, not the plaintext key.
  • TestCM02ApiKeyDecryptionRoundtrip (1 tests)
    • test_api_key_decryption_roundtrip — decrypt(encrypt(key)) must return the original plaintext.
  • TestCM03ApiKeyHmacSha256Hashing (1 tests)
    • test_api_key_hmac_sha256_hashing — hash_api_key produces a hex SHA-256 HMAC, is deterministic, and
  • TestCM04HmacLookupWithoutDecryption (2 tests)
    • test_hmac_lookup_without_decryption — CM-1.4: get_api_key_by_hash() performs a DB lookup by key_hash
    • test_hmac_lookup_returns_none_when_not_found — CM-1.4: get_api_key_by_hash() returns None when no key matches.
  • TestCM05EncryptedKeyNotPlaintextInDb (1 tests)
    • test_encrypted_key_not_plaintext_in_db — create_api_key must store an encrypted_key that differs from the
  • TestCM06RbacThreeTiersExist (1 tests)
    • test_rbac_three_tiers_exist — CM-1.6: Three RBAC tiers (admin, developer, user) are enforced by
  • TestCM07AdminRoleHasAllPermissions (1 tests)
    • test_admin_role_has_all_permissions — CM-1.7: Admin role passes the require_admin gate; non-admin is rejected
  • TestCM08FreeRoleHasMinimumPermissions (1 tests)
    • test_free_role_has_minimum_permissions — CM-1.8: The 'user' role (lowest tier) is rejected by require_admin,
  • TestCM09IpAllowlistBlocksNonListedIp (2 tests)
    • test_ip_allowlist_blocks_non_listed_ip — An IP not in the allowlist must be rejected.
    • test_ip_allowlist_blocks_ip_outside_cidr — An IP outside the CIDR range must be rejected.
  • TestCM10IpAllowlistAllowsListedIp (3 tests)
    • test_ip_allowlist_allows_exact_match — An IP exactly matching an allowlist entry must be accepted.
    • test_ip_allowlist_allows_cidr_match — An IP within an allowed CIDR range must be accepted.
    • test_ip_allowlist_empty_means_unrestricted — An empty allowlist means no restrictions (allow all).
  • TestCM11DomainRestrictionBlocksWrongDomain (5 tests)
    • test_domain_restriction_blocks_wrong_domain — A referer from an unlisted domain must be rejected.
    • test_domain_restriction_allows_correct_domain — A referer from a listed domain must be accepted.
    • test_domain_restriction_allows_subdomain — A subdomain of an allowed domain must be accepted.
    • test_domain_restriction_blocks_missing_referer — A missing referer must be rejected when domains are configured.
    • test_domain_restriction_empty_means_unrestricted — An empty domain list means no restrictions.

tests/conceptual_model/test_cm02_rate_limiting.py

17 tests

  • TestLayer1IPRateLimiting (5 tests)
    • test_ip_rate_limit_under_threshold_allows — Requests under IP RPM limit pass through (are allowed).
    • test_ip_rate_limit_over_threshold_blocks — Requests exceeding IP RPM limit are blocked (return False).
    • test_ip_rate_limit_applied_before_auth — CM-2.1.3: IP rate limit is applied before auth in dispatch.
    • test_velocity_detection_triggers_on_anomalous_pattern — Rapid burst of 5xx errors triggers velocity mode activation.
    • test_authenticated_users_exempt_from_ip_limits — Authenticated requests (with valid API key/Bearer token) bypass IP limits.
  • TestLayer2APIKeyRateLimiting (5 tests)
    • test_key_rate_limit_tracks_rpm — Rate limiter uses Redis INCR with rate_limit key pattern for per-minute tracking.
    • test_key_rate_limit_enforces_plan_tier — CM-2.2.2: Different plan tiers enforce different RPM limits.
    • test_key_rate_limit_tracks_tokens_per_day — Daily token tracking is enforced via rate_limit:{key}:day:{date}:tokens Redis key.
    • test_key_rate_limit_tracks_tokens_per_month — CM-2.2.4: Monthly token tracking is not implemented as a separate window.
    • test_rate_limit_returns_proper_headers — Rate limit result contains X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset.
  • TestLayer3AnonymousRateLimiting (3 tests)
    • test_anonymous_limits_stricter_than_authenticated — CM-2.3.1: Anonymous daily limit is much stricter than authenticated.
    • test_anonymous_rate_limit_uses_ip_hash — Redis key for anonymous rate limiting uses a SHA-256 hash of the IP.
    • test_anonymous_rate_limit_blocks_excess — Anonymous rate limiter returns 'allowed: False' when daily limit exceeded.
  • TestGracefulDegradation (4 tests)
    • test_redis_down_activates_fallback — When Redis is unavailable, the fallback InMemoryRateLimiter is used.
    • test_fallback_lru_cache_500_entries — CM-2.4.2: Fallback InMemoryRateLimiter evicts LRU keys when more
    • test_fallback_ttl_15_minutes — CM-2.4.3: Fallback entries expire after 15 minutes (900s TTL).
    • test_requests_never_blocked_by_infra_failure — When both Redis AND fallback fail, request is ALLOWED (fail-open).

tests/conceptual_model/test_cm03_model_resolution.py

10 tests

  • test_alias_r1_resolves_to_deepseek
  • test_alias_gpt4o_resolves_to_openai
  • test_at_least_120_aliases_defined — CM-3.3: At least 120 aliases are defined and each resolves via apply_model_alias.
  • test_canonical_id_passes_through_unchanged
  • test_provider_detection_explicit_override_highest_priority
  • test_provider_detection_format_based_rules
  • test_provider_detection_org_prefix_fallback
  • test_model_id_transformation_fireworks_format
  • test_model_id_transformation_per_provider
  • test_unknown_alias_returns_error_or_passthrough

tests/conceptual_model/test_cm04_intelligent_routing.py

15 tests

  • TestGeneralRouterParsing (8 tests)
    • test_general_router_parses_quality_mode — CM-4.1.1: 'router:general:quality' parsed correctly.
    • test_general_router_parses_cost_mode — CM-4.1.2: 'router:general:cost' parsed correctly.
    • test_general_router_parses_latency_mode — CM-4.1.3: 'router:general:latency' parsed correctly.
    • test_general_router_parses_balanced_mode — CM-4.1.4: 'router:general:balanced' or bare 'router:general' → balanced.
    • test_general_router_quality_settings_produce_quality_model_string — CM-4.1.5: quality mode settings → model string that parses to quality mode.
    • test_general_router_cost_settings_produce_cost_model_string — CM-4.1.6: cost mode settings → model string that parses to cost mode.
    • test_general_router_latency_settings_produce_latency_model_string — CM-4.1.7: latency mode settings → model string that parses to latency mode.
    • test_general_router_invalid_mode_rejected — CM-4.1.8: 'router:general:invalid' falls back; schema rejects invalid mode.
  • TestCodeRouterParsing (7 tests)
    • test_code_router_parses_auto_mode — CM-4.2.1: 'router:code:auto' or bare 'router:code' → auto.
    • test_code_router_parses_agentic_mode — CM-4.2.2: 'router:code:agentic' parsed correctly.
    • test_code_router_parses_price_mode — CM-4.2.3: 'router:code:price' parsed correctly.
    • test_code_router_parses_quality_mode — CM-4.2.4: 'router:code:quality' parsed correctly.
    • test_code_router_uses_swe_bench_scores — CM-4.2.5: quality mode factors in SWE-bench scores for model selection.
    • test_code_router_classifies_task_complexity — CM-4.2.6: classifier assigns complexity tiers to prompts.
    • test_code_router_matches_tier_to_model — CM-4.2.7: higher complexity tier maps to higher-capability model.

tests/conceptual_model/test_cm05_provider_failover.py

24 tests

  • TestFailoverChain (12 tests)
    • test_failover_chain_has_14_providers — CM-5.1.1: build_provider_failover_chain produces a chain with 14 providers.
    • test_failover_chain_ordered_by_reliability — CM-5.1.2: build_provider_failover_chain preserves reliability ranking after initial.
    • test_failover_retries_on_502 — CM-5.1.3: should_failover returns True for 502.
    • test_failover_retries_on_503 — CM-5.1.4: should_failover returns True for 503.
    • test_failover_retries_on_504 — CM-5.1.5: should_failover returns True for 504.
    • test_failover_retries_on_401 — CM-5.1.6: should_failover returns True for 401.
    • test_failover_retries_on_402 — CM-5.1.7: should_failover returns True for 402.
    • test_failover_retries_on_403 — CM-5.1.8: should_failover returns True for 403.
    • test_failover_retries_on_404 — CM-5.1.9: should_failover returns True for 404.
    • test_failover_does_NOT_trigger_on_400 — CM-5.1.10: should_failover returns False for 400.
    • test_failover_does_NOT_trigger_on_429 — CM-5.1.11: should_failover returns False for 429.
    • test_failover_transparent_to_caller — CM-5.1.12: A successful failover returns a normal response (the caller
  • TestModelAwareRules (3 tests)
    • test_openai_models_failover_only_to_openai_or_openrouter — CM-5.2.1: openai/* models are restricted to [openai, openrouter].
    • test_anthropic_models_failover_only_to_anthropic_or_openrouter — CM-5.2.2: anthropic/* models are restricted to [anthropic, openrouter].
    • test_opensource_models_failover_across_all_providers — CM-5.2.3: meta-llama/* models can fail over to any provider.
  • TestCircuitBreaker (9 tests)
    • test_circuit_breaker_starts_closed — CM-5.3.1: A new circuit breaker starts in CLOSED state.
    • test_circuit_breaker_opens_after_5_failures — CM-5.3.2: 5 consecutive failures transition to OPEN.
    • test_circuit_breaker_4_failures_stays_closed — CM-5.3.3: 4 failures keep the circuit CLOSED.
    • test_circuit_breaker_open_blocks_requests — CM-5.3.4: An OPEN circuit rejects new requests with CircuitBreakerError.
    • test_circuit_breaker_recovery_after_60_seconds — CM-5.3.5: After timeout_seconds elapse, OPEN transitions to HALF_OPEN.
    • test_circuit_breaker_half_open_success_closes — CM-5.3.6: Reaching success_threshold (2) in HALF_OPEN transitions to CLOSED.
    • test_circuit_breaker_half_open_failure_reopens — CM-5.3.7: half_open_max_failures (2) failures in HALF_OPEN re-opens circuit.
    • test_circuit_breaker_success_resets_failure_count — CM-5.3.8: A successful request resets the consecutive failure counter.
    • test_circuit_breaker_independent_per_provider — CM-5.3.9: Each provider has an independent circuit breaker.

tests/conceptual_model/test_cm06_credit_system.py

18 tests

  • TestCostCalculation (4 tests)
    • test_cost_formula_prompt_plus_completion — CM-6.1.1: Cost follows the formula promptprice + completionprice.
    • test_cost_zero_tokens_zero_cost — CM-6.1.2: 0 prompt + 0 completion tokens = $0.00.
    • test_cost_uses_model_specific_pricing — CM-6.1.3: Different models produce different costs for the same tokens.
    • test_cost_calculation_precision — CM-6.1.4: Pricing values support Decimal-level precision without float errors.
  • TestCreditDeductionOrder (4 tests)
    • test_subscription_allowance_used_before_purchased — CM-6.2.1: Allowance is consumed first when both balances are available.
    • test_purchased_credits_used_after_allowance_exhausted — CM-6.2.2: purchased_credits absorb the remainder after allowance is exhausted.
    • test_purchased_credits_never_expire — CM-6.2.3: Purchased credits are deductible regardless of time elapsed.
    • test_subscription_allowance_does_not_roll_over — CM-6.2.4: subscription_allowance is reset on renewal; unused portion is lost.
  • TestPreFlightCreditCheck (4 tests)
    • test_preflight_check_insufficient_returns_402 — CM-6.3.1: Zero credits triggers a 402 before any provider call.
    • test_preflight_check_estimates_max_cost — CM-6.3.2: Pre-flight uses max_tokens * price to estimate worst-case cost.
    • test_preflight_check_passes_when_sufficient — CM-6.3.3: If user has enough credits the check returns allowed=True.
    • test_no_provider_call_on_failed_preflight — CM-6.3.4: When pre-flight fails, the provider client is never invoked.
  • TestIdempotentDeduction (2 tests)
    • test_same_request_id_deducted_once — CM-6.4.1: A duplicate request_id causes the deduction to be skipped.
    • test_different_request_ids_deducted_separately — CM-6.4.2: Different request_ids each produce their own deduction.
  • TestAutoRefund (3 tests)
    • test_provider_5xx_triggers_auto_refund — CM-6.5.1: 500/502/503 provider errors qualify for auto-refund.
    • test_provider_timeout_triggers_auto_refund — CM-6.5.2: Timeout errors qualify for auto-refund.
    • test_user_4xx_does_NOT_trigger_refund — CM-6.5.3: User-caused 400 errors are not refundable.
  • TestHighValueModelProtection (1 tests)
    • test_premium_model_blocked_if_pricing_is_default — CM-6.6.1: GPT-4/Claude/Gemini should be BLOCKED when only default pricing is available.

tests/conceptual_model/test_cm07_plans_trials.py

10 tests

  • test_new_user_gets_5_dollar_credits — create_enhanced_user inserts a user row with credits=5.0 ($5.00).
  • test_new_user_gets_3_day_trial — start_trial_for_key should default to trial_days=3 per CM spec.
  • test_trial_1m_token_limit — track_trial_usage_for_key tracks token usage; DB enforces 1M limit.
  • test_trial_10k_request_limit — track_trial_usage_for_key tracks request usage; DB enforces 10K limit.
  • test_expired_trial_returns_402 — An expired trial requesting a non-free model raises HTTPException (403).
  • test_expired_trial_can_access_free_models — An expired trial can still use models ending with ':free'.
  • test_active_trial_can_access_all_models — A valid, active trial should be able to access standard (non-free) models.
  • test_plan_tiers_exist — get_all_plans returns plans; verify that 4 canonical tiers can be
  • test_team_has_higher_rate_limits_than_dev — Team plan limits exceed Dev plan limits when queried via check_plan_entitlements.
  • test_purchased_credits_survive_plan_change — assign_user_plan changes the plan but does NOT modify the user's

tests/conceptual_model/test_cm08_caching.py

16 tests

  • TestExactMatchResponseCache (6 tests)
    • test_exact_match_cache_hit — CM-8.1.1: Same params produce a cache hit on second lookup.
    • test_exact_match_cache_miss — CM-8.1.2: Different params produce a cache miss.
    • test_exact_match_cache_uses_sha256 — CM-8.1.3: CM says cache keys use SHA-256 hashing.
    • test_exact_match_cache_max_20k_entries — CM-8.1.4: cache_catalog_response calls _evict_lru_if_needed which
    • test_exact_match_cache_60min_ttl — CM-8.1.5: cache_catalog_response stores entries with CATALOG_CACHE_TTL (3600s).
    • test_exact_match_cache_lru_eviction — CM-8.1.6: _evict_lru_if_needed removes oldest entries when count >= MAX_CACHE_ENTRIES.
  • TestSupportingCaches (6 tests)
    • test_auth_cache_ttl_5_to_10_minutes — CM-8.2.1: Auth cache functions call setex with TTLs in the 5-10 min range.
    • test_catalog_l1_cache_ttl_60_minutes — CM-8.2.2: Catalog L1 (response) cache stores entries with 60-minute TTL.
    • test_catalog_l2_cache_ttl_15_to_30_minutes — CM-8.2.3: Per-provider (L2) catalog cache has 15-30 min TTL.
    • test_health_cache_ttl_6_minutes — CM-8.2.4: SimpleHealthCache.set_cache calls setex with the configured TTL.
    • test_local_memory_cache_500_entries — CM-8.2.5: Local memory cache allows max 500 entries.
    • test_local_memory_cache_ttl_15_minutes — CM-8.2.6: Local memory cache default TTL is 15 min (900s).
  • TestCacheDegradation (4 tests)
    • test_redis_down_falls_back_to_local_memory — CM-8.3.1: When Redis is unavailable, catalog_response_cache returns None
    • test_all_caches_miss_falls_through_to_db — CM-8.3.2: When all caches miss, request falls through to database.
    • test_cache_failure_never_blocks_request — CM-8.3.3: Cache exceptions are caught; request proceeds without error.
    • test_db_query_cache_reduces_load — CM-8.3.4: Repeated queries served from cache reduce DB load.

tests/conceptual_model/test_cm09_model_catalog.py

8 tests

  • TestModelMetadata (5 tests)
    • test_every_model_has_required_fields — CM-9.1.1: Calling normalize_fireworks_model on raw provider data
    • test_model_id_is_canonical_format — CM-9.1.2: normalize_fireworks_model produces IDs that include the raw slug.
    • test_pricing_field_never_null — CM-9.1.3: enrich_model_with_pricing populates pricing on a model.
    • test_modality_is_known_type — CM-9.1.4: normalize_fireworks_model outputs a known modality type.
    • test_context_length_is_positive_integer — CM-9.1.5: normalize_fireworks_model preserves context_length as a positive int.
  • TestCatalogInclusionRules (3 tests)
    • test_model_without_pricing_excluded — CM-9.2.1: A gateway-provider model with no pricing is excluded from the catalog.
    • test_model_with_inactive_provider_excluded — CM-9.2.2: A model whose provider slug is not in GATEWAY_REGISTRY is excluded.
    • test_deduplicated_view_no_duplicate_ids — CM-9.2.3: The merge/deduplicated view contains no duplicate model IDs.

tests/conceptual_model/test_cm10_api_compatibility.py

14 tests

  • TestOpenAIResponseFormatHasChoices (1 tests)
    • test_openai_response_format_has_choices
  • TestOpenAIResponseFormatHasUsage (1 tests)
    • test_openai_response_format_has_usage
  • TestOpenAIResponseFormatHasId (1 tests)
    • test_openai_response_format_has_id
  • TestOpenAIResponseFormatHasModel (1 tests)
    • test_openai_response_format_has_model
  • TestOpenAIStreamingSSEFormat (1 tests)
    • test_openai_streaming_sse_format
  • TestOpenAIStreamingEndsWithDone (1 tests)
    • test_openai_streaming_ends_with_done
  • TestOpenAIJsonModeReturnsValidJson (1 tests)
    • test_openai_json_mode_returns_valid_json — When a provider returns JSON content, process_openrouter_response
  • TestOpenAIToolCallingResponseFormat (1 tests)
    • test_openai_tool_calling_response_format — extract_message_with_tools extracts tool_calls from a message dict.
  • TestOpenAILogprobsIncludedWhenRequested (1 tests)
    • test_openai_logprobs_included_when_requested — process_openrouter_response preserves logprobs when present in provider response.
  • TestAnthropicResponseFormatHasContent (1 tests)
    • test_anthropic_response_format_has_content
  • TestAnthropicResponseFormatHasUsage (1 tests)
    • test_anthropic_response_format_has_usage
  • TestAnthropicStreamingEventFormat (1 tests)
    • test_anthropic_streaming_event_format
  • TestResponseNormalizedRegardlessOfProvider (1 tests)
    • test_response_normalized_regardless_of_provider
  • TestProviderSpecificFieldsStripped (1 tests)
    • test_provider_specific_fields_stripped

tests/conceptual_model/test_cm11_health_monitoring.py

9 tests

  • test_health_check_critical_tier_5min_interval — CM-11.1: Top 5% models (CRITICAL tier) are scheduled every 5 minutes (300s).
  • test_health_check_popular_tier_30min_interval — CM-11.2: Next 20% models (POPULAR tier) are scheduled every 30 minutes (1800s).
  • test_health_check_standard_tier_2_to_4hr_interval — CM-11.3: Remaining 75% models (STANDARD tier) are checked every 2-4 hours (7200-14400s).
  • test_passive_health_captures_from_inference — CM-11.4: Every inference call contributes health data via capture_model_health.
  • test_health_endpoint_always_returns_200 — CM-11.5: /health returns HTTP 200 even when subsystems are down.
  • test_health_response_contains_version — CM-11.6: /health response body must include a 'version' field.
  • test_health_response_contains_status — CM-11.7: /health response includes an overall 'status' field.
  • test_health_response_contains_timestamp — CM-11.8: /health response includes a timestamp.
  • test_incident_severity_levels — CM-11.9: System supports Critical, High, Medium, and Low severity levels.

tests/conceptual_model/test_cm12_auth_flow.py

10 tests

  • test_login_rate_limit_10_per_15min — 11th login attempt from same IP within 15 min window returns 429 (not allowed).
  • test_register_rate_limit_3_per_hour — 4th registration attempt from same IP within 1 hour returns 429.
  • test_new_user_provisioned_with_basic_tier — create_enhanced_user sets tier='basic' for new users.
  • test_new_user_gets_auto_created_api_key — create_enhanced_user creates a primary API key for the new user.
  • test_api_key_format_gw_env_prefix — generate_secure_api_key produces keys with gw_{env}_ prefix.
  • test_auth_info_priority_email_first — When user has email + Google + phone linked accounts, email is resolved first.
  • test_auth_info_priority_google_over_phone — When user has Google + phone (no email account), Google email is resolved.
  • test_partner_code_triggers_extended_trial — A valid partner code triggers PartnerTrialService.start_partner_trial.
  • test_referral_code_stored_on_new_user — Referral code is saved to user record via _process_referral_code_background.
  • test_temp_email_detection_blocks_registration — Known disposable email domains are detected by is_temporary_email_domain.

tests/conceptual_model/test_cm13_observability.py

8 tests

  • TestCM1301PrometheusInferenceRequestCounter (1 tests)
    • test_prometheus_inference_request_counter — The model_inference_requests_total Prometheus Counter must exist and accept labels.
  • TestCM1302PrometheusInferenceDurationHistogram (1 tests)
    • test_prometheus_inference_duration_histogram — The model_inference_duration_seconds Histogram must exist and accept observations.
  • TestCM1303PrometheusTokensUsedCounter (1 tests)
    • test_prometheus_tokens_used_counter — The tokens_used_total Prometheus Counter must exist and accept labels.
  • TestCM1304PrometheusCreditsUsedCounter (1 tests)
    • test_prometheus_credits_used_counter — The credits_used_total Prometheus Counter must exist and accept labels.
  • TestCM1305PrometheusTtfcHistogram (1 tests)
    • test_prometheus_ttfc_histogram — The time_to_first_chunk_seconds Histogram must exist and accept observations.
  • TestCM1306SentryCapturesExceptions (1 tests)
    • test_sentry_captures_exceptions — AutoSentryMiddleware calls sentry_sdk.capture_exception
  • TestCM1307OpentelemetryTraceCreatedPerRequest (1 tests)
    • test_opentelemetry_trace_created_per_request — OpenTelemetryConfig.initialize() returns a bool indicating success/failure.
  • TestCM1308AuditLogOnSecurityViolation (1 tests)
    • test_audit_log_on_security_violation — log_security_event writes a row to security_audit_log with the

tests/conceptual_model/test_cm14_token_estimation.py

3 tests

  • TestCM1401TokenEstimationFallback1Per4Chars (1 tests)
    • test_token_estimation_fallback_1_per_4_chars — CM specifies 1 token per 4 characters (400 chars -> ~100 tokens).
  • TestCM1402TokenEstimationUsedWhenProviderOmitsUsage (1 tests)
    • test_token_estimation_used_when_provider_omits_usage — When no explicit max_tokens is provided and messages exist,
  • TestCM1403RealUsagePreferredOverEstimation (1 tests)
    • test_real_usage_preferred_over_estimation — When max_tokens is explicitly provided, estimate_message_tokens

tests/conceptual_model/test_cm15_image_audio.py

3 tests

  • TestCM1501ImageGenerationDeductsCredits (1 tests)
    • test_image_generation_deducts_credits — The image generation route calculates a positive cost for images
  • TestCM1502ImageGenerationInsufficientCredits402 (1 tests)
    • test_image_generation_insufficient_credits_402 — When a user has 0 credits, the image endpoint raises HTTP 402.
  • TestCM1503AudioTranscriptionReturnsText (1 tests)
    • test_audio_transcription_returns_text — The audio transcription endpoint returns a response containing

tests/conceptual_model/test_cm16_webhooks_events.py

6 tests

  • TestCM1601WebhookPayloadHmacSigned (1 tests)
    • test_webhook_payload_hmac_signed — generate_webhook_signature produces a valid HMAC-SHA256 hex digest
  • TestCM1602WebhookCreditsLowEventTriggered (2 tests)
    • test_webhook_credits_low_event_triggered — check_low_balance_alert returns a LowBalanceAlert when the user's
    • test_no_alert_when_credits_above_threshold — check_low_balance_alert returns None when credits are above $5.
  • TestCM1603WebhookCreditsDepletedEventTriggered (1 tests)
    • test_webhook_credits_depleted_event_triggered — The low balance alert fires when credits are 0 (depleted),
  • TestCM1604WebhookRetryExponentialBackoff (1 tests)
    • test_webhook_retry_exponential_backoff — send_webhook_notification should retry failed deliveries with
  • TestCM1605StripeWebhookAlwaysReturns200 (1 tests)
    • test_stripe_webhook_always_returns_200 — The Stripe webhook handler returns HTTP 200 even when processing

tests/conceptual_model/test_cm17_deployment.py

3 tests

  • TestCM1701CreateAppReturnsFastapiInstance (1 tests)
    • test_create_app_returns_fastapi_instance — create_app() must return a FastAPI application instance.
  • TestCM1702VercelEntryPointImportsApp (1 tests)
    • test_vercel_entry_point_imports_app — api/index.py must export an 'app' object that is a FastAPI instance.
  • TestCM1703AppIncludesAllRouteGroups (1 tests)
    • test_app_includes_all_route_groups — The app must have routes for chat, auth, users, admin, health,

tests/conceptual_model/test_cm18_provider_ecosystem.py

3 tests

  • TestCM1801AtLeast30ProvidersRegistered (1 tests)
    • test_at_least_30_providers_registered — GATEWAY_REGISTRY must contain at least 30 provider entries,
  • TestCM1802EachProviderHasClientModule (1 tests)
    • test_each_provider_has_client_module — Key providers from GATEWAY_REGISTRY should have importable *_client modules.
  • TestCM1803ProviderClientImplementsRequiredInterface (1 tests)
    • test_provider_client_implements_required_interface — Provider client modules should have callable functions for sending

tests/config/

7 files, 122 tests

tests/config/test_config.py

42 tests

  • TestConfigEnvironmentDetection (8 tests)
    • test_is_production_detection — Test production environment detection
    • test_is_staging_detection — Test staging environment detection
    • test_is_development_detection — Test development environment detection (default)
    • test_is_testing_detection_with_testing_env — Test testing environment detection with APP_ENV=testing
    • test_is_testing_detection_with_test_env — Test testing environment detection with APP_ENV=test
    • test_is_testing_detection_with_testing_flag_true — Test testing environment detection with TESTING=true
    • test_is_testing_detection_with_testing_flag_1 — Test testing environment detection with TESTING=1
    • test_is_testing_detection_with_testing_flag_yes — Test testing environment detection with TESTING=yes
  • TestConfigProviderKeys (4 tests)
    • test_openrouter_keys — Test OpenRouter configuration
    • test_openrouter_defaults — Test OpenRouter default values
    • test_openrouter_key_strips_whitespace — Ensure OpenRouter API key trimming removes accidental whitespace
    • test_all_provider_keys — Test all provider API keys are loaded
  • TestConfigGoogleVertex (2 tests)
    • test_google_vertex_defaults — Test Google Vertex AI default configuration
    • test_google_vertex_custom_values — Test Google Vertex AI custom configuration
  • TestConfigMonitoring (9 tests)
    • test_prometheus_enabled_by_default — Test Prometheus is enabled by default
    • test_prometheus_enabled_explicit_true — Test Prometheus enabled with explicit true values
    • test_prometheus_disabled — Test Prometheus can be disabled
    • test_prometheus_scrape_enabled_by_default — Test Prometheus scrape is enabled by default
    • test_tempo_disabled_by_default — Test Tempo is disabled by default
    • test_tempo_enabled — Test Tempo can be enabled
    • test_loki_disabled_by_default — Test Loki is disabled by default
    • test_loki_enabled — Test Loki can be enabled
    • test_otel_service_name_default — Test OTEL service name default
  • TestConfigValidation (14 tests)
    • test_validate_success_with_all_vars — Test validate succeeds with all required variables
    • test_validate_skips_in_vercel_environment — Test validate skips validation in Vercel environment
    • test_validate_raises_on_missing_supabase_url — Test validate raises error on missing SUPABASE_URL
    • test_validate_raises_on_missing_multiple_vars — Test validate raises error listing all missing variables
    • test_validate_critical_env_vars_success — Test validate_critical_env_vars with all variables present
    • test_validate_critical_env_vars_missing_vars — Test validate_critical_env_vars with missing variables
    • test_validate_critical_env_vars_skips_in_vercel — Test validate_critical_env_vars skips in Vercel environment
    • test_validate_raises_on_supabase_url_missing_protocol — Test validate raises error when SUPABASE_URL lacks http:// or https:// protocol
    • test_validate_accepts_http_protocol — Test validate accepts SUPABASE_URL with http:// protocol
    • test_validate_accepts_https_protocol — Test validate accepts SUPABASE_URL with https:// protocol
    • test_validate_critical_env_vars_detects_missing_protocol — Test validate_critical_env_vars detects SUPABASE_URL without protocol
    • test_validate_raises_in_vercel_when_url_missing_protocol — Test validate raises error in Vercel env when SUPABASE_URL lacks protocol
    • test_validate_critical_env_vars_detects_missing_protocol_in_vercel — Test validate_critical_env_vars detects SUPABASE_URL without protocol in Vercel
    • test_validate_skips_presence_check_in_vercel — Test validate skips checking presence of keys in Vercel env (but validates URL format)
  • TestConfigGetSupabaseConfig (1 tests)
    • test_get_supabase_config_returns_tuple — Test get_supabase_config returns URL and key as tuple
  • TestConfigClarifai (1 tests)
    • test_clarifai_configuration — Test Clarifai API configuration
  • TestConfigAiHubMix (1 tests)
    • test_aihubmix_configuration — Test AiHubMix API configuration
  • TestConfigAdminAndAnalytics (2 tests)
    • test_admin_email_configuration — Test admin email configuration
    • test_openrouter_cookie_configuration — Test OpenRouter cookie configuration

tests/config/test_db_config.py

32 tests

  • TestDatabaseConfig (5 tests)
    • test_default_configuration — Test default database configuration values
    • test_custom_configuration — Test custom database configuration from environment
    • test_psycopg2_not_available — Test configuration when psycopg2 is not installed
    • test_get_connection_string — Test connection string generation
    • test_get_connection_dict — Test connection dictionary generation
  • TestConnectionPool (7 tests)
    • test_get_connection_pool_success — Test successful connection pool creation
    • test_get_connection_pool_reuses_existing — Test connection pool reuse
    • test_get_connection_pool_not_available — Test connection pool when psycopg2 not available
    • test_get_connection_pool_failure — Test connection pool creation failure
    • test_close_all_connections — Test closing all connections in pool
    • test_close_all_connections_when_none — Test closing connections when pool is None
    • test_close_all_connections_error — Test error handling when closing connections
  • TestConnectionContextManager (6 tests)
    • test_get_connection_not_available — Test get_connection when psycopg2 not available
    • test_get_connection_success — Test successful connection retrieval
    • test_get_connection_getconn_returns_none — Test when getconn returns None
    • test_get_connection_with_exception — Test connection rollback on exception
    • test_get_connection_rollback_fails — Test when rollback itself fails
    • test_get_connection_putconn_fails — Test when putconn fails
  • TestDatabaseOperations (9 tests)
    • test_test_connection_not_available — Test test_connection when psycopg2 not available
    • test_test_connection_success — Test successful connection test
    • test_test_connection_failure — Test failed connection test
    • test_get_database_info_not_available — Test get_database_info when psycopg2 not available
    • test_get_database_info_success — Test successful database info retrieval
    • test_execute_query_success — Test successful query execution
    • test_execute_query_with_params — Test query execution with parameters
    • test_execute_query_fetch_one — Test query execution with fetch_one=True
    • test_execute_many_success — Test batch execution success
  • TestGlobalFunctions (5 tests)
    • test_get_db_config_singleton — Test get_db_config returns singleton
    • test_get_db_connection — Test get_db_connection helper
    • test_test_db_connection — Test test_db_connection helper
    • test_close_db_connections — Test close_db_connections helper
    • test_is_db_available — Test is_db_available helper

tests/config/test_logging_config.py

2 tests

  • TestLoggingConfig (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module has expected public API

tests/config/test_opentelemetry_config.py

2 tests

  • TestOpentelemetryConfig (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module has expected public API

tests/config/test_railway_config.py

11 tests

  • TestRailwayConfiguration (9 tests)
    • test_railway_toml_exists — Verify railway.toml exists in project root.
    • test_railway_json_exists — Verify railway.json exists in project root.
    • test_healthcheck_timeout_sufficient — Verify healthcheck timeout is sufficient for app startup.
    • test_healthcheck_initial_delay_sufficient — Verify initial delay is sufficient for first healthcheck.
    • test_healthcheck_path_configured — Verify healthcheck path is properly configured.
    • test_healthcheck_interval_configured — Verify healthcheck interval is properly configured.
    • test_railway_configs_consistency — Verify railway.toml and railway.json have consistent healthcheck settings.
    • test_start_command_configured — Verify start command is properly configured.
    • test_restart_policy_configured — Verify restart policy is properly configured.
  • TestRailwayHealthcheckRegression (2 tests)
    • test_healthcheck_timeout_not_10_seconds — Regression test: Ensure healthcheck timeout is not 10 seconds.
    • test_healthcheck_initial_delay_not_30_seconds — Regression test: Ensure initial delay is not 30 seconds.

tests/config/test_redis_config.py

2 tests

  • TestRedisConfig (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module has expected public API

tests/config/test_supabase_config.py

31 tests

  • TestGetSupabaseClientValidation (6 tests)
    • test_raises_error_when_supabase_url_not_set — Test that get_supabase_client raises RuntimeError when SUPABASE_URL is not set
    • test_raises_error_when_supabase_url_missing_protocol — Test that get_supabase_client raises RuntimeError when SUPABASE_URL lacks protocol
    • test_error_message_includes_example_fix — Test that error message for missing protocol includes example of correct format
    • test_accepts_valid_https_url — Test that get_supabase_client accepts valid https:// URL
    • test_accepts_valid_http_url_for_local_dev — Test that get_supabase_client accepts valid http:// URL for local development
    • test_logs_masked_url_on_initialization — Test that initialization logs a masked version of the URL
  • TestHttpxClientConfiguration (8 tests)
    • test_httpx_client_includes_apikey_header — Test that httpx client is created with apikey header.
    • test_httpx_client_includes_authorization_header — Test that httpx client is created with Authorization Bearer header.
    • test_httpx_client_auth_headers_match_supabase_key — Test that both auth headers use the same SUPABASE_KEY value.
    • test_httpx_client_includes_base_url — Test that httpx client is created with correct PostgREST base_url.
    • test_httpx_client_injected_into_postgrest_session — Test that the configured httpx client is injected into postgrest.session.
    • test_httpx_client_has_http2_disabled — Test that httpx client is created with HTTP/2 disabled to prevent connection errors.
    • test_httpx_client_has_connection_pooling — Test that httpx client is created with connection pooling limits.
    • test_httpx_client_has_timeout_configured — Test that httpx client is created with appropriate timeout settings.
  • TestLazySupabaseClient (2 tests)
    • test_lazy_client_repr — Test that lazy client has a meaningful repr
    • test_lazy_client_raises_attribute_error_for_dunder_attrs — Test that lazy client raises AttributeError for dunder attributes
  • TestGetInitializationStatus (3 tests)
    • test_get_initialization_status_when_not_initialized — Test initialization status when client is not yet initialized
    • test_get_initialization_status_when_initialized — Test initialization status when client is successfully initialized
    • test_get_initialization_status_when_error_occurred — Test initialization status when initialization failed
  • TestErrorPersistence (2 tests)
    • test_get_supabase_client_reraises_previous_error — Test that get_supabase_client re-raises previous initialization errors within TTL
    • test_initialization_error_captured_on_failure — Test that initialization errors are captured for future reference
  • TestTestConnectionInternal (2 tests)
    • test_test_connection_internal_success — Test successful connection test
    • test_test_connection_internal_failure — Test connection test failure
  • TestTestConnection (1 tests)
    • test_test_connection_uses_cached_client — Test that test_connection uses the cached client
  • TestConnectionErrorHandling (7 tests)
    • test_is_connection_error_detects_write_error — Test that is_connection_error detects WriteError with bad file descriptor
    • test_is_connection_error_detects_connection_terminated — Test that is_connection_error detects connection terminated errors
    • test_is_connection_error_ignores_other_errors — Test that is_connection_error doesn't flag non-connection errors
    • test_execute_with_retry_retries_on_connection_error — Test that execute_with_retry retries when a connection error occurs
    • test_execute_with_retry_exhausts_retries — Test that execute_with_retry raises after exhausting retries
    • test_execute_with_retry_doesnt_retry_non_connection_errors — Test that execute_with_retry doesn't retry non-connection errors
    • test_refresh_supabase_client_closes_old_connection — Test that refresh_supabase_client properly closes the old connection

tests/db/

32 files, 533 tests

tests/db/test_activity.py

27 tests

  • TestLogActivity (4 tests)
    • test_log_activity_success — Test successfully logging an activity
    • test_log_activity_with_defaults — Test logging activity with default values
    • test_log_activity_no_data — Test logging activity when no data is returned
    • test_log_activity_error — Test error handling during activity logging
  • TestActivityStatistics (5 tests)
    • test_get_activity_stats_with_data — Test retrieving activity statistics with data
    • test_get_activity_stats_empty — Test retrieving statistics when no activity
    • test_get_activity_stats_with_date_range — Test retrieving statistics with specific date range
    • test_get_activity_stats_default_30_days — Test default 30-day lookback
    • test_get_activity_stats_error — Test error handling in statistics
  • TestActivityLogRetrieval (7 tests)
    • test_get_activity_log_success — Test retrieving activity log
    • test_get_activity_log_with_pagination — Test activity log with pagination
    • test_get_activity_log_with_date_filters — Test activity log with date filtering
    • test_get_activity_log_with_model_filter — Test activity log with model filter
    • test_get_activity_log_with_provider_filter — Test activity log with provider filter
    • test_get_activity_log_empty — Test retrieving empty activity log
    • test_get_activity_log_error — Test error handling in activity log retrieval
  • TestProviderDetection (9 tests)
    • test_detect_openai_models — Test detecting OpenAI models
    • test_detect_anthropic_models — Test detecting Anthropic models
    • test_detect_google_models — Test detecting Google models
    • test_detect_meta_models — Test detecting Meta models
    • test_detect_mistral_models — Test detecting Mistral AI models
    • test_detect_alibaba_models — Test detecting Alibaba models
    • test_detect_deepseek_models — Test detecting DeepSeek models
    • test_detect_unknown_models — Test detecting unknown/other models
    • test_provider_detection_case_insensitive — Test provider detection is case-insensitive
  • TestActivityIntegration (2 tests)
    • test_log_and_retrieve_activity — Test logging activity and retrieving it
    • test_stats_include_logged_activity — Test that logged activities appear in statistics

tests/db/test_api_keys.py

18 tests

  • test_check_key_name_uniqueness
  • test_create_api_key_primary_sets_trial_and_prefix_and_audit
  • test_create_api_key_refreshes_schema_cache_on_pgrst204
  • test_create_api_key_schema_cache_error_fallback — Ensure we recover from PostgREST schema cache misses for new columns.
  • test_get_user_api_keys_builds_fields
  • test_delete_api_key_new_and_legacy
  • test_validate_api_key_prefers_api_keys_then_fallback
  • test_increment_api_key_usage_updates_new_or_legacy
  • test_get_api_key_usage_stats_new_vs_legacy
  • test_update_api_key_name_uniqueness_and_expiration_and_rate_limit
  • test_validate_api_key_permissions
  • test_get_api_key_by_id
  • test_get_user_all_api_keys_usage
  • test_get_api_key_by_key_retries_on_http2_error — HTTP/2 connection errors should trigger retry via execute_with_retry
  • test_get_api_key_by_key_retries_on_connection_terminated — ConnectionTerminated errors should trigger retry
  • test_get_api_key_by_key_retries_on_send_headers_error — SEND_HEADERS state errors should trigger retry
  • test_get_api_key_by_key_returns_none_after_max_retries — Should return None after max retries are exhausted
  • test_get_api_key_by_key_no_retry_on_non_connection_error — Non-connection errors should not trigger retry

tests/db/test_api_keys_graceful_degradation.py

6 tests

  • TestAPIKeyGracefulDegradation (6 tests)
    • test_create_api_key_missing_rate_limit_configs_table — Test that API key creation succeeds even when rate_limit_configs table is missing.
    • test_create_api_key_missing_audit_logs_table — Test that API key creation succeeds even when api_key_audit_logs table is missing.
    • test_create_api_key_both_tables_missing — Test that API key creation succeeds even when BOTH auxiliary tables are missing.
    • test_delete_api_key_missing_tables — Test that API key deletion succeeds when auxiliary tables are missing.
    • test_update_api_key_missing_tables — Test that API key update succeeds when auxiliary tables are missing.
    • test_create_api_key_real_error_still_logged — Test that real errors (non-missing-table) are still logged properly.

tests/db/test_api_keys_hash_salt_validation.py

7 tests

  • TestKeyHashSaltValidation (7 tests)
    • test_create_api_key_fails_without_key_hash_salt — Test that API key creation fails when KEY_HASH_SALT is missing
    • test_create_api_key_fails_with_short_key_hash_salt — Test that API key creation fails when KEY_HASH_SALT is too short
    • test_hash_salt_check_not_bypassed_by_encryption_failure — Test that KEY_HASH_SALT validation happens BEFORE encryption.
    • test_create_api_key_succeeds_with_valid_salt_and_no_encryption — Test that API key creation succeeds with valid KEY_HASH_SALT but no encryption
    • test_create_api_key_succeeds_with_salt_and_encryption — Test that API key creation succeeds with both KEY_HASH_SALT and encryption
    • test_error_message_includes_generation_command — Test that error message includes the command to generate KEY_HASH_SALT
    • test_hash_validation_order_before_encryption — Test that hash validation happens strictly before encryption.

tests/db/test_backend_error_fixes_jan2.py

24 tests

  • TestAPIKeysDataAccessSafety (7 tests)
    • test_create_api_key_handles_empty_result — Test that create_api_key handles empty database result safely
    • test_get_api_key_usage_stats_handles_empty_result — Test that get_api_key_usage_stats handles missing API key safely
    • test_update_api_key_handles_empty_key_lookup — Test that update_api_key handles missing API key safely
    • test_update_api_key_handles_empty_update_result — Test that update_api_key handles failed update safely
    • test_delete_api_key_handles_empty_lookup — Test that delete_api_key handles missing API key safely
    • test_create_api_key_audit_log_uses_safe_id — Test that create_api_key uses safely extracted ID for audit logs
    • test_delete_api_key_uses_safe_id_for_cleanup — Test that delete_api_key uses safely extracted ID for cleanup operations
  • TestChatCompletionRequestsDataAccessSafety (4 tests)
    • test_get_model_id_by_name_handles_empty_provider_result — Test that get_model_id_by_name handles missing provider safely
    • test_get_model_id_by_name_handles_empty_model_result — Test that get_model_id_by_name handles missing model safely
    • test_save_chat_completion_request_handles_empty_insert_result — Test that save_chat_completion_request handles failed insert safely
    • test_get_model_performance_metrics_handles_empty_model_data — Test that get_model_performance_metrics handles missing model info safely
  • TestAISDKClientProviderSafety (4 tests)
    • test_process_ai_sdk_response_handles_empty_choices — Test that _process_ai_sdk_response handles empty choices safely
    • test_process_ai_sdk_response_handles_missing_choices_attribute — Test that _process_ai_sdk_response handles missing choices attribute safely
    • test_process_ai_sdk_response_handles_missing_message_attributes — Test that _process_ai_sdk_response handles missing message attributes safely
    • test_process_ai_sdk_response_handles_valid_response — Test that _process_ai_sdk_response processes valid response correctly
  • TestIntegrationScenarios (5 tests)
    • test_concurrent_empty_result_handling — Test that multiple concurrent empty results don't cause crashes
    • test_error_propagation_chain — Test that errors propagate correctly through the chain
    • test_model_lookup_fallback_chain — Test that model lookup gracefully falls through multiple strategies
    • test_safe_get_first_validates_keys — Test that safe_get_first validates required keys exist
    • test_provider_error_distinguishes_from_generic_errors — Test that ProviderError is distinct from generic exceptions
  • TestEdgeCases (4 tests)
    • test_safe_get_first_with_none_data — Test safe_get_first handles None data attribute
    • test_safe_get_first_with_dict_instead_of_list — Test safe_get_first handles dict data instead of list
    • test_ai_sdk_response_with_malformed_choice_attributes — Test AI SDK handles choices with missing or malformed attributes
    • test_save_chat_completion_request_preserves_result_on_success — Test that save_chat_completion_request returns the created record

tests/db/test_chat_history.py

5 tests

  • test_create_and_get_session_with_messages
  • test_get_user_chat_sessions_pagination_and_sort
  • test_update_and_delete_chat_session
  • test_chat_session_stats
  • test_search_chat_sessions_title_and_message

tests/db/test_chat_history_deduplication.py

8 tests

  • test_save_message_no_duplicate_normal_path — Test normal save when no duplicate exists
  • test_save_message_duplicate_detected_returns_existing — Test that duplicate message returns existing instead of creating new
  • test_save_message_skip_duplicate_check_creates_new — Test that skip_duplicate_check=True bypasses duplicate detection
  • test_save_message_empty_content_allowed — Test that empty content is allowed and saved
  • test_save_message_different_sessions_not_duplicate — Test that same content in different sessions is not considered duplicate
  • test_save_message_different_roles_not_duplicate — Test that same content with different roles is not considered duplicate
  • test_save_message_updates_session_timestamp — Test that saving a message updates session updated_at
  • test_save_message_with_model_updates_session_model — Test that saving a message with model updates session model

tests/db/test_coupons.py

24 tests

  • TestCreateCoupon (5 tests)
    • test_create_global_coupon_success — Test creating a global coupon
    • test_create_user_specific_coupon_success — Test creating a user-specific coupon
    • test_create_user_specific_without_user_id_fails — Test creating user-specific coupon without user ID fails
    • test_create_global_coupon_with_user_id_fails — Test creating global coupon with user ID fails
    • test_create_user_specific_with_max_uses_not_1_fails — Test creating user-specific coupon with max_uses != 1 fails
  • TestGetCoupon (5 tests)
    • test_get_coupon_by_code_success — Test retrieving coupon by code
    • test_get_coupon_by_code_not_found — Test retrieving non-existent coupon
    • test_get_coupon_by_id_success — Test retrieving coupon by ID
    • test_list_coupons_all — Test listing all coupons
    • test_list_coupons_with_filters — Test listing coupons with filters
  • TestUpdateCoupon (3 tests)
    • test_update_coupon_success — Test updating coupon fields
    • test_update_coupon_filters_invalid_fields — Test that invalid fields are filtered out
    • test_deactivate_coupon_success — Test deactivating a coupon
  • TestValidateCoupon (2 tests)
    • test_validate_coupon_valid — Test validating a valid coupon
    • test_validate_coupon_invalid — Test validating an invalid coupon
  • TestRedeemCoupon (2 tests)
    • test_redeem_coupon_success — Test successfully redeeming a coupon
    • test_redeem_coupon_invalid — Test redeeming an invalid coupon
  • TestUserCouponQueries (2 tests)
    • test_get_available_coupons_for_user — Test getting available coupons for a user
    • test_get_user_redemption_history — Test getting user redemption history
  • TestCouponAnalytics (2 tests)
    • test_get_coupon_analytics — Test getting analytics for a coupon
    • test_get_all_coupons_stats — Test getting overall coupon stats
  • TestCouponErrorHandling (3 tests)
    • test_get_coupon_by_code_error — Test error handling when getting coupon by code
    • test_list_coupons_error — Test error handling when listing coupons
    • test_validate_coupon_error — Test error handling during coupon validation

tests/db/test_credit_transactions.py

24 tests

  • TestLogCreditTransaction (6 tests)
    • test_log_deduction_transaction_success — Test successfully logging a credit deduction
    • test_log_addition_transaction_success — Test successfully logging a credit addition
    • test_log_transaction_with_all_types — Test logging transactions with all available transaction types
    • test_log_transaction_no_data_returned — Test handling when no data is returned from insert
    • test_log_transaction_exception_handling — Test exception handling during transaction logging
    • test_log_transaction_with_metadata — Test logging transaction with complex metadata
  • TestGetUserTransactions (5 tests)
    • test_get_transactions_basic — Test basic transaction retrieval
    • test_get_transactions_with_pagination — Test transaction retrieval with pagination
    • test_get_transactions_with_type_filter — Test filtering transactions by type
    • test_get_transactions_empty_result — Test handling empty transaction history
    • test_get_transactions_exception_handling — Test exception handling during retrieval
  • TestAddCredits (7 tests)
    • test_add_credits_success_with_api_key — Test successfully adding credits using API key
    • test_add_credits_success_with_user_id — Test successfully adding credits using user ID
    • test_add_credits_negative_amount — Test rejection of negative credit amounts
    • test_add_credits_zero_amount — Test rejection of zero credit amounts
    • test_add_credits_user_not_found — Test handling when user is not found
    • test_add_credits_update_fails — Test handling when credit update fails
    • test_add_credits_exception_handling — Test exception handling during credit addition
  • TestGetTransactionSummary (5 tests)
    • test_get_summary_with_mixed_transactions — Test summary calculation with various transaction types
    • test_get_summary_empty_transactions — Test summary with no transactions
    • test_get_summary_only_additions — Test summary with only credit additions
    • test_get_summary_only_deductions — Test summary with only credit deductions
    • test_get_summary_exception_handling — Test exception handling during summary calculation
  • TestTransactionType (1 tests)
    • test_transaction_type_values — Test all transaction type constants are defined

tests/db/test_data_access_safety.py

12 tests

  • TestUsersDataAccessSafety (1 tests)
    • test_deduct_credits_v2_handles_empty_concurrent_balance_check — Test that concurrent modification error handling doesn't crash on empty data
  • TestRateLimitsDataAccessSafety (4 tests)
    • test_get_user_rate_limits_handles_empty_key_record — Test that empty key record is handled gracefully
    • test_get_user_rate_limits_handles_empty_config_record — Test that empty rate config record is handled gracefully
    • test_get_rate_limit_config_handles_empty_results — Test get_rate_limit_config handles empty results at all levels
    • test_update_rate_limit_usage_handles_empty_existing_record — Test that empty existing record check is handled gracefully
  • TestAuthDataAccessSafety (2 tests)
    • test_password_reset_handles_empty_user_result — Test password reset doesn't crash on empty user result
    • test_reset_password_handles_empty_token_result — Test reset password handles empty token result
  • TestRateLimitingConcurrencyReenabled (5 tests)
    • test_concurrency_limit_enforced — Test that concurrency limits are now enforced (not disabled)
    • test_fallback_concurrency_limit_enforced — Test that fallback concurrency limits are now enforced (not disabled)
    • test_concurrency_limit_allows_under_limit — Test that requests under concurrency limit are allowed
    • test_main_limiter_increments_concurrency_on_allowed_request — Test that SlidingWindowRateLimiter increments concurrency counter when request allowed
    • test_main_limiter_does_not_increment_on_rejected_request — Test that concurrency counter is NOT incremented when request is rejected

tests/db/test_failover_db_model_name_migration.py

7 tests

  • TestFailoverDbModelNameMigration (7 tests)
    • test_get_providers_for_model_uses_model_name — Test that get_providers_for_model queries by model_name, not model_id
    • test_get_providers_extracts_pricing_from_model_pricing_table — Test that pricing is extracted from model_pricing relationship, not direct columns
    • test_get_providers_handles_pricing_as_list — Test that model_pricing as list (PostgREST one-to-many) is handled correctly
    • test_get_providers_handles_missing_pricing_data — Test that missing pricing data defaults to 0
    • test_get_provider_model_id_uses_model_name — Test that get_provider_model_id queries by model_name, not model_id
    • test_check_model_available_uses_model_name — Test that check_model_available_on_provider queries by model_name
    • test_get_providers_returns_empty_list_for_nonexistent_model — Test that empty list is returned when model not found

tests/db/test_feedback.py

26 tests

  • test_save_thumbs_up_feedback
  • test_save_thumbs_down_feedback_with_comment
  • test_save_regenerate_feedback_with_metadata
  • test_save_feedback_invalid_type_raises_error
  • test_save_feedback_invalid_rating_raises_error
  • test_get_user_feedback_all
  • test_get_user_feedback_by_type
  • test_get_user_feedback_by_session
  • test_get_user_feedback_by_model
  • test_get_user_feedback_pagination
  • test_get_feedback_by_session
  • test_get_feedback_by_message
  • test_update_feedback_type
  • test_update_feedback_rating
  • test_update_feedback_comment
  • test_update_feedback_wrong_user_returns_none
  • test_update_feedback_invalid_type_raises_error
  • test_update_feedback_invalid_rating_raises_error
  • test_delete_feedback_success
  • test_delete_feedback_wrong_user_returns_false
  • test_delete_feedback_nonexistent_returns_false
  • test_get_feedback_stats_basic
  • test_get_feedback_stats_with_ratings
  • test_get_feedback_stats_by_model
  • test_get_feedback_stats_filter_by_model
  • test_get_feedback_stats_empty

tests/db/test_gateway_analytics.py

8 tests

  • TestGetProviderStats (5 tests)
    • test_get_provider_stats_success — Test successfully getting provider statistics
    • test_get_provider_stats_with_user_filter — Test getting provider stats filtered by user
    • test_get_provider_stats_no_data — Test provider stats with no matching data
    • test_get_provider_stats_error_handling — Test error handling in provider stats
    • test_get_provider_stats_with_gateway_filter — Test provider stats with gateway filter
  • TestGetGatewayStats (3 tests)
    • test_get_gateway_stats_success — Test successfully getting gateway statistics
    • test_get_gateway_stats_with_user_filter — Test getting gateway stats with user filter
    • test_get_gateway_stats_time_ranges — Test gateway stats with different time ranges

tests/db/test_model_health.py

13 tests

  • TestRecordModelCall (8 tests)
    • test_record_model_call_new_record — Test recording a new model call creates a record
    • test_record_model_call_existing_record — Test recording a model call updates existing record
    • test_record_model_call_error_status — Test recording an error increments error count
    • test_record_model_call_table_not_found — Test graceful handling when table doesn't exist
    • test_record_model_call_duplicate_key_handled — Test duplicate key error is handled gracefully (race condition)
    • test_record_model_call_with_tokens — Test recording model call with token usage
    • test_record_model_call_with_gateway — Test recording model call with explicit gateway parameter
    • test_record_model_call_gateway_defaults_to_provider — Test gateway defaults to provider when not specified
  • TestGetModelHealth (2 tests)
    • test_get_model_health_found — Test retrieving health data for existing model
    • test_get_model_health_not_found — Test retrieving health data for non-existent model
  • TestGetUnhealthyModels (1 tests)
    • test_get_unhealthy_models — Test retrieving models with high error rates
  • TestGetModelHealthStats (2 tests)
    • test_get_model_health_stats — Test aggregating health stats across all models
    • test_get_model_health_stats_empty — Test stats with no models tracked

tests/db/test_models_catalog_db.py

30 tests

  • test_transform_db_model_to_api_format_complete — Test transformation with all fields present
  • test_transform_db_model_with_missing_fields — Test transformation gracefully handles missing fields
  • test_transform_db_model_with_null_pricing — Test transformation when pricing fields are null
  • test_transform_db_model_error_handling — Test transformation handles errors gracefully
  • test_transform_db_models_batch — Test batch transformation
  • test_transform_preserves_no_data_loss — Test that transformation preserves all important data
  • test_get_all_models_for_catalog — Test fetching all models from database
  • test_get_all_models_include_inactive — Test including inactive models
  • test_get_models_by_gateway_for_catalog — Test filtering by provider
  • test_get_models_for_catalog_with_filters_no_filters — Test advanced query with no filters (should return all)
  • test_get_models_for_catalog_with_filters_by_gateway — Test filtering by gateway
  • test_get_models_for_catalog_with_filters_by_modality — Test filtering by modality
  • test_get_models_for_catalog_with_filters_search — Test search functionality
  • test_get_models_for_catalog_with_filters_pagination — Test pagination
  • test_get_models_for_catalog_combined_filters — Test combining multiple filters
  • test_get_models_count_by_filters — Test count function
  • test_get_models_count_matches_query — Test count matches actual query results
  • test_get_model_by_model_id_string — Test looking up single model by ID
  • test_get_model_by_model_id_string_not_found — Test lookup with non-existent model ID
  • test_get_catalog_statistics — Test catalog statistics function
  • test_catalog_query_performance — Test query performance meets targets (< 100ms)
  • test_transformation_performance — Test transformation performance (< 1ms per model)
  • test_filtered_query_performance — Test performance of filtered queries
  • test_count_query_performance — Test performance of count queries
  • test_empty_result_handling — Test handling of queries that return no results
  • test_special_characters_in_search — Test search with special characters
  • test_transform_empty_list — Test batch transformation with empty list
  • test_all_models_have_required_fields — Test that all models have required fields for API
  • test_transformed_models_match_api_schema — Test that transformed models match expected API schema
  • test_provider_join_integrity — Test that provider join always works

tests/db/test_payments.py

20 tests

  • TestCreatePayment (4 tests)
    • test_create_payment_success — Test successful payment creation
    • test_create_payment_with_metadata — Test payment creation with metadata
    • test_create_payment_no_data_returned — Test handling when no data returned
    • test_create_payment_exception — Test exception handling during creation
  • TestRetrievePayments (7 tests)
    • test_get_payment_success — Test retrieving payment by ID
    • test_get_payment_not_found — Test getting non-existent payment
    • test_get_payment_by_stripe_intent_payment_intent — Test getting payment by Stripe payment intent ID
    • test_get_payment_by_stripe_intent_session_id — Test getting payment by Stripe session ID (fallback)
    • test_get_user_payments — Test getting all payments for a user
    • test_get_user_payments_with_status_filter — Test getting user payments filtered by status
    • test_get_recent_payments — Test getting recent payments (admin function)
  • TestUpdatePayments (4 tests)
    • test_update_payment_status_to_completed — Test updating payment status to completed
    • test_update_payment_status_sets_checkout_session_id — Ensure stripe session updates use the checkout session column
    • test_update_payment_status_to_failed_with_error — Test updating payment status to failed with error message
    • test_update_payment_metadata — Test updating payment metadata
  • TestDeletePayment (2 tests)
    • test_delete_payment_success — Test successful payment deletion
    • test_delete_payment_failure — Test payment deletion failure
  • TestPaymentStatistics (3 tests)
    • test_get_payment_statistics_user_specific — Test getting payment statistics for specific user
    • test_get_total_revenue — Test getting total revenue statistics
    • test_get_payment_trends — Test getting payment trends over time

tests/db/test_payments_retry.py

24 tests

  • TestCreatePaymentWithRetry (7 tests)
    • test_create_payment_success — Test successful payment creation with retry wrapper
    • test_create_payment_no_data_returned — Test payment creation fails when no data returned
    • test_create_payment_http2_connection_error — Test HTTP/2 ConnectionTerminated error during payment creation
    • test_create_payment_stream_id_error — Test StreamIDTooLowError during payment creation
    • test_create_payment_send_headers_error — Test SEND_HEADERS LocalProtocolError during payment creation
    • test_create_payment_with_metadata — Test payment creation with custom metadata
    • test_create_payment_amount_calculation — Test payment amount is correctly converted to cents
  • TestUpdatePaymentStatusWithRetry (7 tests)
    • test_update_payment_status_success — Test successful payment status update
    • test_update_payment_status_to_failed — Test updating payment status to failed with error message
    • test_update_payment_status_no_data_returned — Test update fails when no data returned
    • test_update_payment_status_http2_error — Test HTTP/2 connection error during status update
    • test_update_payment_status_stream_id_error — Test StreamIDTooLowError during status update
    • test_update_payment_status_remote_protocol_error — Test RemoteProtocolError during status update
    • test_update_payment_status_with_stripe_fields — Test status update with Stripe payment intent and session IDs
  • TestPaymentsIntegration (3 tests)
    • test_payment_creation_and_update_flow — Test full payment flow: create -> update status
    • test_resilience_to_transient_errors — Test that execute_with_retry handles transient errors gracefully
    • test_multiple_payment_operations — Test multiple payment operations with consistent retry behavior
  • TestPaymentErrorHandling (7 tests)
    • test_create_payment_generic_exception — Test generic exception handling during payment creation
    • test_get_payment_with_retry_success — Test get_payment with retry wrapper
    • test_get_payment_http2_error — Test get_payment with HTTP/2 connection error
    • test_get_payment_not_found — Test get_payment when payment not found
    • test_update_payment_status_generic_exception — Test generic exception handling during status update
    • test_create_payment_with_zero_amount — Test payment creation with zero amount
    • test_create_payment_with_large_amount — Test payment creation with large amount

tests/db/test_ping.py

2 tests

  • TestPing (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/db/test_plans.py

20 tests

  • test_get_all_plans_returns_active_sorted
  • test_get_plan_by_id_converts_features_dict_to_list
  • test_get_user_plan_combines_user_and_plan
  • test_assign_user_plan_deactivates_existing_and_updates_user
  • test_check_plan_entitlements_no_plan_defaults
  • test_check_plan_entitlements_expired_plan_marks_inactive_and_user_expired
  • test_check_plan_entitlements_active_plan_allows_feature
  • test_get_user_usage_within_plan_limits_aggregates
  • test_enforce_plan_limits_checks_and_env_multiplier
  • test_get_subscription_plans_active_only
  • test_get_all_plans_error_returns_empty
  • test_get_plan_id_by_tier_finds_matching_plan
  • test_get_plan_id_by_tier_returns_none_when_not_found
  • test_get_plan_id_by_tier_only_returns_active_plans
  • test_get_plan_id_by_tier_error_returns_none
  • test_usage_cache_hit_skips_database — Second call with same user_id should use cache (not hit database)
  • test_usage_cache_different_users_separate_entries — Different user IDs should have separate cache entries
  • test_clear_usage_cache_all — clear_usage_cache() with no arguments should clear entire cache
  • test_invalidate_usage_cache_specific_user — invalidate_usage_cache should clear cache for specific user
  • test_get_usage_cache_stats — get_usage_cache_stats should return cache statistics

tests/db/test_postgrest_schema.py

2 tests

  • test_refresh_postgrest_schema_cache_falls_back_to_direct_notify
  • test_refresh_postgrest_schema_cache_returns_false_without_dsn

tests/db/test_ranking.py

2 tests

  • TestRanking (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/db/test_rate_limits.py

14 tests

  • test_get_user_rate_limits_new_path
  • test_get_user_rate_limits_legacy_fallback
  • test_get_user_rate_limits_none_when_no_rows
  • test_set_user_rate_limits_insert_then_update
  • test_check_rate_limit_allowed_when_under
  • test_check_rate_limit_blocks_on_request_minute
  • test_check_rate_limit_blocks_on_tokens_minute
  • test_update_rate_limit_usage_inserts_then_updates
  • test_get_environment_usage_summary
  • test_get_update_rate_limit_config_and_list
  • test_bulk_update_rate_limit_configs
  • test_get_rate_limit_usage_stats_minute
  • test_get_system_rate_limit_stats
  • test_create_and_get_rate_limit_alerts

tests/db/test_rate_limits_configs_migration.py

10 tests

  • test_module_imports — Smoke test to ensure the module imports correctly after migration
  • TestGetUserRateLimitsConfigsMigration (3 tests)
    • test_get_rate_limits_from_configs_table — Test retrieving rate limits from rate_limit_configs table
    • test_get_rate_limits_returns_none_when_no_config — Test that None is returned when no rate limits are configured
    • test_get_rate_limits_handles_missing_api_key — Test handling when API key doesn't exist
  • TestSetUserRateLimitsConfigsMigration (3 tests)
    • test_set_rate_limits_creates_new_config — Test creating new rate limit config in rate_limit_configs table
    • test_set_rate_limits_updates_existing_config — Test updating existing rate limit config
    • test_set_rate_limits_raises_error_for_missing_key — Test that ValueError is raised when API key not found
  • TestRateLimitsLegacyTableRemoval (2 tests)
    • test_no_fallback_to_legacy_table — Test that get_user_rate_limits does NOT fallback to legacy rate_limits table.
    • test_set_user_rate_limits_uses_configs_table — Verify that set_user_rate_limits uses rate_limit_configs table.
  • TestRateLimitConfigsIntegration (1 tests)
    • test_full_lifecycle_create_retrieve_update — Test full lifecycle: create config, retrieve it, update it

tests/db/test_referral.py

16 tests

  • TestReferralCodeGeneration (3 tests)
    • test_generate_referral_code_length — Test referral code has correct length
    • test_generate_referral_code_uppercase_digits — Test referral code contains only uppercase and digits
    • test_generate_referral_code_uniqueness — Test that multiple codes are likely unique
  • TestUserModel (7 tests)
    • test_user_init — Test User initialization
    • test_user_init_with_referral — Test User initialization with referral code
    • test_user_to_dict — Test User to_dict method
    • test_user_get_remaining_referral_uses_zero_used — Test remaining uses when none used
    • test_user_get_remaining_referral_uses_some_used — Test remaining uses when partially used
    • test_user_get_remaining_referral_uses_max_used — Test remaining uses when max reached
    • test_user_get_remaining_referral_uses_over_max — Test remaining uses when over max (edge case)
  • TestCouponUsageModel (3 tests)
    • test_coupon_usage_to_dict — Test CouponUsage to_dict method
    • test_coupon_usage_default_bonus — Test CouponUsage default bonus amount
    • test_coupon_usage_default_is_valid — Test CouponUsage default is_valid
  • TestPurchaseModel (3 tests)
    • test_purchase_to_dict — Test Purchase to_dict method
    • test_purchase_default_referral_bonus — Test Purchase default referral bonus
    • test_purchase_default_referral_code — Test Purchase default referral code

tests/db/test_roles.py

34 tests

  • TestPermissionChecking (7 tests)
    • test_check_user_permission_granted — Test permission check when permission is granted
    • test_check_user_permission_denied — Test permission check when permission is denied
    • test_check_user_permission_no_data — Test permission check when no data is returned
    • test_check_user_permission_error — Test permission check with database error
    • test_get_user_permissions_success — Test retrieving all user permissions
    • test_get_user_permissions_empty — Test retrieving permissions when user has none
    • test_get_user_permissions_error — Test error handling when retrieving permissions
  • TestUserRoleManagement (8 tests)
    • test_get_user_role_success — Test retrieving user's role
    • test_get_user_role_not_found — Test retrieving role for non-existent user
    • test_get_user_role_error — Test error handling when getting user role
    • test_update_user_role_success — Test successfully updating user's role
    • test_update_user_role_with_audit_log — Test role update creates audit log entry
    • test_update_user_role_invalid_role — Test updating to invalid role
    • test_update_user_role_no_data — Test role update when no data is returned
    • test_update_user_role_error — Test error handling during role update
  • TestRoleAuditLog (4 tests)
    • test_get_role_audit_log_all — Test retrieving all audit log entries
    • test_get_role_audit_log_by_user — Test retrieving audit log for specific user
    • test_get_role_audit_log_empty — Test retrieving empty audit log
    • test_get_role_audit_log_error — Test error handling when retrieving audit log
  • TestUsersByRole (4 tests)
    • test_get_users_by_role_success — Test retrieving users with specific role
    • test_get_users_by_role_admin — Test retrieving admin users
    • test_get_users_by_role_empty — Test retrieving users when none have role
    • test_get_users_by_role_error — Test error handling when retrieving users by role
  • TestRolePermissionsManagement (8 tests)
    • test_get_role_permissions_success — Test retrieving all permissions for a role
    • test_get_role_permissions_empty — Test retrieving permissions when role has none
    • test_add_role_permission_success — Test adding permission to a role
    • test_add_role_permission_no_data — Test adding permission when no data is returned
    • test_add_role_permission_error — Test error handling when adding permission
    • test_remove_role_permission_success — Test removing permission from a role
    • test_remove_role_permission_no_data — Test removing permission when no data is returned
    • test_remove_role_permission_error — Test error handling when removing permission
  • TestRoleConstants (3 tests)
    • test_user_role_constants — Test role constant values
    • test_role_constants_are_strings — Test all role constants are strings
    • test_role_constants_uniqueness — Test all role constants are unique

tests/db/test_shared_chats.py

16 tests

  • test_generate_share_token_is_unique — Test that generate_share_token produces unique tokens
  • test_generate_share_token_is_url_safe — Test that generated tokens are URL-safe
  • test_create_shared_chat_success — Test successfully creating a shared chat
  • test_create_shared_chat_with_expiry — Test creating shared chat with expiration date
  • test_get_shared_chat_by_token_success — Test getting shared chat by token
  • test_get_shared_chat_by_token_not_found — Test getting non-existent shared chat
  • test_get_shared_chat_by_token_expired — Test getting expired shared chat returns None
  • test_get_user_shared_chats_success — Test getting user's shared chats
  • test_get_user_shared_chats_empty — Test getting shared chats when user has none
  • test_delete_shared_chat_success — Test deleting a shared chat
  • test_delete_shared_chat_not_found — Test deleting non-existent shared chat
  • test_verify_session_ownership_true — Test verifying session ownership returns True for owner
  • test_verify_session_ownership_false — Test verifying session ownership returns False for non-owner
  • test_check_share_rate_limit_within_limit — Test rate limit check when within limit
  • test_check_share_rate_limit_exceeded — Test rate limit check when limit exceeded
  • test_check_share_rate_limit_custom_limit — Test rate limit check with custom limit

tests/db/test_subscription_products.py

15 tests

  • TestSubscriptionProducts (15 tests)
    • test_get_tier_from_product_id_pro — Test getting tier from PRO product ID
    • test_get_tier_from_product_id_max — Test getting tier from MAX product ID
    • test_get_tier_from_product_id_not_found — Test getting tier from unknown product ID defaults to basic
    • test_get_tier_from_product_id_recovers_from_schema_cache_miss — Ensure schema cache refresh is attempted when PostgREST cannot find the table.
    • test_get_tier_from_product_id_schema_cache_refresh_failure — Ensure we fall back to basic tier when schema cache refresh cannot run.
    • test_get_credits_from_tier_pro — Test getting credits for PRO tier
    • test_get_credits_from_tier_max — Test getting credits for MAX tier
    • test_get_credits_from_tier_not_found — Test getting credits from unknown tier defaults to 0
    • test_get_subscription_product — Test getting full product configuration
    • test_get_all_active_products — Test getting all active products
    • test_add_subscription_product — Test adding new subscription product
    • test_add_subscription_product_failure — Test adding product failure
    • test_update_subscription_product — Test updating subscription product
    • test_update_subscription_product_no_fields — Test updating product with no fields returns False
    • test_database_error_handling — Test that database errors are handled gracefully

tests/db/test_tiered_credits.py

11 tests

  • TestTieredCreditDeduction (4 tests)
    • test_deduct_from_allowance_first — Verify allowance is consumed before purchased credits.
    • test_deduct_split_across_both — Test deduction that spans both allowance and purchased credits.
    • test_deduct_from_purchased_when_no_allowance — Test deduction when allowance is 0 (basic tier or depleted).
    • test_insufficient_total_credits_error — Verify error when total balance is insufficient.
  • TestAllowanceReset (2 tests)
    • test_reset_sets_correct_amount_pro — PRO tier should get $15 allowance on reset.
    • test_reset_sets_correct_amount_max — MAX tier should get $150 allowance on reset.
  • TestSubscriptionCancellation (2 tests)
    • test_allowance_forfeited_on_cancel — Verify allowance is zeroed on cancellation.
    • test_purchased_credits_preserved_on_cancel — Verify purchased credits remain after cancellation.
  • TestAllowanceFromTier (3 tests)
    • test_get_pro_allowance — PRO tier should return $15 allowance.
    • test_get_max_allowance — MAX tier should return $150 allowance.
    • test_get_basic_allowance — Basic tier should return $0 allowance.

tests/db/test_trials.py

22 tests

  • TestStartTrial (5 tests)
    • test_start_trial_success — Test successfully starting a trial
    • test_start_trial_custom_duration — Test starting trial with custom duration
    • test_start_trial_api_key_not_found — Test starting trial with invalid API key
    • test_start_trial_database_error — Test error handling during trial start
    • test_start_trial_exception — Test exception handling
  • TestGetTrialStatus (4 tests)
    • test_get_trial_status_active — Test getting status for active trial
    • test_get_trial_status_expired — Test getting status for expired trial
    • test_get_trial_status_not_trial — Test getting status for non-trial key
    • test_get_trial_status_api_key_not_found — Test getting status with invalid API key
  • TestConvertTrialToPaid (4 tests)
    • test_convert_trial_to_paid_success — Test successfully converting trial to paid
    • test_convert_trial_different_plans — Test converting to different plan types
    • test_convert_trial_api_key_not_found — Test converting with invalid API key
    • test_convert_trial_exception — Test exception handling during conversion
  • TestTrackTrialUsage (3 tests)
    • test_track_trial_usage_success — Test successfully tracking trial usage
    • test_track_trial_usage_custom_requests — Test tracking usage with custom request count
    • test_track_trial_usage_api_key_not_found — Test tracking usage with invalid API key
  • TestTrialAnalytics (5 tests)
    • test_get_trial_analytics_success — Test getting trial analytics
    • test_get_trial_analytics_no_trials — Test analytics with no trials
    • test_get_trial_analytics_no_data — Test analytics with no data
    • test_get_trial_analytics_calculates_averages — Test analytics calculates correct averages
    • test_get_trial_analytics_exception — Test exception handling in analytics
  • TestTrialsIntegration (1 tests)
    • test_complete_trial_workflow — Test complete trial workflow: start -> use -> convert

tests/db/test_user_credit_updates.py

6 tests

  • TestReducedTrialCredits (2 tests)
    • test_create_enhanced_user_default_credits — Test that default credits are $5 with $1/day limit enforced
    • test_create_enhanced_user_custom_credits — Test that custom credit amounts can still be specified
  • TestDailyLimitEnforcement (3 tests)
    • test_deduct_credits_checks_daily_limit — Test that deduct_credits calls enforce_daily_usage_limit
    • test_deduct_credits_raises_on_daily_limit_exceeded — Test that DailyUsageLimitExceeded is converted to ValueError
    • test_admin_users_bypass_daily_limit — Test that admin users bypass daily usage limits
  • TestCreditDeductionWithDailyLimits (1 tests)
    • test_multiple_small_deductions_within_limit — Test multiple small deductions that stay within $1 daily limit

tests/db/test_users.py

61 tests

  • test_create_enhanced_user_creates_trial_and_primary
  • test_get_user_prefers_new_api_keys_then_legacy
  • test_get_user_by_id_and_privy
  • test_add_credits_and_deduct_credits
  • test_get_all_users_delete_user_count
  • test_record_usage_and_metrics
  • test_admin_monitor_data
  • test_admin_monitor_data_deduplication — Test that duplicate records in activity_log and usage_records are correctly deduplicated.
  • test_admin_monitor_data_deduplication_api_key_only — Test deduplication when legacy records only have api_key (no user_id).
  • test_update_and_get_user_profile
  • test_get_user_profile_returns_credits_in_cents — Test that get_user_profile returns credits in cents, not dollars.
  • test_get_user_profile_pro_tier_allowance_in_cents — Test Pro tier user returns $15.00 allowance as 1500 cents.
  • test_get_user_profile_max_tier_allowance_in_cents — Test Max tier user returns $150.00 allowance as 15000 cents.
  • test_get_user_profile_basic_tier_zero_allowance — Test Basic tier user returns zero allowance correctly.
  • test_get_user_profile_fractional_cents — Test that fractional dollar amounts are correctly converted to integer cents.
  • test_get_user_profile_null_credit_fields — Test that null/missing credit fields default to 0 cents.
  • test_get_user_profile_large_credits_in_cents — Test that large credit amounts are correctly converted to cents.
  • test_mark_welcome_email_sent_and_delete_user_account
  • test_create_enhanced_user_with_privy_id — Test user creation with privy_user_id
  • test_create_enhanced_user_failure_no_data — Test user creation when insert returns no data
  • test_create_enhanced_user_exception_handling — Test exception handling in create_enhanced_user
  • test_get_user_not_found — Test get_user when API key doesn't exist
  • test_get_user_exception_handling — Test get_user exception handling
  • test_get_user_by_id_not_found — Test get_user_by_id when user doesn't exist
  • test_get_user_by_id_exception_handling — Test get_user_by_id exception handling
  • test_get_user_by_username_not_found — Test get_user_by_username when user doesn't exist
  • test_get_user_by_username_found — Test get_user_by_username when user exists
  • test_get_user_by_username_exception_handling — Test get_user_by_username exception handling
  • test_add_credits_to_user_success — Test add_credits_to_user successfully adds credits
  • test_add_credits_to_user_exception_handling — Test add_credits_to_user exception handling
  • test_add_credits_success — Test add_credits function
  • test_deduct_credits_with_metadata — Test deduct_credits with metadata
  • test_deduct_credits_user_not_found — Test deduct_credits when user not found
  • test_deduct_credits_exception_handling — Test deduct_credits exception handling
  • test_get_all_users_exception_handling — Test get_all_users exception handling
  • test_delete_user_exception_handling — Test delete_user exception handling
  • test_get_user_count_exception_handling — Test get_user_count exception handling
  • test_record_usage_exception_handling — Test record_usage exception handling
  • test_record_usage_with_latency — Test record_usage with latency parameter
  • test_get_user_usage_metrics_no_user — Test get_user_usage_metrics when user not found
  • test_get_user_usage_metrics_exception_handling — Test get_user_usage_metrics exception handling
  • test_get_admin_monitor_data_exception_handling — Test get_admin_monitor_data exception handling
  • test_update_user_profile_exception_handling — Test update_user_profile exception handling
  • test_get_user_profile_not_found — Test get_user_profile when user not found
  • test_get_user_profile_exception_handling — Test get_user_profile exception handling
  • test_mark_welcome_email_sent_exception_handling — Test mark_welcome_email_sent exception handling
  • test_delete_user_account_not_found — Test delete_user_account when user not found
  • test_delete_user_account_exception_handling — Test delete_user_account exception handling
  • test_user_cache_hit_on_second_call — Second call to get_user with same API key should use cache (not hit database)
  • test_user_cache_miss_for_different_keys — Different API keys should have separate cache entries
  • test_clear_user_cache_specific_key — clear_user_cache(api_key) should clear only that key
  • test_clear_user_cache_all — clear_user_cache() with no arguments should clear entire cache
  • test_invalidate_user_cache — invalidate_user_cache should clear cache for specific user
  • test_user_cache_not_found_user — Cache should not cache None returns for invalid keys
  • test_is_temporary_api_key_empty — Test _is_temporary_api_key with empty/None values
  • test_is_temporary_api_key_short_gw_live — Test is_temporary_api_key detects short gw_live keys as temporary
  • test_is_temporary_api_key_proper_length — Test _is_temporary_api_key does not flag proper-length keys as temporary
  • test_is_temporary_api_key_non_gw_live_prefix — Test is_temporary_api_key ignores keys without gw_live prefix
  • test_is_temporary_api_key_realistic_keys — Test _is_temporary_api_key with realistic key formats
  • test_migrate_legacy_api_key_skips_temporary_keys — Test _migrate_legacy_api_key does not migrate temporary keys
  • test_get_user_detects_temporary_key_in_legacy — Test get_user correctly detects and handles temporary keys in legacy lookup

tests/db/test_webhook_events_retry.py

19 tests

  • TestIsEventProcessedWithRetry (5 tests)
    • test_is_event_processed_success — Test successful event check (event exists)
    • test_is_event_processed_not_found — Test event not found
    • test_is_event_processed_http2_error_retry — Test HTTP/2 connection error triggers retry via execute_with_retry
    • test_is_event_processed_stream_id_error — Test StreamIDTooLowError is handled by execute_with_retry
    • test_is_event_processed_generic_error — Test generic database errors
  • TestRecordProcessedEventWithRetry (4 tests)
    • test_record_processed_event_success — Test successful event recording
    • test_record_processed_event_no_data_returned — Test recording fails when no data returned
    • test_record_processed_event_http2_error — Test HTTP/2 connection error during recording
    • test_record_processed_event_send_headers_error — Test SEND_HEADERS LocalProtocolError
  • TestGetProcessedEventWithRetry (3 tests)
    • test_get_processed_event_success — Test successful event retrieval
    • test_get_processed_event_not_found — Test event not found
    • test_get_processed_event_http2_error — Test HTTP/2 connection error during retrieval
  • TestCleanupOldEventsWithRetry (4 tests)
    • test_cleanup_old_events_success — Test successful cleanup
    • test_cleanup_old_events_no_data — Test cleanup with no events to delete
    • test_cleanup_old_events_http2_error — Test HTTP/2 connection error during cleanup
    • test_cleanup_old_events_custom_days — Test cleanup with custom retention period
  • TestWebhookEventsIntegration (3 tests)
    • test_idempotency_flow_new_event — Test full flow: check event doesn't exist, then record it
    • test_idempotency_flow_duplicate_event — Test full flow: detect duplicate event
    • test_resilience_to_transient_errors — Test that execute_with_retry handles transient errors gracefully

tests/e2e/

8 files, 125 tests

tests/e2e/test_allenai_models_e2e.py

15 tests

  • TestAllenAIModelsE2E (11 tests)
    • test_allenai_model_basic_request — Test basic chat completion request for each AllenAI model.
    • test_allenai_model_streaming — Test streaming chat completion for each AllenAI model.
    • test_allenai_model_with_system_prompt — Test AllenAI models with system prompt.
    • test_allenai_model_conversation_history — Test AllenAI models with conversation history.
    • test_allenai_model_with_temperature — Test AllenAI models with custom temperature.
    • test_allenai_model_with_top_p — Test AllenAI models with top_p parameter.
    • test_olmo_instruct_follows_instructions — Test that OLMo instruct model follows instructions.
    • test_thinking_models_reasoning_task — Test thinking models with a reasoning task.
    • test_allenai_model_with_openrouter_provider — Test AllenAI models with explicit OpenRouter provider.
    • test_allenai_model_response_has_usage — Test that AllenAI model responses include usage information.
    • test_allenai_model_invalid_should_fail — Test that invalid AllenAI model ID returns appropriate error.
  • TestAllenAIModelsStreamingE2E (2 tests)
    • test_streaming_with_custom_params — Test streaming with custom temperature and max_tokens.
    • test_streaming_response_structure — Test that streaming responses have correct SSE structure.
  • TestAllenAIModelsDeveloperRoleE2E (2 tests)
    • test_allenai_model_with_developer_role — Test AllenAI models with developer role instead of system role.
    • test_allenai_model_developer_role_streaming — Test streaming with developer role for AllenAI models.

tests/e2e/test_chat_completions_e2e.py

21 tests

  • TestChatCompletionsE2E (18 tests)
    • test_chat_completions_basic_request — Test basic chat completion request and response.
    • test_chat_completions_with_system_prompt — Test chat completion with system prompt.
    • test_chat_completions_with_all_parameters — Test chat completion with all optional parameters.
    • test_chat_completions_streaming — Test streaming chat completion.
    • test_chat_completions_with_provider_openrouter — Test chat completion with explicit OpenRouter provider.
    • test_chat_completions_with_provider_featherless — Test chat completion with Featherless provider.
    • test_chat_completions_with_provider_fireworks — Test chat completion with Fireworks provider.
    • test_chat_completions_missing_api_key — Test chat completion without API key.
    • test_chat_completions_empty_messages — Test chat completion with empty messages array.
    • test_chat_completions_invalid_role — Test chat completion with invalid message role.
    • test_chat_completions_empty_content — Test chat completion with empty message content.
    • test_chat_completions_multiple_messages — Test chat completion with conversation history.
    • test_chat_completions_with_tools — Test chat completion with function calling tools.
    • test_chat_completions_response_format_json — Test chat completion with JSON response format.
    • test_chat_completions_very_long_message — Test chat completion with very long message.
    • test_chat_completions_session_id_parameter — Test chat completion with session_id query parameter.
    • test_chat_completions_default_max_tokens — Test that max_tokens defaults to 950.
    • test_chat_completions_default_temperature — Test that temperature defaults to 1.0.
  • TestChatCompletionsDeveloperRoleE2E (3 tests)
    • test_chat_completions_with_developer_role — Test chat completion with developer role message.
    • test_chat_completions_developer_role_streaming — Test streaming chat completion with developer role.
    • test_reasoning_models_with_developer_role — Test OpenAI reasoning models (o1/o3) with developer role.

tests/e2e/test_images_e2e.py

17 tests

  • TestImagesE2E (17 tests)
    • test_images_basic_request — Test basic image generation request and response.
    • test_images_with_single_image — Test image generation with single image.
    • test_images_with_multiple_images — Test image generation with multiple images.
    • test_images_different_sizes — Test image generation with different sizes.
    • test_images_with_deepinfra_provider — Test image generation with DeepInfra provider.
    • test_images_with_google_vertex_provider — Test image generation with Google Vertex AI provider.
    • test_images_with_fal_provider — Test image generation with Fal.ai provider.
    • test_images_invalid_provider — Test image generation with unsupported provider.
    • test_images_missing_prompt — Test image generation without prompt.
    • test_images_empty_prompt — Test image generation with empty prompt.
    • test_images_missing_api_key — Test image generation without API key.
    • test_images_very_long_prompt — Test image generation with very long prompt.
    • test_images_special_characters_in_prompt — Test image generation with special characters in prompt.
    • test_images_invalid_size — Test image generation with invalid size.
    • test_images_invalid_number_of_images — Test image generation with invalid n parameter.
    • test_images_default_size — Test image generation without size (should use default).
    • test_images_default_provider — Test image generation without provider (should default to deepinfra).

tests/e2e/test_messages_e2e.py

19 tests

  • TestMessagesE2E (19 tests)
    • test_messages_basic_request — Test basic messages API request and response.
    • test_messages_with_system_prompt — Test messages API with system prompt (Anthropic style).
    • test_messages_with_all_parameters — Test messages API with all optional parameters.
    • test_messages_streaming — Test streaming messages API.
    • test_messages_with_provider_openrouter — Test messages API with explicit OpenRouter provider.
    • test_messages_with_provider_featherless — Test messages API with Featherless provider.
    • test_messages_missing_api_key — Test messages API without API key.
    • test_messages_missing_max_tokens — Test messages API without required max_tokens (Anthropic requirement).
    • test_messages_invalid_max_tokens — Test messages API with invalid max_tokens.
    • test_messages_empty_messages — Test messages API with empty messages array.
    • test_messages_invalid_role — Test messages API with invalid message role (Anthropic only allows user/assistant).
    • test_messages_empty_content — Test messages API with empty message content.
    • test_messages_conversation_history — Test messages API with conversation history.
    • test_messages_with_tools — Test messages API with tool definitions (Claude tool use).
    • test_messages_with_tool_choice — Test messages API with tool_choice parameter.
    • test_messages_with_stop_sequences — Test messages API with stop_sequences (Claude feature).
    • test_messages_with_top_k — Test messages API with top_k parameter (Anthropic-specific).
    • test_messages_very_long_content — Test messages API with very long message.
    • test_messages_metadata — Test messages API with metadata parameter.

tests/e2e/test_responses_e2e.py

15 tests

  • TestResponsesE2E (15 tests)
    • test_responses_basic_request — Test basic unified responses endpoint request and response.
    • test_responses_with_all_parameters — Test responses endpoint with all optional parameters.
    • test_responses_streaming — Test streaming responses endpoint.
    • test_responses_with_provider — Test responses endpoint with specific provider.
    • test_responses_with_json_response_format — Test responses endpoint with JSON response format.
    • test_responses_multimodal_input — Test responses endpoint with multimodal input (text + image).
    • test_responses_multiple_input_items — Test responses endpoint with multiple input items.
    • test_responses_missing_api_key — Test responses endpoint without API key.
    • test_responses_empty_input — Test responses endpoint with empty input array.
    • test_responses_missing_model — Test responses endpoint without model.
    • test_responses_with_tools — Test responses endpoint with tool definitions.
    • test_responses_very_long_input — Test responses endpoint with very long input.
    • test_responses_default_max_tokens — Test that max_tokens defaults to 950.
    • test_responses_with_featherless_provider — Test responses endpoint with Featherless provider.
    • test_responses_with_fireworks_provider — Test responses endpoint with Fireworks provider.

tests/e2e/test_simplismart_models_e2e.py

15 tests

  • TestSimplismartModelsE2E (8 tests)
    • test_simplismart_model_basic_request — Test basic chat completion request for each Simplismart model.
    • test_simplismart_model_streaming — Test streaming chat completion for each Simplismart model.
    • test_simplismart_model_with_system_prompt — Test Simplismart models with system prompt.
    • test_simplismart_model_conversation_history — Test Simplismart models with conversation history.
    • test_simplismart_model_with_temperature — Test Simplismart models with custom temperature.
    • test_simplismart_model_with_top_p — Test Simplismart models with top_p parameter.
    • test_simplismart_model_response_has_usage — Test that Simplismart model responses include usage information.
    • test_simplismart_model_invalid_should_fail — Test that invalid Simplismart model ID returns appropriate error.
  • TestSimplismartModelAliasesE2E (1 tests)
    • test_simplismart_alias_resolution — Test that model aliases are correctly resolved.
  • TestSimplismartStreamingE2E (2 tests)
    • test_streaming_with_custom_params — Test streaming with custom temperature and max_tokens.
    • test_streaming_response_structure — Test that streaming responses have correct SSE structure.
  • TestSimplismartLlamaModelsE2E (2 tests)
    • test_llama_3_1_8b_instruction_following — Test that Llama 3.1 8B follows instructions.
    • test_llama_3_3_70b_reasoning_task — Test Llama 3.3 70B with a reasoning task.
  • TestSimplismartGemmaModelsE2E (1 tests)
    • test_gemma_3_27b_basic — Test Gemma 3 27B basic functionality.
  • TestSimplismartQwenModelsE2E (1 tests)
    • test_qwen_2_5_32b_basic — Test Qwen 2.5 32B basic functionality.

tests/e2e/test_streaming_providers_e2e.py

19 tests

  • TestStreamingE2E (6 tests)
    • test_streaming_chat_completions — Test streaming on chat completions endpoint.
    • test_streaming_messages — Test streaming on messages endpoint.
    • test_streaming_responses — Test streaming on responses endpoint.
    • test_streaming_with_custom_parameters — Test streaming with custom temperature and tokens.
    • test_non_streaming_chat_completions — Test non-streaming chat completions.
    • test_non_streaming_messages — Test non-streaming messages endpoint.
  • TestProviderParameterE2E (13 tests)
    • test_provider_openrouter_chat — Test explicit OpenRouter provider on chat endpoint.
    • test_provider_featherless_chat — Test explicit Featherless provider on chat endpoint.
    • test_provider_fireworks_chat — Test explicit Fireworks provider on chat endpoint.
    • test_provider_together_chat — Test explicit Together provider on chat endpoint.
    • test_provider_huggingface_chat — Test explicit HuggingFace provider on chat endpoint.
    • test_provider_openrouter_messages — Test explicit OpenRouter provider on messages endpoint.
    • test_provider_openrouter_responses — Test explicit OpenRouter provider on responses endpoint.
    • test_provider_deepinfra_images — Test explicit DeepInfra provider on images endpoint.
    • test_provider_default_fallback — Test default provider fallback when not specified.
    • test_provider_auto_detection_from_model_id — Test provider auto-detection from model ID.
    • test_provider_alias_hug_to_huggingface — Test provider alias normalization (hug -> huggingface).
    • test_provider_with_streaming — Test provider parameter works with streaming.
    • test_multiple_providers_in_sequence — Test using different providers in sequence.

tests/e2e/test_sybil_models_e2e.py

4 tests

  • TestSybilModelsE2E (4 tests)
    • test_sybil_model_basic_request — Test basic chat completion request for each Sybil model.
    • test_sybil_model_streaming — Test streaming chat completion for each Sybil model.
    • test_sybil_model_with_tools — Test Sybil model with tool calling (for models that support it).
    • test_sybil_model_with_json_mode — Test Sybil model with JSON mode (for models that support it).

tests/health/

1 files, 19 tests

tests/health/test_gateway_health.py

19 tests

  • TestGatewayHealthChecker (10 tests)
    • test_gateway_config_exists — Test that gateway configuration is properly defined
    • test_all_required_gateways_configured — Test that all expected gateways are in the configuration
    • test_gateway_config_has_required_fields — Test that each gateway config has required fields
    • test_build_headers_bearer_token — Test header building for bearer token auth
    • test_build_headers_no_api_key — Test header building when no API key is provided
    • test_cache_test_with_empty_cache — Test cache testing with empty cache
    • test_cache_test_with_valid_cache — Test cache testing with valid cached models
    • test_clear_gateway_cache — Test clearing gateway cache
    • test_gateway_config_min_expected_models — Test that each gateway has reasonable min_expected_models
    • test_gateway_urls_are_valid — Test that all gateway URLs are properly formatted
  • TestGatewayEndpointChecks (2 tests)
    • test_endpoint_check_timeout_handling — Test that timeout is handled properly
    • test_endpoint_check_response_parsing — Test proper response parsing for different formats
  • TestGatewayHealthCheckFlow (3 tests)
    • test_build_headers_for_all_gateway_types — Test header building for all configured gateway types
    • test_comprehensive_check_result_structure — Test structure of comprehensive check results
    • test_gateway_count_consistency — Test that gateway counts add up correctly
  • TestGatewayCache (2 tests)
    • test_cache_with_expired_timestamp — Test cache age calculation for expired cache
    • test_cache_model_count_validation — Test that cache validates minimum model count
  • TestGatewayIntegration (2 tests)
    • test_openrouter_endpoint_reachable — Test that OpenRouter endpoint is reachable
    • test_script_runs_without_errors — Test that the full script runs without exceptions

tests/integration/

34 files, 326 tests

tests/integration/test_allenai_models.py

7 tests

  • TestAllenAIModelsDirectOpenRouter (5 tests)
    • test_model_basic_response — Test that each AllenAI model responds to a basic prompt.
    • test_model_streaming — Test that each AllenAI model supports streaming.
    • test_thinking_models_reasoning — Test that thinking models provide reasoning for complex questions.
    • test_olmo_instruct_follows_instructions — Test that OLMo instruct model follows instructions well.
    • test_model_with_conversation_history — Test that models handle multi-turn conversations.
  • TestAllenAIModelsViaGatewayz (2 tests)
    • test_model_via_gatewayz — Test AllenAI models through Gatewayz API.
    • test_model_streaming_via_gatewayz — Test AllenAI models streaming through Gatewayz API.

tests/integration/test_auth_referral_integration.py

19 tests

  • test_auth_users — Create test users for auth testing
  • TestAuthRegistrationReferralIntegration (3 tests)
    • test_new_user_signup_with_referral_code — Test that new user signup WITH referral code:
    • test_new_user_signup_with_invalid_referral_code — Test that invalid referral code doesn't block signup
    • test_existing_user_login_ignores_referral_code — Test that existing users logging in don't process referral codes
  • TestReferralTrackingIntegration (3 tests)
    • test_track_referral_signup_creates_pending_record — Test that track_referral_signup creates pending referral record
    • test_store_referred_by_code_on_signup — Test that referred_by_code is stored correctly
    • test_multiple_signups_same_code — Test that multiple users can sign up with the same referral code
  • TestReferralNotificationIntegration (5 tests)
    • test_signup_notification_sent_to_referrer — Test that referrer receives email when someone signs up with their code
    • test_notification_failure_doesnt_block_signup — Test that notification failures don't block the signup process
    • test_notification_logs_attempt_before_sending — Test that notification logs attempt before sending email
    • test_notification_logs_success_with_email — Test that successful notification logs include email address
    • test_notification_logs_warning_on_failure — Test that failed notification logs warning with email address
  • TestReferralBackgroundTaskLogging (4 tests)
    • test_background_task_logs_when_referrer_has_no_email — Test that background task logs warning when referrer has no email
    • test_background_task_logs_success_with_email — Test that background task logs success with referrer email
    • test_background_task_logs_warning_when_notification_returns_false — Test that background task logs warning when notification returns False
    • test_background_task_logs_notification_error_with_details — Test that background task logs notification errors with full details
  • TestReferralValidationDuringSignup (3 tests)
    • test_validate_code_before_signup — Test validating referral code before signup
    • test_validate_prevents_self_referral_during_signup — Test that validation prevents self-referral during signup
    • test_validate_checks_usage_limit — Test that validation checks usage limit

tests/integration/test_braintrust_integration.py

5 tests

  • test_braintrust_import — Test that Braintrust can be imported
  • test_api_key_configured — Test that BRAINTRUST_API_KEY is configured
  • test_logger_initialization — Test that Braintrust logger can be initialized
  • test_basic_tracing — Test basic tracing functionality
  • test_endpoint_tracing_syntax — Test that the chat endpoint tracing syntax is correct

tests/integration/test_cerebras_qwen.py

8 tests

  • test_cerebras_qwen_api_direct — Direct test without pytest fixtures - useful for manual testing
  • TestCerebrasQwen (7 tests)
    • test_cerebras_qwen_basic — Test basic Qwen model inference
    • test_cerebras_qwen_streaming — Test streaming Qwen model inference
    • test_cerebras_qwen_system_prompt — Test Qwen model with system prompt
    • test_cerebras_qwen_multi_turn — Test multi-turn conversation with Qwen model
    • test_cerebras_qwen_temperature_variation — Test Qwen model with different temperature settings
    • test_cerebras_qwen_max_tokens — Test Qwen model with different max_tokens settings
    • test_cerebras_qwen_response_format — Test response format from Qwen model

tests/integration/test_chat_errors.py

21 tests

  • TestChatCompletionsErrors (21 tests)
    • test_invalid_api_key — Test invalid API key returns detailed error.
    • test_missing_authorization_header — Test missing authorization header returns detailed error.
    • test_model_not_found — Test model not found returns suggestions.
    • test_model_typo_suggests_correct_model — Test that typo in model name suggests correct model.
    • test_empty_messages_array — Test empty messages array error.
    • test_missing_messages_field — Test missing messages field error.
    • test_invalid_message_format — Test invalid message format error.
    • test_invalid_temperature — Test invalid parameter error for temperature.
    • test_negative_max_tokens — Test invalid max_tokens parameter.
    • test_insufficient_credits — Test insufficient credits error.
    • test_request_id_in_all_errors — Test that all errors include request_id.
    • test_request_id_propagation — Test that provided X-Request-ID is propagated.
    • test_timestamp_in_errors — Test that all errors include timestamp.
    • test_docs_url_in_errors — Test that appropriate errors include docs_url.
    • test_suggestions_in_errors — Test that errors include actionable suggestions.
    • test_provider_error — Test provider error returns detailed error.
    • test_streaming_error_format — Test that errors in streaming mode are properly formatted.
    • test_error_response_structure_complete — Test that error response has all expected fields.
    • test_concurrent_requests_unique_ids — Test that concurrent requests get unique request IDs.
    • test_internal_error_detailed — Test that internal errors return detailed responses.
    • test_no_error_fields_are_none_in_response — Test that no fields with None values are included in response.

tests/integration/test_chat_history_edge_cases.py

33 tests

  • TestDuplicateDetection (3 tests)
    • test_exact_duplicate_messages_same_content — Test that exact duplicate messages are detected
    • test_duplicate_messages_different_timestamps — Test duplicate messages with different timestamps (e.g., retry scenario)
    • test_similar_but_different_messages — Test that similar but different messages are both saved
  • TestHistoryInjection (3 tests)
    • test_history_injection_prepends_correctly — Test that history is prepended to new messages in correct order
    • test_empty_history_handling — Test handling of session with no history
    • test_history_respects_user_ownership — Test that users can only access their own chat history
  • TestMessageOrdering (2 tests)
    • test_messages_ordered_by_created_at — Test that messages are returned in chronological order
    • test_alternating_user_assistant_pattern — Test proper conversation flow alternating between user and assistant
  • TestEdgeCases (5 tests)
    • test_empty_message_content — Test handling of empty message content
    • test_very_long_message_content — Test handling of very long messages
    • test_special_characters_in_content — Test handling of special characters and unicode
    • test_null_model_field — Test handling when model field is None
    • test_zero_and_negative_tokens — Test handling of zero and negative token counts
  • TestMultimodalContent (2 tests)
    • test_text_only_content — Test standard text-only message
    • test_json_string_content — Test handling of JSON-like string content
  • TestSessionManagement (4 tests)
    • test_create_session_with_defaults — Test session creation with default values
    • test_update_session_title — Test updating session title
    • test_delete_session_soft_delete — Test that delete is a soft delete (sets is_active=False)
    • test_get_user_sessions_excludes_deleted — Test that deleted sessions are not returned
  • TestPagination (2 tests)
    • test_session_pagination — Test pagination of user sessions
    • test_session_ordering_by_updated_at — Test that sessions are ordered by updated_at desc
  • TestSearch (3 tests)
    • test_search_by_title — Test searching sessions by title
    • test_search_by_message_content — Test searching sessions by message content
    • test_search_case_insensitive — Test that search is case-insensitive
  • TestStatistics (2 tests)
    • test_session_stats_accuracy — Test that session stats are calculated correctly
    • test_stats_exclude_inactive_sessions — Test that stats exclude deleted (inactive) sessions
  • TestConcurrency (2 tests)
    • test_concurrent_message_saves — Test saving messages concurrently to same session
    • test_concurrent_session_creation — Test creating sessions concurrently
  • TestErrorHandling (3 tests)
    • test_get_nonexistent_session — Test getting a session that doesn't exist
    • test_update_nonexistent_session — Test updating a session that doesn't exist
    • test_delete_nonexistent_session — Test deleting a session that doesn't exist
  • TestRealWorldScenarios (2 tests)
    • test_full_conversation_flow — Test complete conversation lifecycle
    • test_multi_session_user — Test user with multiple concurrent sessions

tests/integration/test_chutes_integration.py

1 tests

  • test_chutes_models — Test loading Chutes models from the catalog

tests/integration/test_credit_deduction_models.py

14 tests

  • TestCreditDeductionForOpenAIModels (4 tests)
    • test_gpt4o_pricing_lookup — Test that GPT-4o model gets correct pricing
    • test_gpt4o_without_prefix_uses_alias — Test that 'gpt-4o' (without openai/ prefix) resolves via alias
    • test_cost_calculation_openai — Test cost calculation for OpenAI models
    • test_credit_handler_deducts_for_openai — Test that credit handler deducts credits for OpenAI model
  • TestCreditDeductionForAnthropicModels (4 tests)
    • test_claude_opus_pricing_lookup — Test that Claude Opus model gets correct pricing
    • test_claude_sonnet_pricing_lookup — Test that Claude Sonnet model gets correct pricing
    • test_cost_calculation_anthropic — Test cost calculation for Anthropic Claude Opus
    • test_credit_handler_deducts_for_anthropic — Test that credit handler deducts credits for Anthropic model
  • TestTrialUserCreditHandling (1 tests)
    • test_trial_user_no_credit_deduction — Test that trial users don't have credits deducted
  • TestTrialOverrideForPaidUsers (1 tests)
    • test_paid_user_with_stale_trial_gets_charged — Test that paid users with stale is_trial=True flag get charged
  • TestDefaultPricingAlerts (2 tests)
    • test_unknown_model_uses_default_pricing — Test that unknown models fall back to default pricing and are tracked
    • test_high_value_model_default_pricing_alert — Test that high-value models (OpenAI, Anthropic) trigger alerts on default pricing
  • TestAsyncCostCalculation (2 tests)
    • test_calculate_cost_async_works — Test that async cost calculation works correctly
    • test_free_model_returns_zero_cost_async — Test that free models return $0 cost in async version

tests/integration/test_e2e_coupon.py

26 tests

  • test_api_keys — Create test users with API keys
  • TestCouponAdminEndpoints (10 tests)
    • test_create_global_coupon — Test creating a global promotional coupon
    • test_create_user_specific_coupon — Test creating a user-specific coupon
    • test_create_coupon_invalid_negative_value — Test that negative coupon values are rejected
    • test_create_user_specific_without_assigned_user — Test that user-specific coupons require assigned_to_user_id
    • test_list_coupons — Test listing all coupons
    • test_list_coupons_with_filters — Test listing coupons with filters
    • test_get_coupon_by_id — Test getting a specific coupon by ID
    • test_get_nonexistent_coupon — Test getting a coupon that doesn't exist
    • test_update_coupon — Test updating coupon fields
    • test_deactivate_coupon — Test deactivating a coupon
  • TestCouponUserEndpoints (10 tests)
    • test_get_available_coupons — Test getting available coupons for a user
    • test_get_available_coupons_unauthorized — Test that getting available coupons requires authentication
    • test_redeem_global_coupon — Test redeeming a global coupon
    • test_redeem_user_specific_coupon — Test redeeming a user-specific coupon
    • test_redeem_coupon_twice_rejected — Test that redeeming the same coupon twice is rejected
    • test_redeem_invalid_coupon — Test redeeming an invalid coupon code
    • test_redeem_expired_coupon — Test that expired coupons cannot be redeemed
    • test_redeem_deactivated_coupon — Test that deactivated coupons cannot be redeemed
    • test_get_redemption_history — Test getting user redemption history
    • test_get_redemption_history_with_limit — Test getting redemption history with limit parameter
  • TestCouponAnalytics (2 tests)
    • test_get_coupon_analytics — Test getting analytics for a specific coupon
    • test_get_system_stats — Test getting system-wide coupon statistics
  • TestCouponEdgeCases (3 tests)
    • test_create_coupon_without_admin_key — Test that creating coupons requires admin API key
    • test_redeem_without_authentication — Test that redeeming requires authentication
    • test_wrong_user_cannot_redeem_user_specific_coupon — Test that user-specific coupons can only be redeemed by the assigned user

tests/integration/test_e2e_referral.py

16 tests

  • test_users — Create test users with API keys for referral testing
  • TestEndToEndReferralFlow (8 tests)
    • test_complete_referral_flow_success — Test complete successful referral flow:
    • test_referral_stats_accuracy — Test that referral stats are accurate after multiple referrals
    • test_referral_bonus_minimum_purchase_enforcement — Test that referral bonus requires minimum $10 purchase
    • test_self_referral_prevention — Test that users cannot use their own referral code
    • test_max_referral_uses_enforcement — Test that referral codes cannot exceed maximum uses
    • test_one_referral_code_per_user — Test that users can only use one referral code
    • test_referral_code_uniqueness — Test that all generated referral codes are unique
    • test_bonus_only_on_first_purchase — Test that referral bonus only applies on first purchase
  • TestReferralEdgeCases (3 tests)
    • test_invalid_referral_code — Test handling of invalid referral code
    • test_deleted_referrer — Test handling when referrer is deleted
    • test_partial_credit_failure_handling — Test handling when credit addition fails for one user
  • TestReferralAPIEndpoints (4 tests)
    • test_get_referral_code_endpoint — Test GET /referral/code endpoint
    • test_get_referral_stats_endpoint — Test GET /referral/stats endpoint
    • test_validate_referral_endpoint — Test POST /referral/validate endpoint
    • test_generate_referral_endpoint — Test POST /referral/generate endpoint

tests/integration/test_end_to_end_normalization.py

2 tests

  • test_fireworks_with_transform — Test that Fireworks accepts model IDs after transformation
  • test_openrouter_with_transform — Test that OpenRouter still works with transformed model IDs

tests/integration/test_endpoint_regression.py

28 tests

  • TestHealthEndpoints (3 tests)
    • test_health_endpoint_exists — Regression: GET /health must exist and return 200
    • test_ping_endpoint_exists — Regression: GET /ping must exist
    • test_root_endpoint_exists — Regression: GET / must exist
  • TestAuthenticationEndpoints (3 tests)
    • test_user_balance_endpoint_exists — Regression: GET /user/balance must exist
    • test_user_profile_endpoint_exists — Regression: GET /user/profile must exist
    • test_user_monitor_endpoint_exists — Regression: GET /user/monitor must exist
  • TestChatCompletionsEndpoints (2 tests)
    • test_v1_chat_completions_endpoint_exists — CRITICAL: POST /v1/chat/completions must exist and process requests
    • test_v1_chat_completions_streaming_exists — CRITICAL: POST /v1/chat/completions with stream=true must work
  • TestUnifiedResponsesEndpoint (2 tests)
    • test_v1_responses_endpoint_exists — CRITICAL: POST /v1/responses must exist (unified API)
    • test_v1_responses_with_json_format — Regression: /v1/responses must support response_format parameter
  • TestAnthropicMessagesEndpoint (2 tests)
    • test_v1_messages_endpoint_exists — CRITICAL: POST /v1/messages must exist (Anthropic/Claude API)
    • test_v1_messages_with_system_parameter — Regression: /v1/messages must support separate 'system' parameter
  • TestAPIKeyEndpoints (2 tests)
    • test_list_api_keys_endpoint_exists — Regression: GET /user/api-keys must exist
    • test_create_api_key_endpoint_exists — Regression: POST /user/api-keys must exist
  • TestPaymentEndpoints (2 tests)
    • test_stripe_checkout_session_endpoint_exists — Regression: Stripe checkout session endpoint must exist
    • test_list_payments_endpoint_exists — Regression: Payments list endpoint must exist
  • TestChatHistoryEndpoints (3 tests)
    • test_list_chat_sessions_endpoint_exists — Regression: Chat sessions list endpoint must exist
    • test_create_chat_session_endpoint_exists — Regression: Chat session creation endpoint must exist
    • test_get_chat_session_endpoint_exists — Regression: GET /chat-history/sessions/{session_id} must exist
  • TestCatalogEndpoints (2 tests)
    • test_catalog_models_endpoint_exists — Regression: GET /v1/models must exist
    • test_catalog_providers_endpoint_exists — Regression: GET /v1/provider must exist
  • TestRankingEndpoints (2 tests)
    • test_ranking_models_endpoint_exists — Regression: GET /ranking/models must exist
    • test_ranking_apps_endpoint_exists — Regression: GET /ranking/apps must exist
  • TestAdminEndpoints (2 tests)
    • test_admin_list_users_endpoint_exists — Regression: Admin list users endpoint must exist
    • test_admin_add_credits_endpoint_exists — Regression: Admin add credits endpoint must exist
  • TestRateLimitEndpoints (1 tests)
    • test_get_rate_limits_endpoint_exists — Regression: Rate limits endpoint must exist
  • TestActivityEndpoints (1 tests)
    • test_get_activity_endpoint_exists — Regression: Activity endpoint must exist
  • TestNotificationEndpoints (1 tests)
    • test_get_notifications_endpoint_exists — Regression: Notifications endpoint must exist

tests/integration/test_endpoints.py

28 tests

  • TestHealthEndpoints (3 tests)
    • test_health_check — Test GET /health returns 200
    • test_root_endpoint — Test GET / returns welcome message
    • test_ping_endpoint — Test GET /ping returns pong
  • TestAuthEndpoints (2 tests)
    • test_auth_endpoint_exists — Test POST /auth endpoint exists
    • test_user_balance_endpoint — Test GET /user/balance requires authentication
  • TestChatEndpoints (2 tests)
    • test_chat_completions_endpoint_exists — Test POST /v1/chat/completions endpoint exists and processes requests
    • test_chat_completions_requires_auth — Test /v1/chat/completions requires authentication
  • TestUserEndpoints (3 tests)
    • test_credit_transactions_endpoint — Test GET /user/credit-transactions endpoint exists
    • test_user_profile_endpoint — Test GET /user/profile endpoint exists
    • test_user_monitor_endpoint — Test GET /user/monitor endpoint exists
  • TestPaymentEndpoints (3 tests)
    • test_stripe_checkout_session_endpoint — Test POST /api/stripe/checkout-session endpoint exists
    • test_stripe_webhook_endpoint_exists — Test POST /api/stripe/webhook endpoint exists
    • test_payments_list_endpoint — Test GET /api/stripe/payments endpoint exists
  • TestRankingEndpoints (2 tests)
    • test_ranking_models_endpoint — Test GET /ranking/models endpoint exists
    • test_ranking_apps_endpoint — Test GET /ranking/apps endpoint exists
  • TestAPIKeyEndpoints (2 tests)
    • test_list_api_keys_endpoint — Test GET /api-keys endpoint exists
    • test_create_api_key_endpoint — Test POST /api-keys endpoint exists
  • TestAdminEndpoints (2 tests)
    • test_admin_list_users_endpoint — Test GET /admin/users endpoint exists
    • test_admin_add_credits_endpoint — Test POST /admin/users/{user_id}/credits endpoint exists
  • TestCatalogEndpoints (6 tests)
    • test_catalog_models_endpoint — Test GET /catalog/models endpoint exists
    • test_catalog_providers_endpoint — Test GET /catalog/providers endpoint exists
    • test_public_models_endpoint — Test GET /models endpoint mirrors catalog route
    • test_public_model_detail_endpoint_with_hf_developer — Test GET /models/{developer}/{model} supports Hugging Face-style slugs
    • test_catalog_models_groq_gateway — Test GET /catalog/models with Groq gateway parameter
    • test_public_models_groq_gateway — Test GET /models with Groq gateway parameter
  • TestChatHistoryEndpoints (2 tests)
    • test_chat_sessions_list_endpoint — Test GET /chat/sessions endpoint exists
    • test_create_chat_session_endpoint — Test POST /chat/sessions endpoint exists
  • TestIntegration (1 tests)
    • test_full_chat_flow — Test complete chat completion flow: auth -> credit check -> API call -> deduct credits

tests/integration/test_fireworks.py

1 tests

  • test_fireworks_api — Test Fireworks API directly

tests/integration/test_function_calling.py

2 tests

  • TestFunctionCallingIntegration (2 tests)
    • test_chat_completions_with_tools — Test that tools parameter is passed through to provider
    • test_huggingface_with_tools — Test that HuggingFace receives tools parameter

tests/integration/test_google_vertex_e2e.py

6 tests

  • test_user — Create a test user with initial credits
  • TestGoogleVertexE2E (4 tests)
    • test_01_vertex_credentials_available — Test that Google Vertex credentials are properly configured
    • test_02_create_user_and_purchase_credits — Test user creation and credit purchase
    • test_03_call_all_gemini_models — Test all Google Gemini models via Vertex AI
    • test_04_verify_streaming_support — Test streaming support for Google Gemini models
  • TestVertexAIDirectCall (1 tests)
    • test_vertex_client_can_make_request — Test that the Vertex AI client can make a direct API call

tests/integration/test_huggingface_case.py

1 tests

  • test_hf_case

tests/integration/test_huggingface_integration.py

5 tests

  • test_fetch_models — Test fetching models from Hugging Face API
  • test_cache — Test caching mechanism
  • test_fetch_hug_wrapper — Test the fetch_models_from_hug wrapper function
  • test_search — Test search functionality
  • test_model_info — Test fetching specific model info

tests/integration/test_model_transform.py

1 tests

  • test_api — Test the API with different model formats

tests/integration/test_near_qwen.py

1 tests

  • test_near_qwen_api — Test Near AI API with Qwen3-30B model

tests/integration/test_openrouter_auto_live.py

2 tests

  • test_openrouter_auto — Send a test message to openrouter/auto
  • test_code_only — Test just the code paths without making API calls

tests/integration/test_openrouter_direct.py

1 tests

  • test_model

tests/integration/test_payment_webhook_referral.py

8 tests

  • test_users_for_webhook — Create test users for webhook testing
  • TestPaymentWebhookReferralIntegration (5 tests)
    • test_checkout_completed_triggers_referral_bonus — Test that checkout.session.completed webhook triggers referral bonus
    • test_checkout_below_minimum_no_bonus — Test that purchases < $10 do NOT trigger referral bonus
    • test_second_purchase_no_bonus — Test that second purchase does NOT trigger referral bonus
    • test_payment_without_referral_code — Test that users without referral code just get payment credits
    • test_payment_succeeds_even_if_referral_fails — CRITICAL: Test that payment processing succeeds even if referral bonus fails
  • TestWebhookEdgeCases (2 tests)
    • test_webhook_with_missing_metadata — Test webhook handling when metadata is missing
    • test_webhook_with_invalid_user_id — Test webhook handling with non-existent user ID

tests/integration/test_provider_case_sensitivity.py

1 tests

  • test_provider_case_sensitivity

tests/integration/test_referral_comprehensive.py

10 tests

  • test_constants — Verify referral system constants
  • TestReferralSignupTracking (4 tests)
    • test_track_referral_signup_success — Test successful tracking of referral signup
    • test_track_referral_signup_invalid_code — Test tracking with invalid referral code
    • test_track_referral_signup_self_referral — Test that self-referral is blocked
    • test_track_referral_signup_max_uses_reached — Test that codes with max uses cannot be used
  • TestReferralNotifications (2 tests)
    • test_send_referral_signup_notification — Test sending signup notification to referrer
    • test_send_referral_bonus_notification — Test sending bonus notification to both users
  • TestReferralBonusWithPendingRecord (1 tests)
    • test_apply_bonus_updates_pending_referral — Test that bonus application updates existing pending referral record
  • TestTrialCredits (1 tests)
    • test_new_user_gets_trial_credits — Test new users receive $5 trial credits
  • TestEndToEndReferralFlow (1 tests)
    • test_complete_referral_flow — Test complete referral flow:

tests/integration/test_referral_database_integrity.py

12 tests

  • test_db_users — Create test users for database testing
  • TestReferralCodeUniqueness (2 tests)
    • test_referral_code_uniqueness_constraint — Test that database enforces referral code uniqueness
    • test_generate_unique_codes_at_scale — Test that code generation produces unique codes at scale
  • TestConcurrentReferralUsage (2 tests)
    • test_concurrent_signup_same_code — Test multiple users signing up with same code concurrently
    • test_concurrent_bonus_application — Test concurrent bonus applications (edge case)
  • TestReferralUsageLimits (1 tests)
    • test_usage_limit_enforced_in_database — Test that database correctly tracks and enforces usage limits
  • TestReferralStatusTransitions (2 tests)
    • test_pending_to_completed_transition — Test that referral status correctly transitions from pending to completed
    • test_cannot_complete_twice — Test that referral cannot be completed twice
  • TestDataIntegrity (3 tests)
    • test_orphaned_referral_prevention — Test that referral records reference valid users
    • test_referral_code_persistence — Test that referral codes persist correctly in database
    • test_referred_by_code_persistence — Test that referred_by_code persists correctly
  • TestCreditTransactionIntegrity (1 tests)
    • test_credit_transactions_created_for_bonuses — Test that credit transactions are created when bonuses are applied

tests/integration/test_routing_refactor.py

18 tests

  • TestProviderSelectorBasics (2 tests)
    • test_get_model_providers — Test getting available providers for a model
    • test_check_provider_health — Test checking provider health status
  • TestMultiProviderRegistry (3 tests)
    • test_get_model — Test retrieving a model from registry
    • test_select_provider — Test provider selection logic
    • test_get_fallback_providers — Test getting fallback providers
  • TestProviderFailover (3 tests)
    • test_execute_with_failover_success — Test successful execution with failover
    • test_execute_with_failover_retry — Test failover retries with different providers
    • test_execute_with_failover_all_fail — Test when all providers fail
  • TestCircuitBreaker (3 tests)
    • test_health_tracker_records_success — Test recording successful requests
    • test_health_tracker_records_failure — Test recording failed requests
    • test_circuit_breaker_opens_after_threshold — Test circuit breaker opens after failure threshold
  • TestModelPrompting (2 tests)
    • test_prompt_gpt4_through_api — Test prompting GPT-4 through the chat API
    • test_model_routing_with_mock_providers — Test model routing with mocked provider responses
  • TestProviderSpecificRouting (2 tests)
    • test_route_to_openrouter — Test explicit routing to OpenRouter
    • test_route_gemini_model — Test routing Google Gemini model
  • TestModelTransformation (1 tests)
    • test_detect_provider_from_model_id — Test automatic provider detection from model ID
  • TestCatalogIntegration (2 tests)
    • test_get_models_catalog — Test retrieving models catalog
    • test_catalog_includes_multi_provider_models — Test that catalog includes multi-provider model info

tests/integration/test_streaming_comprehensive.py

3 tests

  • test_streaming_response — Test 1: Streaming Response
  • test_non_streaming_response — Test 2: Non-Streaming Response (for comparison)
  • test_all_smoke_tests_summary — Final summary after all smoke tests

tests/integration/test_stripe_webhook_metadata.py

11 tests

  • TestCheckoutSessionMetadata (2 tests)
    • test_checkout_session_includes_credits_cents_field — Verify checkout session metadata includes 'credits_cents' field
    • test_checkout_session_includes_backward_compatible_credits_field — Verify checkout session metadata includes 'credits' field for backward compatibility
  • TestWebhookMetadataHandling (3 tests)
    • test_webhook_parses_credits_cents_field — Verify webhook correctly parses credits_cents from metadata
    • test_webhook_handles_missing_credits_cents_fallback_to_credits — Verify webhook falls back to 'credits' field if 'credits_cents' missing
    • test_webhook_always_marked_processed_even_on_error — Verify webhook is marked as processed even if handler fails
  • TestWebhookHttpStatus (2 tests)
    • test_webhook_endpoint_returns_200_on_success — Verify webhook endpoint returns HTTP 200 on success
    • test_webhook_endpoint_returns_200_on_error — Verify webhook endpoint returns HTTP 200 even on processing errors
  • TestMetadataExtraction (4 tests)
    • test_coerce_to_int_handles_string_numbers — Verify _coerce_to_int correctly converts string numbers
    • test_coerce_to_int_handles_float_numbers — Verify _coerce_to_int correctly converts floats
    • test_coerce_to_int_handles_none — Verify _coerce_to_int handles None gracefully
    • test_metadata_to_dict_handles_various_formats — Verify _metadata_to_dict handles various metadata formats

tests/integration/test_supabase_auth_integration.py

7 tests

  • TestSupabaseHttpxClientAuth (5 tests)
    • test_httpx_client_with_auth_headers_can_query_supabase — Test that an httpx client configured like our production client can query Supabase.
    • test_httpx_client_without_auth_headers_fails — Test that an httpx client WITHOUT auth headers fails to authenticate.
    • test_httpx_client_with_only_apikey_header_succeeds — Test that apikey header alone is sufficient for read operations.
    • test_supabase_client_initialization_succeeds — Test that the actual Supabase client can be initialized and query the database.
    • test_postgrest_session_has_correct_headers — Test that after initialization, the postgrest session has the correct auth headers.
  • TestSupabaseConnectionHealth (2 tests)
    • test_can_reach_supabase_health_endpoint — Test basic connectivity to Supabase.
    • test_supabase_url_has_correct_format — Test that SUPABASE_URL is correctly formatted.

tests/integration/test_thinking_tags.py

1 tests

  • test_thinking_tags_in_stream — Test that thinking tags are properly passed through in streaming responses

tests/integration/test_trial_credits_with_daily_limits.py

11 tests

  • TestTrialCreditAllocation (2 tests)
    • test_new_user_receives_5_dollar_credits — Test that new users receive $5 in trial credits
    • test_trial_credits_configuration — Test that configuration is set correctly
  • TestDailyLimitDuringTrial (2 tests)
    • test_trial_user_limited_to_1_dollar_per_day — Test that trial users can only use $1/day even with $5 credits
    • test_multiple_small_requests_within_daily_limit — Test multiple small API calls that stay within $1/day limit
  • TestTrialPeriodScenario (1 tests)
    • test_three_day_trial_usage_pattern — Simulate a user using $1/day for 3 days during trial
  • TestPostTrialCredits (1 tests)
    • test_post_trial_credits_available — Test that $2 remaining credits can be used after trial expires
  • TestFraudMitigation (1 tests)
    • test_bot_cannot_drain_credits_instantly — Test that a bot cannot drain all $5 credits in one day
  • TestConfigurationValues (4 tests)
    • test_trial_credits_is_5_dollars — Verify TRIAL_CREDITS_AMOUNT is $5
    • test_daily_limit_is_1_dollar — Verify DAILY_USAGE_LIMIT is $1
    • test_trial_duration_is_3_days — Verify TRIAL_DURATION_DAYS is 3
    • test_trial_daily_limit_is_1_dollar — Verify TRIAL_DAILY_LIMIT is $1

tests/integration/test_unique_models_basic.py

10 tests

  • TestUniqueModelsBasic (10 tests)
    • test_database_layer_get_unique_models — Test database layer can fetch unique models.
    • test_database_layer_transformation — Test database-to-API transformation.
    • test_service_layer_unique_models_cache — Test service layer unique models caching.
    • test_service_layer_get_cached_models_flat_mode — Test backward compatibility - flat mode.
    • test_service_layer_get_cached_models_unique_mode — Test unique mode via get_cached_models.
    • test_route_layer_default_behavior — Test /models endpoint default behavior (backward compatibility).
    • test_route_layer_unique_models_false — Test /models endpoint with unique_models=false.
    • test_route_layer_unique_models_true — Test /models endpoint with unique_models=true.
    • test_route_layer_unique_models_with_specific_gateway — Test that unique_models is ignored for provider-specific queries.
    • test_deduplication_check — Test that unique_models=true actually deduplicates.

tests/integration/test_vercel_ai_gateway.py

6 tests

  • test_vercel_config — Test that Vercel AI Gateway configuration is present
  • test_vercel_client_import — Test that the Vercel AI Gateway client can be imported
  • test_vercel_model_mappings — Test Vercel AI Gateway model ID transformations
  • test_vercel_provider_detection — Test that Vercel AI Gateway models are properly detected
  • test_failover_chain — Test that Vercel AI Gateway is in the failover chain
  • test_chat_route_integration — Test that Vercel AI Gateway is integrated in the chat route

tests/middleware/

3 files, 65 tests

tests/middleware/test_auto_sentry_middleware.py

19 tests

  • TestAutoSentryMiddleware (16 tests)
    • test_successful_request_no_sentry_capture — Test that successful requests don't trigger Sentry
    • test_error_captured_to_sentry — Test that errors are captured to Sentry with context
    • test_request_context_extraction — Test that request context is properly extracted
    • test_endpoint_type_detection_inference — Test endpoint type detection for inference endpoints
    • test_endpoint_type_detection_payment — Test endpoint type detection for payment endpoints
    • test_endpoint_type_detection_auth — Test endpoint type detection for auth endpoints
    • test_sensitive_headers_sanitized — Test that sensitive headers are sanitized
    • test_revenue_critical_tag — Test that revenue-critical endpoints are tagged
    • test_http_exception_handling — Test that HTTPExceptions are not captured by middleware to avoid duplicates
    • test_http_exception_filtering_in_middleware — Test that 4xx HTTPException is filtered in middleware exception handler
    • test_http_exception_various_status_codes — Test that all HTTPExceptions (4xx and 5xx) are filtered to avoid duplicates
    • test_regular_exception_still_captured — Test that non-HTTPException errors are still captured
    • test_sentry_unavailable_no_error — Test that middleware works even when Sentry is unavailable
    • test_slow_request_breadcrumb — Test that slow requests generate breadcrumbs
    • test_middleware_determines_endpoint_types_correctly — Test endpoint type determination logic
    • test_middleware_categorizes_http_errors_correctly — Test HTTP error categorization
  • TestAutoSentryMiddlewareEdgeCases (3 tests)
    • test_missing_client_info — Test handling when request.client is None
    • test_empty_headers — Test handling of empty headers
    • test_sanitize_headers_edge_cases — Test header sanitization with various input

tests/middleware/test_security_middleware.py

34 tests

  • TestVelocityModeActivation (4 tests)
    • test_velocity_mode_not_activated_below_threshold — Test that velocity mode does not activate when error rate is below threshold
    • test_velocity_mode_activated_above_threshold — Test that velocity mode activates when error rate exceeds threshold
    • test_velocity_mode_not_activated_insufficient_samples — Test that velocity mode requires minimum sample size
    • test_velocity_mode_activation_increments_counter — Test that activation counter increments
  • TestVelocityModeDeactivation (2 tests)
    • test_velocity_mode_deactivates_after_cooldown — Test that velocity mode deactivates after cooldown period
    • test_velocity_mode_stays_active_during_cooldown — Test that velocity mode stays active during cooldown
  • TestErrorClassification (4 tests)
    • test_4xx_errors_not_counted — Test that 4xx errors (client errors) are not counted as system errors
    • test_5xx_errors_counted — Test that 5xx errors (server errors) are counted
    • test_499_slow_requests_counted — Test that 499 errors with slow duration (>5s) are counted
    • test_499_fast_requests_not_counted — Test that 499 errors with fast duration (<5s) are not counted
  • TestRateLimitCalculation (3 tests)
    • test_effective_limit_normal_mode — Test that limits are normal when velocity mode is inactive
    • test_effective_limit_velocity_mode — Test that limits are reduced during velocity mode
    • test_effective_limit_minimum_of_one — Test that effective limit never goes below 1
  • TestAuthenticatedUserExemption (5 tests)
    • test_bearer_token_detected — Test that Bearer token format is detected
    • test_gw_api_key_detected — Test that Gatewayz API key format is detected
    • test_generic_api_key_detected — Test that generic long API keys are detected
    • test_no_authorization_header — Test that request without auth header is not authenticated
    • test_short_authorization_header — Test that short auth headers are not considered authenticated
  • TestIPTierDetection (3 tests)
    • test_datacenter_user_agent_detected — Test that datacenter IPs are detected by user agent
    • test_proxy_headers_detected — Test that proxy headers indicate datacenter IP
    • test_residential_ip_not_detected_as_datacenter — Test that residential IPs are not flagged as datacenter
  • TestFingerprintGeneration (3 tests)
    • test_fingerprint_generation — Test that fingerprint is generated from headers
    • test_same_headers_produce_same_fingerprint — Test that identical headers produce identical fingerprints
    • test_different_headers_produce_different_fingerprints — Test that different headers produce different fingerprints
  • TestRequestOutcomeRecording (2 tests)
    • test_request_outcome_recorded — Test that request outcomes are recorded
    • test_old_entries_cleaned — Test that old entries are removed from request log
  • TestRateLimitHeaders (1 tests)
    • test_rate_limit_headers_present — Test that 429 responses include rate limit headers
  • TestIntegrationScenarios (2 tests)
    • test_health_endpoint_bypasses_security — Test that health endpoints bypass security checks
    • test_normal_request_allowed — Test that normal requests are allowed
  • TestVelocityModeConfiguration (2 tests)
    • test_configuration_constants — Test that configuration constants are set correctly
    • test_ip_limit_constants — Test that IP limit constants are set correctly
  • TestEdgeCases (3 tests)
    • test_empty_request_log — Test that velocity mode doesn't activate with empty log
    • test_deque_max_length_respected — Test that request log respects maxlen (10000)
    • test_client_ip_extraction — Test client IP extraction from X-Forwarded-For

tests/middleware/test_selective_gzip_middleware.py

12 tests

  • TestSelectiveGZipMiddleware (8 tests)
    • test_small_json_not_compressed — Test that small JSON responses are NOT compressed (below minimum_size)
    • test_large_json_compressed — Test that large JSON responses ARE compressed
    • test_sse_stream_not_compressed — Test that SSE streaming responses are NOT compressed
    • test_sse_stream_without_explicit_headers_not_compressed — Test that SSE streaming is NOT compressed even without explicit headers
    • test_ndjson_stream_not_compressed — Test that NDJSON streaming responses are NOT compressed
    • test_binary_stream_with_no_buffering_header_not_compressed — Test that responses with X-Accel-Buffering: no are NOT compressed
    • test_no_gzip_without_accept_encoding — Test that responses are NOT compressed when client doesn't accept gzip
    • test_vary_header_added_for_compressed_response — Test that Vary: Accept-Encoding header is added for compressed responses
  • TestSelectiveGZipMiddlewareEdgeCases (2 tests)
    • test_streaming_media_types_constant — Test that STREAMING_MEDIA_TYPES contains expected types
    • test_compress_method — Test the internal _compress method
  • TestStreamingHeadersIntegration (2 tests)
    • test_sse_headers_complete — Test that all expected SSE headers are present
    • test_chunks_received_individually — Test that SSE chunks can be received as they stream

tests/routes/

61 files, 958 tests

tests/routes/test_activity.py

23 tests

  • TestActivityStatsEndpoint (6 tests)
    • test_get_activity_stats_default — Test getting activity stats with defaults
    • test_get_activity_stats_with_days — Test getting activity stats with days parameter
    • test_get_activity_stats_with_date_range — Test getting activity stats with date range
    • test_get_activity_stats_empty — Test getting stats when no activity exists
    • test_get_activity_stats_error — Test error handling in stats endpoint
    • test_get_activity_stats_days_validation — Test validation for days parameter
  • TestActivityLogEndpoint (12 tests)
    • test_get_activity_log_default — Test getting activity log with defaults
    • test_get_activity_log_with_pagination — Test activity log with pagination
    • test_get_activity_log_with_page_number — Test activity log with page number
    • test_get_activity_log_with_date_filters — Test activity log with date filtering
    • test_get_activity_log_with_model_filter — Test activity log with model filter
    • test_get_activity_log_with_provider_filter — Test activity log with provider filter
    • test_get_activity_log_combined_filters — Test activity log with multiple filters
    • test_get_activity_log_empty — Test getting empty activity log
    • test_get_activity_log_error — Test error handling in log endpoint
    • test_get_activity_log_limit_validation — Test validation for limit parameter
    • test_get_activity_log_offset_validation — Test validation for offset parameter
    • test_get_activity_log_page_validation — Test validation for page parameter
  • TestActivityAuthentication (2 tests)
    • test_stats_requires_authentication — Test that stats endpoint requires authentication
    • test_log_requires_authentication — Test that log endpoint requires authentication
  • TestActivityIntegration (3 tests)
    • test_stats_and_log_consistency — Test that stats and log endpoints return consistent data
    • test_pagination_workflow — Test pagination workflow across multiple pages
    • test_date_range_filtering — Test date range filtering across stats endpoint

tests/routes/test_admin.py

26 tests

  • TestUserCreation (3 tests)
    • test_create_user_success — Successfully create a new user
    • test_create_user_invalid_environment — Create user fails with invalid environment tag
    • test_create_user_missing_fields — Create user fails with missing required fields
  • TestAdminAuthentication (3 tests)
    • test_admin_endpoint_requires_authentication — Admin endpoint rejects requests without authentication
    • test_admin_endpoint_rejects_non_admin_user — Regular user cannot access admin endpoints
    • test_admin_endpoint_accepts_valid_admin — Admin user can access admin endpoints
  • TestCreditManagement (3 tests)
    • test_add_credits_success — Admin can add credits to user
    • test_add_credits_user_not_found — Add credits fails when user not found
    • test_add_negative_credits — Admin can add negative credits (deduct)
  • TestRateLimitManagement (1 tests)
    • test_set_rate_limits_success — Admin can set user rate limits
  • TestSystemOperations (1 tests)
    • test_get_all_users — Admin can view all users
  • TestAdminValidation (2 tests)
    • test_add_credits_requires_api_key — Add credits requires api_key field
    • test_add_credits_requires_credits_amount — Add credits requires credits amount
  • TestAdminEdgeCases (1 tests)
    • test_add_zero_credits — Adding zero credits should work
  • TestPricingSchedulerStatus (4 tests)
    • test_get_scheduler_status_success — Admin can get scheduler status
    • test_get_scheduler_status_requires_admin — Scheduler status requires admin role
    • test_get_scheduler_status_requires_authentication — Scheduler status requires authentication
    • test_get_scheduler_status_handles_error — Scheduler status handles errors gracefully
  • TestPricingSchedulerTrigger (6 tests)
    • test_trigger_manual_sync_success — Admin can trigger manual pricing sync
    • test_trigger_manual_sync_failure — Manual sync handles failure gracefully
    • test_trigger_manual_sync_requires_admin — Manual sync trigger requires admin role
    • test_trigger_manual_sync_requires_authentication — Manual sync trigger requires authentication
    • test_trigger_manual_sync_logs_admin_user — Manual sync logs the admin user who triggered it
    • test_trigger_manual_sync_handles_exception — Manual sync handles unexpected exceptions
  • TestPricingSchedulerIntegration (2 tests)
    • test_status_after_manual_trigger — Status endpoint shows updated state after manual trigger
    • test_multiple_admin_users_can_trigger — Multiple admin users can trigger manual sync

tests/routes/test_ai_sdk.py

47 tests

  • TestAISDKEndpoint (8 tests)
    • test_ai_sdk_endpoint_exists — Test that the AI SDK endpoint is registered
    • test_ai_sdk_completions_endpoint_exists — Test that the AI SDK completions endpoint is registered
    • test_ai_sdk_chat_completion_success — Test successful AI SDK chat completion request
    • test_ai_sdk_missing_api_key — Test that missing API key returns proper error and captures to Sentry
    • test_ai_sdk_invalid_request_format — Test that invalid request format returns proper error
    • test_ai_sdk_streaming_request — Test streaming response from AI SDK endpoint
    • test_ai_sdk_endpoint_schema — Test that endpoint properly validates request schema
    • test_ai_sdk_completions_endpoint_works — Test that the -completions variant endpoint works identically
  • TestAISDKConfiguration (2 tests)
    • test_ai_sdk_config_variable_exists — Test that AI_SDK_API_KEY is available in Config
    • test_ai_sdk_config_loading — Test that AI_SDK_API_KEY is properly loaded from environment
  • TestAISDKModelRouting (8 tests)
    • test_is_openrouter_model_auto — Test that openrouter/auto is detected correctly
    • test_is_openrouter_model_other_openrouter_models — Test that other openrouter/* models are detected
    • test_is_openrouter_model_case_insensitive — Test that openrouter/* detection is case insensitive
    • test_is_openrouter_model_false_for_regular_models — Test that regular models are not detected as openrouter/*
    • test_is_openrouter_model_false_for_empty — Test that empty model returns False
    • test_is_openrouter_model_false_for_none — Test that None model returns False
    • test_openrouter_auto_routes_to_openrouter — Test that openrouter/auto is routed directly to OpenRouter
    • test_openrouter_auto_streaming_routes_to_openrouter — Test that openrouter/auto streaming is routed directly to OpenRouter
  • TestAISDKErrorMessages (5 tests)
    • test_openrouter_missing_api_key_shows_openrouter_error — Test that missing OpenRouter API key shows OpenRouter-specific error message
    • test_ai_sdk_missing_api_key_shows_ai_sdk_error — Test that missing AI SDK API key shows AI SDK-specific error message
    • test_openrouter_error_captured_to_sentry — Test that OpenRouter configuration errors are captured to Sentry
    • test_openrouter_streaming_missing_api_key_returns_503 — Test that missing OpenRouter API key returns HTTP 503 for streaming requests
    • test_openrouter_streaming_error_captured_to_sentry — Test that OpenRouter streaming config errors are captured to Sentry
  • TestAISDKStreamingReasoningContent (5 tests)
    • test_streaming_with_reasoning_content — Test streaming response includes reasoning_content for thinking models
    • test_streaming_only_reasoning_content — Test streaming response with only reasoning content (no text)
    • test_streaming_with_reasoning_via_reasoning_attr — Test streaming response with 'reasoning' attribute (alternative name)
    • test_openrouter_streaming_with_reasoning_content — Test OpenRouter streaming response includes reasoning_content
    • test_streaming_empty_reasoning_content_no_fallback — Test that empty reasoning_content doesn't incorrectly fallback to reasoning attribute
  • TestAISDKAuthentication (4 tests)
    • test_ai_sdk_requires_authentication — Test that AI SDK endpoints require API key authentication
    • test_ai_sdk_completions_requires_authentication — Test that AI SDK completions endpoint requires API key authentication
    • test_ai_sdk_invalid_api_key — Test that invalid API key returns 401
    • test_ai_sdk_trial_expired — Test that expired trial returns 403
  • TestAISDKCreditDeduction (3 tests)
    • test_ai_sdk_deducts_credits_for_non_trial — Test that AI SDK deducts credits for non-trial users
    • test_ai_sdk_tracks_trial_usage — Test that AI SDK tracks trial usage for trial users
    • test_ai_sdk_streaming_deducts_credits — Test that AI SDK streaming deducts credits after stream completes
  • TestAISDKSentryIntegration (3 tests)
    • test_general_error_captured_to_sentry — Test that general errors are captured to Sentry
    • test_error_when_sentry_unavailable — Test that errors still work when Sentry is unavailable
    • test_streaming_error_captured_to_sentry — Test that streaming errors are captured to Sentry
  • TestTrialStatusOverride (9 tests)
    • test_check_trial_override_non_trial_user_returns_false — Test that non-trial users return False (should be billed)
    • test_check_trial_override_legitimate_trial_user_returns_true — Test that legitimate trial users return True (should NOT be billed)
    • test_check_trial_override_paid_user_with_stale_trial_flag — Test that paid users with stale is_trial=TRUE are overridden to billing path
    • test_check_trial_override_pro_tier_with_trial_flag — Test that pro tier users with is_trial=TRUE are overridden
    • test_check_trial_override_max_tier_with_trial_flag — Test that max tier users with is_trial=TRUE are overridden
    • test_check_trial_override_admin_tier_with_trial_flag — Test that admin tier users with is_trial=TRUE are overridden
    • test_check_trial_override_inactive_subscription_not_overridden — Test that cancelled subscriptions are NOT overridden
    • test_check_trial_override_null_user_returns_trial_status — Test that None user returns original trial status
    • test_check_trial_override_empty_trial_dict_returns_false — Test that empty trial dict is treated as non-trial

tests/routes/test_alibaba_debug.py

2 tests

  • TestAlibabaDebugRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_analytics.py

17 tests

  • TestLogEvent (6 tests)
    • test_log_event_authenticated_user — Test logging event with authenticated user
    • test_log_event_anonymous_user — Test logging event without authentication
    • test_log_event_with_provided_user_id — Test logging event with user_id in payload
    • test_log_event_without_metadata — Test logging event without optional metadata
    • test_log_event_statsig_error — Test error handling when Statsig fails
    • test_log_event_missing_event_name — Test validation error for missing event_name
  • TestLogBatchEvents (5 tests)
    • test_log_batch_events_authenticated_user — Test logging multiple events for authenticated user
    • test_log_batch_events_mixed_user_ids — Test batch events with different user IDs
    • test_log_batch_events_empty_list — Test batch logging with empty events list
    • test_log_batch_events_large_batch — Test logging large batch of events
    • test_log_batch_events_partial_failure — Test error handling when batch logging fails
  • TestEventMetadataHandling (2 tests)
    • test_event_with_complex_metadata — Test event with complex nested metadata
    • test_event_with_array_metadata — Test event with array in metadata
  • TestUserIDPriority (3 tests)
    • test_user_id_priority_authenticated_over_provided — Test authenticated user ID takes priority over provided user_id
    • test_user_id_priority_provided_over_anonymous — Test provided user_id takes priority over anonymous
    • test_user_id_default_to_anonymous — Test default to anonymous when no user info available
  • TestAnalyticsIntegration (1 tests)
    • test_single_and_batch_consistency — Test that single and batch endpoints produce consistent results

tests/routes/test_api_keys.py

22 tests

  • TestApiKeyCreation (5 tests)
    • test_create_api_key_success — Test successful API key creation
    • test_create_api_key_insufficient_permissions — Test API key creation with insufficient permissions
    • test_create_api_key_invalid_environment — Test API key creation with invalid environment tag
    • test_create_api_key_negative_expiration — Test API key creation with negative expiration days
    • test_create_api_key_duplicate_name — Test API key creation with duplicate name
  • TestApiKeyUpdate (5 tests)
    • test_update_api_key_success — Test successfully updating API key
    • test_update_api_key_not_found — Test updating non-existent API key
    • test_update_api_key_no_fields — Test updating API key with no fields
    • test_rotate_api_key_success — Test successfully rotating API key
    • test_bulk_rotate_api_keys_success — Test bulk rotation of API keys
  • TestApiKeyListing (3 tests)
    • test_list_api_keys_success — Test successfully listing API keys
    • test_list_api_keys_insufficient_permissions — Test listing keys with insufficient permissions
    • test_list_api_keys_empty — Test listing when no API keys exist
  • TestApiKeyDeletion (4 tests)
    • test_delete_api_key_success — Test successfully deleting API key
    • test_delete_api_key_wrong_confirmation — Test deleting API key with wrong confirmation
    • test_delete_api_key_not_found — Test deleting non-existent API key
    • test_delete_api_key_failed — Test API key deletion failure
  • TestApiKeyUsage (3 tests)
    • test_get_usage_stats_success — Test successfully getting usage statistics
    • test_get_usage_stats_failed — Test usage statistics retrieval failure
    • test_get_usage_stats_invalid_key — Test usage stats with invalid API key
  • TestApiKeyIntegration (2 tests)
    • test_complete_key_lifecycle — Test complete API key lifecycle: create -> list -> use
    • test_update_then_delete_workflow — Test updating then deleting a key

tests/routes/test_api_models.py

13 tests

  • TestApiModelsDetailEndpoint (12 tests)
    • test_get_model_detail_with_model_id — Test getting model detail using modelId parameter
    • test_get_model_detail_with_separate_params — Test getting model detail using separate developer and modelName params
    • test_get_model_detail_with_all_params — Test with both modelId and separate params (separate params take precedence)
    • test_get_model_detail_with_gateway — Test getting model detail with specific gateway
    • test_get_model_detail_not_found — Test 404 when model not found
    • test_get_model_detail_missing_params — Test 400 when required parameters are missing
    • test_get_model_detail_only_developer — Test 400 when only developer is provided
    • test_get_model_detail_only_model_name — Test when only modelName is provided without developer
    • test_get_model_detail_with_huggingface — Test including HuggingFace data
    • test_get_model_detail_returns_providers_list — Test that providers list is included in response
    • test_get_model_detail_url_encoded_model_id — Test with URL-encoded modelId (as sent by frontend)
    • test_get_model_detail_with_complex_model_name — Test with complex model names containing multiple slashes
  • TestApiModelsDetailErrorHandling (1 tests)
    • test_internal_error_handling — Test that internal errors return 500

tests/routes/test_audio.py

15 tests

  • TestAudioTranscriptions (7 tests)
    • test_transcription_endpoint_exists — Test that the transcription endpoint is registered.
    • test_transcription_requires_file — Test that file parameter is required.
    • test_transcription_success — Test successful transcription.
    • test_transcription_with_language_hint — Test transcription with language hint.
    • test_transcription_with_prompt — Test transcription with prompt context.
    • test_transcription_empty_file — Test that empty files are rejected.
    • test_transcription_unsupported_format — Test that unsupported formats are handled.
  • TestAudioTranscriptionsBase64 (5 tests)
    • test_base64_endpoint_exists — Test that the base64 transcription endpoint is registered.
    • test_base64_transcription_success — Test successful base64 transcription.
    • test_base64_data_url_format — Test handling of data URL format.
    • test_base64_invalid_data — Test that invalid base64 data is rejected.
    • test_base64_empty_data — Test that empty base64 data is rejected.
  • TestAudioFormats (1 tests)
    • test_supported_formats — Test that various audio formats are accepted.
  • TestAudioErrorHandling (2 tests)
    • test_whisper_api_error — Test handling of Whisper API errors.
    • test_client_unavailable — Test handling when OpenAI client is unavailable.

tests/routes/test_audit.py

18 tests

  • TestGetAuditLogs (7 tests)
    • test_get_audit_logs_success — Test successfully retrieving audit logs
    • test_get_audit_logs_with_key_filter — Test retrieving audit logs filtered by key_id
    • test_get_audit_logs_with_action_filter — Test retrieving audit logs filtered by action
    • test_get_audit_logs_with_date_range — Test retrieving audit logs with date range
    • test_get_audit_logs_with_limit — Test retrieving audit logs with custom limit
    • test_get_audit_logs_combined_filters — Test retrieving audit logs with multiple filters
    • test_get_audit_logs_empty — Test retrieving empty audit logs
  • TestAuditAuthentication (3 tests)
    • test_get_audit_logs_invalid_api_key — Test retrieving audit logs with invalid API key
    • test_get_audit_logs_insufficient_permissions — Test retrieving audit logs without sufficient permissions
    • test_get_audit_logs_missing_api_key — Test retrieving audit logs without API key
  • TestAuditDateValidation (3 tests)
    • test_get_audit_logs_invalid_start_date — Test invalid start_date format
    • test_get_audit_logs_invalid_end_date — Test invalid end_date format
    • test_get_audit_logs_valid_iso_dates — Test valid ISO date formats
  • TestAuditErrorHandling (2 tests)
    • test_get_audit_logs_database_error — Test error handling when database fails
    • test_get_audit_logs_user_lookup_error — Test error handling when user lookup fails
  • TestAuditIntegration (3 tests)
    • test_audit_log_filtering_workflow — Test complete filtering workflow
    • test_audit_log_pagination — Test pagination with limit parameter
    • test_audit_log_metadata_structure — Test audit log response includes metadata

tests/routes/test_auth_error_handling.py

5 tests

  • TestAuthConfigurationErrorHandling (5 tests)
    • test_returns_503_for_url_protocol_error — Test that URL protocol errors return 503 with user-friendly message
    • test_returns_503_for_supabase_url_must_start_with_error — Test that SUPABASE_URL validation errors return 503
    • test_returns_503_for_supabase_url_not_set_error — Test that missing SUPABASE_URL error returns 503
    • test_returns_500_for_other_errors — Test that non-configuration errors return 500
    • test_does_not_expose_internal_details_for_config_errors — Test that configuration errors don't expose internal URL values to users

tests/routes/test_auth_temp_email_bot_status.py

12 tests

  • test_privy_auth_temp_email_sets_bot_status — Test that new users with temporary email get subscription_status='bot'
  • test_privy_auth_normal_email_sets_trial_status — Test that new users with normal email get subscription_status='trial'
  • test_privy_auth_10minutemail_sets_bot_status — Test that 10minutemail.com domain gets bot status
  • test_privy_auth_guerrillamail_sets_bot_status — Test that guerrillamail.com domain gets bot status
  • test_privy_auth_mailinator_sets_bot_status — Test that mailinator.com domain gets bot status
  • test_privy_auth_yopmail_sets_bot_status — Test that yopmail.com domain gets bot status
  • test_privy_auth_blocked_domain_still_rejected — Test that blocked abuse domains are still rejected (not just marked as bot)
  • test_register_temp_email_sets_bot_status — Test that direct registration with temporary email sets bot status
  • test_register_normal_email_sets_trial_status — Test that direct registration with normal email sets trial status
  • test_register_blocked_domain_still_rejected — Test that blocked domains are still rejected in registration
  • test_temp_email_user_stored_with_bot_status_in_db — Verify that users with temp email are stored with bot status in database
  • test_normal_email_user_stored_with_trial_status_in_db — Verify that users with normal email are stored with trial status in database

tests/routes/test_auth_v2.py

28 tests

  • test_privy_auth_existing_user_success — Test successful authentication for existing user
  • test_privy_auth_existing_user_no_api_keys_uses_legacy — Test existing user with no API keys falls back to legacy key
  • test_privy_auth_fallback_to_username_lookup — Test fallback to username lookup if Privy ID not found
  • test_privy_auth_new_user_creation — Test creating new user via Privy auth
  • test_privy_auth_email_address_field_extraction — Email accounts that use address field should still be processed
  • test_privy_auth_google_oauth — Test authentication with Google OAuth
  • test_privy_auth_github — Test authentication with GitHub
  • test_privy_auth_email_fallback — Test email fallback when no email found in linked accounts
  • test_privy_auth_phone_new_user — Test authentication with phone number for new user
  • test_privy_auth_phone_existing_user — Test authentication with phone number for existing user
  • test_privy_auth_phone_with_email — Test user with both phone and email linked accounts
  • test_privy_auth_phone_camelcase_field — Test that phoneNumber (camelCase from Privy) is properly handled
  • test_privy_auth_phone_international_format — Test phone authentication with various international phone number formats
  • test_privy_auth_phone_short_number — Test phone authentication with a short phone number (edge case)
  • test_privy_auth_phone_first_then_email_added — Test user who signed up with phone, then added email later
  • test_privy_auth_phone_with_google_oauth — Test user with phone and Google OAuth linked
  • test_privy_auth_phone_returning_user_updates_phone — Test returning user who adds phone number to their account
  • test_privy_auth_phone_only_no_email_placeholder — Test that phone-only users get proper placeholder email in database
  • test_privy_auth_phone_username_generation_various_lengths — Test username generation from phone numbers of various lengths
  • test_register_user_success — Test successful user registration
  • test_register_user_duplicate_email — Test registration with existing email
  • test_register_user_duplicate_username — Test registration with existing username
  • test_request_password_reset_success — Test successful password reset request
  • test_request_password_reset_user_not_found — Test password reset for non-existent email (security - don't reveal)
  • test_reset_password_with_valid_token — Test resetting password with valid token
  • test_reset_password_with_invalid_token — Test resetting password with invalid token
  • test_reset_password_with_expired_token — Test resetting password with expired token
  • test_reset_password_with_used_token — Test resetting password with already used token

tests/routes/test_availability.py

2 tests

  • TestAvailabilityRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_catalog_endpoints.py

29 tests

  • TestGetProvidersEndpoint (17 tests)
    • test_get_providers_openrouter_default — Test getting OpenRouter providers (default)
    • test_get_providers_with_gateway_param — Test with specific gateway parameter
    • test_get_providers_all_gateways — Test getting providers from all gateways
    • test_get_providers_huggingface — Test getting Hugging Face providers
    • test_get_providers_huggingface_alias — Test huggingface gateway alias
    • test_get_providers_moderated_only — Test filtering for moderated providers
    • test_get_providers_with_limit — Test pagination with limit parameter
    • test_get_providers_with_offset — Test pagination with offset parameter
    • test_get_providers_empty_data — Test when no provider data available - should return 200 with empty response (graceful degradation)
    • test_get_providers_none_data — Test when provider data is None - should return 200 with empty response (graceful degradation)
    • test_get_providers_groq — Test Groq gateway
    • test_get_providers_fireworks — Test Fireworks gateway
    • test_get_providers_together — Test Together gateway - should return 200 with empty data (graceful degradation)
    • test_get_providers_cerebras — Test Cerebras gateway - should return 200 with empty data (graceful degradation)
    • test_get_providers_xai — Test xAI gateway
    • test_get_providers_deepinfra — Test DeepInfra gateway
    • test_get_providers_featherless — Test Featherless gateway - should return 200 with empty data (graceful degradation)
  • TestModelsEndpoint (3 tests)
    • test_nebius_gateway_returns_empty_catalog — Requests for Nebius should return 200 even if no catalog is available.
    • test_xai_gateway_returns_empty_catalog — Requests for xAI should return 200 even if no catalog is available.
    • test_models_page_all_gateway_loads — Ensure the models catalog page can load aggregated data.
  • TestMergeProviderLists (1 tests)
    • test_merge_providers_from_multiple_sources — Test that providers from multiple gateways are merged
  • TestMergeModelsBySlug (1 tests)
    • test_models_merged_correctly — Test that duplicate models are handled
  • TestGetGatewaysEndpoint (7 tests)
    • test_get_gateways_returns_list — Test that gateways endpoint returns a list of gateways
    • test_get_gateways_has_required_fields — Test that each gateway has required fields
    • test_get_gateways_includes_simplismart — Test that SimpliSmart is in the gateway list
    • test_get_gateways_includes_major_providers — Test that major providers are included
    • test_get_gateways_fast_gateways_first — Test that fast gateways are sorted before slow gateways
    • test_get_gateways_logo_urls — Test that gateways have logo URLs generated from site URLs
    • test_get_gateways_aliases — Test that gateways with aliases include them

tests/routes/test_catalog_simplismart_models_fix.py

9 tests

  • TestSimplismartModelsInitialization (6 tests)
    • test_get_models_alpaca_gateway_no_unbound_error — Test that requesting models from 'alpaca' gateway doesn't cause UnboundLocalError.
    • test_get_models_clarifai_gateway_no_unbound_error — Test that requesting models from 'clarifai' gateway doesn't cause UnboundLocalError.
    • test_get_models_huggingface_gateway_no_unbound_error — Test that requesting models from 'huggingface' gateway doesn't cause UnboundLocalError.
    • test_get_models_multiple_non_simplismart_gateways — Test multiple different gateways to ensure robustness of the fix.
    • test_get_models_simplismart_gateway_still_works — Verify that the fix doesn't break the intended simplismart gateway functionality.
    • test_get_models_all_gateway_includes_simplismart — Verify that 'all' gateway properly includes simplismart models.
  • TestRootModelsEndpoint (2 tests)
    • test_root_models_endpoint_alpaca_no_error — Test /models endpoint with alpaca gateway
    • test_root_models_endpoint_clarifai_no_error — Test /models endpoint with clarifai gateway
  • TestProviderDerivationBlock (1 tests)
    • test_provider_derivation_with_all_gateway — Test that 'all' gateway properly handles provider derivation for simplismart.

tests/routes/test_catalog_utils.py

17 tests

  • TestNormalizeDeveloperSegment (5 tests)
    • test_normalize_none
    • test_normalize_empty_string
    • test_normalize_with_at_symbol
    • test_normalize_regular_string
    • test_normalize_non_string
  • TestNormalizeModelSegment (5 tests)
    • test_normalize_none
    • test_normalize_empty_string
    • test_normalize_preserves_casing
    • test_normalize_strips_whitespace
    • test_normalize_non_string
  • TestAnnotateProviderSources (7 tests)
    • test_annotate_empty_list
    • test_annotate_none
    • test_annotate_single_provider
    • test_annotate_multiple_providers
    • test_annotate_preserves_existing_data
    • test_annotate_doesnt_duplicate_source
    • test_annotate_adds_new_source

tests/routes/test_chat.py

35 tests

  • test_happy_path_openrouter — Test successful chat completion with OpenRouter
  • test_invalid_api_key — Test that invalid API key returns 401
  • test_plan_limit_exceeded_precheck — Test that plan limit exceeded returns 429
  • test_rate_limit_exceeded_precheck — Test that rate limit exceeded returns 429
  • test_insufficient_credits_non_trial — Test that insufficient credits returns 402
  • test_trial_valid_usage_tracked — Test that trial user usage is tracked correctly
  • test_trial_expired_403 — Test that expired trial returns 403
  • test_expired_trial_can_use_free_model — Test that expired trial can still access free models (with :free suffix)
  • test_trial_limits_exceeded_can_use_free_model — Test that trial with exceeded limits can still access free models
  • test_expired_trial_blocked_for_non_free_model — Test that expired trial is blocked for non-free models
  • test_valid_trial_free_model_usage_tracked — Test that free model usage is tracked for valid trials too
  • test_paid_user_free_model_usage_tracked — Test that free model usage is tracked for paid users
  • test_is_free_model_with_free_suffix — Test that models with :free suffix are detected as free
  • test_is_free_model_without_free_suffix — Test that models without :free suffix are not detected as free
  • test_is_free_model_edge_cases — Test edge cases for is_free_model
  • test_validate_trial_with_free_model_bypass_does_not_mutate_original_dict — Test that validate_trial_with_free_model_bypass does not mutate the original trial dict.
  • test_validate_trial_with_free_model_bypass_cache_isolation — Test that multiple calls don't share state through the original dict.
  • test_upstream_429_maps_429 — Test that upstream 429 error is properly mapped to 429
  • test_upstream_401_maps_500_in_your_code — Test that upstream 401 error is mapped to 500
  • test_upstream_request_error_maps_503 — Test that upstream request error is mapped to 503
  • test_upstream_timeout_maps_504 — Test that upstream timeout is handled properly
  • test_saves_chat_history_when_session_id — Test that chat history is saved when session_id is provided
  • test_streaming_response — Test streaming response
  • test_provider_failover_to_huggingface — Test provider failover from featherless to huggingface
  • test_provider_failover_on_404_to_huggingface — Test provider failover on 404 from featherless to huggingface
  • test_response_with_none_choices — Test that responses with None choices are handled safely (no NoneType error)
  • test_response_with_empty_choices — Test that responses with empty choices are handled safely
  • test_response_with_none_message — Test that responses with None message in choices are handled safely
  • test_happy_path_onerouter — Test successful chat completion with OneRouter provider
  • test_onerouter_streaming_response — Test streaming response with OneRouter provider
  • test_onerouter_upstream_error_handling — Test that OneRouter upstream errors are properly handled
  • test_onerouter_versioned_model_format — Test OneRouter with @ versioned model format (e.g., claude-3-5-sonnet@20240620)
  • test_onerouter_network_error_handling — Test that OneRouter network errors are properly handled
  • test_fal_model_rejected_in_chat_completions — Test that FAL models (image/video generation only) are rejected with a clear error in chat completions
  • test_fal_model_rejected_various_models — Test that various FAL model IDs are rejected with a clear error

tests/routes/test_chat_auto_web_search.py

21 tests

  • TestAutoWebSearchParameter (5 tests)
    • test_auto_web_search_default_is_auto — Test that auto_web_search defaults to 'auto'.
    • test_auto_web_search_can_be_true — Test that auto_web_search can be set to True.
    • test_auto_web_search_can_be_false — Test that auto_web_search can be set to False.
    • test_web_search_threshold_default — Test that web_search_threshold defaults to 0.5.
    • test_web_search_threshold_validation — Test that web_search_threshold is validated between 0 and 1.
  • TestQueryClassifierIntegration (3 tests)
    • test_travel_query_triggers_classification — Test that travel queries are classified correctly.
    • test_code_query_does_not_trigger_search — Test that code queries don't trigger search.
    • test_greeting_does_not_trigger_search — Test that simple greetings don't trigger search.
  • TestMessageAugmentation (3 tests)
    • test_search_context_format — Test that search context is formatted correctly.
    • test_system_message_insertion_logic — Test that system message is inserted after existing system messages.
    • test_system_message_insertion_no_existing_system — Test system message insertion when no existing system message.
  • TestAutoWebSearchModes (3 tests)
    • test_auto_mode_uses_classifier — Test that 'auto' mode uses the query classifier.
    • test_explicit_true_always_searches — Test that auto_web_search=True would always search.
    • test_explicit_false_never_searches — Test that auto_web_search=False never searches.
  • TestErrorHandling (3 tests)
    • test_search_failure_continues_request — Test that search failure doesn't break the request.
    • test_classifier_exception_handled — Test that classifier exceptions are handled gracefully.
    • test_empty_search_results_handled — Test handling of empty search results.
  • TestRealWorldScenarios (2 tests)
    • test_various_queries — Test classification of various query types.
    • test_conversation_context_uses_last_message — Test that classification uses the last user message.
  • TestThresholdBehavior (2 tests)
    • test_low_threshold_more_searches — Test that lower threshold triggers more searches.
    • test_threshold_boundary_cases — Test behavior at threshold boundaries.

tests/routes/test_chat_butter_integration.py

14 tests

  • TestButterProviderConfig (7 tests)
    • test_openrouter_config_exists — Test that OpenRouter config is defined.
    • test_together_config_exists — Test that Together config is defined.
    • test_fireworks_config_exists — Test that Fireworks config is defined.
    • test_groq_config_exists — Test that Groq config is defined.
    • test_onerouter_config_exists — Test that OneRouter/Infron config is defined - this is the default provider.
    • test_all_compatible_providers_have_config — Test that all providers in BUTTER_COMPATIBLE_PROVIDERS have a config entry.
    • test_all_configs_have_required_fields — Test that all provider configs have required fields.
  • TestMakeButterProxiedStream (3 tests)
    • test_raises_for_unknown_provider — Test that unknown provider raises ValueError.
    • test_raises_for_missing_api_key — Test that missing API key raises ValueError.
    • test_creates_butter_client_correctly — Test that Butter client is created with correct parameters.
  • TestButterIntegrationInChatCompletions (3 tests)
    • test_should_use_butter_cache_is_imported — Test that should_use_butter_cache is available in chat module.
    • test_butter_cache_timer_is_imported — Test that ButterCacheTimer is available in chat module.
    • test_get_butter_pooled_async_client_is_imported — Test that get_butter_pooled_async_client is available in chat module.
  • TestButterCacheHeaderLogic (1 tests)
    • test_butter_reason_codes — Test all possible reason codes from should_use_butter_cache.

tests/routes/test_chat_completions_comprehensive.py

16 tests

  • TestChatCompletionsBasic (3 tests)
    • test_chat_completions_simple_request — Test simple chat completions request with minimal parameters.
    • test_chat_completions_system_message — Test chat completions with system message.
    • test_chat_completions_multiple_messages — Test chat completions with conversation history.
  • TestChatCompletionsParameters (6 tests)
    • test_chat_completions_with_temperature — Test chat completions with temperature parameter.
    • test_chat_completions_with_top_p — Test chat completions with top_p parameter.
    • test_chat_completions_with_frequency_penalty — Test chat completions with frequency penalty.
    • test_chat_completions_with_presence_penalty — Test chat completions with presence penalty.
    • test_chat_completions_with_seed — Test chat completions with seed parameter for reproducibility.
    • test_chat_completions_with_user_identifier — Test chat completions with user identifier.
  • TestChatCompletionsStreaming (1 tests)
    • test_chat_completions_streaming — Test streaming chat completions.
  • TestChatCompletionsResponseStructure (2 tests)
    • test_chat_completions_response_structure — Test that chat completions response has correct structure.
    • test_chat_completions_usage_information — Test that response includes usage information when available.
  • TestChatCompletionsErrorHandling (3 tests)
    • test_chat_completions_missing_model — Test error when model is missing.
    • test_chat_completions_missing_messages — Test error when messages are missing.
    • test_chat_completions_invalid_auth — Test error with invalid authentication.
  • TestChatCompletionsPerformance (1 tests)
    • test_chat_completions_response_time — Test that chat completions responds in reasonable time.

tests/routes/test_chat_comprehensive.py

20 tests

  • test_chat_completions_no_api_key — Test chat endpoint without API key - anonymous access is now allowed
  • test_chat_completions_invalid_api_key — Test chat endpoint with invalid API key
  • test_chat_completions_success — Test successful chat completion
  • test_chat_completions_with_optional_params — Test chat completion with optional parameters
  • test_chat_completions_credits_deducted — Test that credits are properly deducted
  • test_chat_completions_streaming_success — Test successful streaming chat completion
  • test_chat_completions_streaming_anonymous_success — Test streaming chat completion for anonymous users (no API key).
  • test_chat_completions_insufficient_credits — Test chat endpoint with insufficient credits
  • test_chat_completions_trial_user_success — Test chat endpoint with trial user
  • test_chat_completions_trial_expired — Test chat endpoint with expired trial
  • test_chat_completions_missing_model — Test chat endpoint without model parameter
  • test_chat_completions_missing_messages — Test chat endpoint without messages parameter
  • test_chat_completions_empty_messages — Test chat endpoint with empty messages array
  • test_unified_responses_success — Test /v1/responses endpoint
  • test_unified_responses_with_multimodal_input — Test /v1/responses with multimodal input
  • test_unified_responses_with_output_text_transformation — Test /v1/responses properly transforms output_text content type to text.
  • test_chat_completions_with_session_id — Test chat endpoint with session ID for history
  • test_chat_completions_with_specific_provider — Test chat endpoint with specific provider
  • test_chat_completions_very_long_message — Test chat endpoint with very long message
  • test_chat_completions_multiple_messages — Test chat endpoint with conversation history

tests/routes/test_chat_function_calling.py

3 tests

  • TestChatCompletionsFunctionCalling (3 tests)
    • test_tools_parameter_extracted — Test that tools parameter is correctly extracted from request
    • test_tools_parameter_validation — Test that invalid tools parameter is rejected
    • test_multiple_tools_passed_through — Test that multiple tools are passed through correctly

tests/routes/test_chat_history.py

1 tests

  • test_chat_history_data — Check if chat history data exists in database

tests/routes/test_chat_history_performance.py

9 tests

  • TestChatHistoryPerformance (9 tests)
    • test_create_session_uses_cached_user_lookup — create_session should use cached user lookup
    • test_create_session_uses_background_logging — create_session should use background activity logging
    • test_create_session_logs_performance_metrics — create_session should log performance metrics
    • test_create_session_returns_success_response — create_session should return successful response
    • test_create_session_invalid_api_key — create_session should return 401 for invalid API key
    • test_create_session_handles_background_logging_failure — create_session should handle background logging failures gracefully
    • test_create_session_background_logging_includes_metadata — create_session should pass metadata to background logging
    • test_create_session_with_default_model — create_session should work without explicit model
    • test_create_session_with_empty_title — create_session should generate title if not provided

tests/routes/test_chat_metrics.py

18 tests

  • TestTokensPerSecondEndpoints (12 tests)
    • test_get_all_tokens_per_second_valid — Test /tokens-per-second/all with valid parameters.
    • test_get_all_tokens_per_second_missing_provider — Test /tokens-per-second/all without provider_id parameter.
    • test_get_all_tokens_per_second_missing_model — Test /tokens-per-second/all without model_id parameter.
    • test_get_tokens_per_second_valid_hour — Test /tokens-per-second with valid hour parameter.
    • test_get_tokens_per_second_valid_week — Test /tokens-per-second with valid week parameter.
    • test_get_tokens_per_second_valid_month — Test /tokens-per-second with valid month parameter.
    • test_get_tokens_per_second_valid_1year — Test /tokens-per-second with valid 1year parameter.
    • test_get_tokens_per_second_valid_2year — Test /tokens-per-second with valid 2year parameter.
    • test_get_tokens_per_second_invalid_time — Test /tokens-per-second with invalid time parameter.
    • test_get_tokens_per_second_missing_time — Test /tokens-per-second without time parameter.
    • test_get_tokens_per_second_missing_model — Test /tokens-per-second without model_id parameter.
    • test_get_tokens_per_second_missing_provider — Test /tokens-per-second without provider_id parameter.
  • TestPrometheusFormat (3 tests)
    • test_prometheus_format_valid_structure — Test that response follows Prometheus text format.
    • test_prometheus_format_labels — Test that metrics include proper labels.
    • test_content_type_is_plain_text — Test that content-type is text/plain.
  • TestErrorHandling (3 tests)
    • test_invalid_model_id_type — Test error handling with invalid model_id type.
    • test_empty_provider_id — Test error handling with empty provider_id.
    • test_server_error_handling — Test that server errors return 500.

tests/routes/test_chat_requests_endpoints.py

23 tests

  • TestChatRequestsCountsEndpoint (6 tests)
    • test_counts_endpoint_returns_200 — Test that counts endpoint returns 200 OK
    • test_counts_endpoint_response_structure — Test that response has correct structure
    • test_counts_endpoint_uses_real_data — Test that endpoint returns real data from database (not mock)
    • test_counts_endpoint_data_is_sorted — Test that data is sorted by request count (descending)
    • test_counts_endpoint_metadata_accuracy — Test that metadata totals match actual data
    • test_counts_endpoint_timestamp_is_valid — Test that timestamp is valid ISO-8601
  • TestChatRequestsModelsEndpoint (5 tests)
    • test_models_endpoint_returns_200 — Test that models endpoint returns 200 OK
    • test_models_endpoint_response_structure — Test that response has correct structure
    • test_models_endpoint_returns_real_model_data — Test that endpoint returns real models with stats (not mock)
    • test_models_endpoint_provider_data — Test that provider information is included and real
    • test_models_endpoint_sorted_by_requests — Test that models are sorted by request count (descending)
  • TestChatRequestsEndpoint (8 tests)
    • test_chat_requests_returns_200 — Test that endpoint returns 200 OK
    • test_chat_requests_response_structure — Test that response has correct structure
    • test_chat_requests_returns_real_requests — Test that endpoint returns real request data (not mock)
    • test_chat_requests_pagination — Test that pagination works correctly
    • test_chat_requests_filter_by_model_id — Test filtering by model_id
    • test_chat_requests_filter_by_model_name — Test filtering by model name
    • test_chat_requests_returned_count_matches_data — Test that returned_count matches actual data length
    • test_chat_requests_limit_validation — Test that limit parameter is validated
  • TestChatRequestsDataIntegrity (4 tests)
    • test_counts_and_models_consistency — Test that counts and models endpoints return consistent data
    • test_all_endpoints_return_real_timestamps — Test that all endpoints return valid timestamps
    • test_no_mock_data_markers — Test that responses don't contain mock/test data markers
    • test_success_flag_always_true_on_200 — Test that 200 responses always have success=true

tests/routes/test_chat_trial_status_override.py

6 tests

  • TestTrialStatusDefenseInDepth (6 tests)
    • test_trial_override_with_active_stripe_subscription — Test that user with is_trial=TRUE but active Stripe subscription is charged
    • test_trial_override_with_pro_tier — Test that user with is_trial=TRUE but pro tier is charged
    • test_trial_override_with_max_tier — Test that user with is_trial=TRUE but max tier is charged
    • test_trial_override_with_admin_tier — Test that user with is_trial=TRUE but admin tier is charged
    • test_legitimate_trial_user_not_charged — Test that legitimate trial users are NOT charged
    • test_paid_user_with_correct_is_trial_false — Test that paid users with correct is_trial=FALSE are charged normally

tests/routes/test_coupons.py

2 tests

  • TestCouponsRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_credits.py

15 tests

  • TestCreditsAdd (3 tests)
    • test_add_credits_success — Successfully add credits to a user
    • test_add_credits_user_not_found — Return 404 when user not found
    • test_add_credits_negative_amount — Reject negative credit amounts
  • TestCreditsAdjust (2 tests)
    • test_adjust_credits_positive — Successfully add credits via adjustment
    • test_adjust_credits_negative_prevents_overdraft — Prevent adjustment that would result in negative balance
  • TestCreditsBulkAdd (3 tests)
    • test_bulk_add_credits_success — Successfully add credits to multiple users with different balances
    • test_bulk_add_deduplicates_user_ids — Duplicate user IDs are deduplicated to prevent incorrect balance tracking
    • test_bulk_add_empty_list — Reject empty user list
  • TestCreditsRefund (1 tests)
    • test_refund_credits_success — Successfully refund credits to a user
  • TestCreditsSummary (2 tests)
    • test_get_summary_for_user — Get credit summary for specific user
    • test_get_system_summary — Get system-wide credit summary
  • TestCreditsTransactions (3 tests)
    • test_get_transactions_default — Get transactions with default parameters
    • test_get_transactions_with_filters — Get transactions with filters applied
    • test_get_transactions_invalid_direction — Reject invalid direction filter
  • TestCreditsAuthorization (1 tests)
    • test_add_credits_requires_admin — Credits endpoints require admin authentication

tests/routes/test_error_monitor.py

2 tests

  • TestErrorMonitorRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_feedback.py

22 tests

  • test_submit_thumbs_up_feedback — Test submitting thumbs up feedback
  • test_submit_thumbs_down_with_comment — Test submitting thumbs down feedback with comment
  • test_submit_regenerate_feedback — Test submitting regenerate feedback
  • test_submit_feedback_invalid_type — Test submitting feedback with invalid type returns 422 (Pydantic validation)
  • test_submit_feedback_invalid_rating — Test submitting feedback with invalid rating returns 422 (Pydantic validation)
  • test_submit_feedback_session_not_found — Test submitting feedback with non-existent session returns 404
  • test_submit_feedback_message_not_owned — Test submitting feedback with message_id not owned by user returns 404
  • test_submit_feedback_message_not_in_session — Test submitting feedback with message_id not in specified session returns 404
  • test_submit_feedback_unauthorized — Test submitting feedback without auth returns 401
  • test_get_user_feedback_all — Test getting all user feedback
  • test_get_user_feedback_filtered_by_type — Test getting user feedback filtered by type
  • test_get_user_feedback_filtered_by_session — Test getting user feedback filtered by session
  • test_get_user_feedback_pagination — Test getting user feedback with pagination
  • test_get_feedback_stats — Test getting feedback statistics
  • test_get_feedback_stats_with_model_filter — Test getting feedback statistics filtered by model
  • test_get_session_feedback — Test getting feedback for a specific session
  • test_get_session_feedback_not_found — Test getting feedback for non-existent session returns 404
  • test_update_feedback_success — Test updating feedback successfully
  • test_update_feedback_not_found — Test updating non-existent feedback returns 404
  • test_update_feedback_invalid_type — Test updating feedback with invalid type returns 422 (Pydantic validation)
  • test_delete_feedback_success — Test deleting feedback successfully
  • test_delete_feedback_not_found — Test deleting non-existent feedback returns 404

tests/routes/test_gemini_routing_fix.py

3 tests

  • test_gemini_provider_detection_logic — Unit test for the provider detection logic used in chat.py
  • test_gemini_with_vertex_credentials — Test that google/gemini models route to Google Vertex when credentials exist.
  • test_various_gemini_models_route_correctly — Test that various Google/Gemini model IDs are routed correctly.

tests/routes/test_gpt5_pricing.py

16 tests

  • TestGPT51ModelAvailability (1 tests)
    • test_gpt51_in_openrouter_models — Test that GPT-5.1 is returned from OpenRouter models endpoint
  • TestGPT51Pricing (4 tests)
    • test_gpt51_pricing_structure — Test that GPT-5.1 pricing has correct structure
    • test_gpt51_pricing_lookup — Test that pricing can be looked up for GPT-5.1
    • test_gpt51_pricing_lookup_hyphen_alias — Test that pricing lookup works for hyphenated GPT-5-1 alias
    • test_gpt51_cost_calculation — Test that costs are calculated correctly for GPT-5.1
  • TestGPT51ModelCatalogEndpoints (3 tests)
    • test_models_endpoint_includes_gpt51 — Test that /v1/models includes GPT-5.1
    • test_provider_endpoint_openai_includes_gpt51 — Test that /v1/provider returns OpenAI provider with GPT-5.1
    • test_catalog_page_loads_gpt51 — Test that models catalog page loads with GPT-5.1
  • TestGPT51DynamicPricing (3 tests)
    • test_fetch_gpt51_from_openrouter_api — Test that GPT-5.1 can be fetched from OpenRouter API
    • test_gpt51_pricing_sanitization_filters_dynamic — Test that negative pricing values return None (filter model)
    • test_gpt51_pricing_sanitization_all_negative_filters — Test sanitization when all pricing is dynamic (-1)
  • TestGPT51Variants (1 tests)
    • test_all_gpt5_variants_have_pricing — Test that all GPT-5 variants have pricing
  • TestGPT51CostEstimation (3 tests)
    • test_estimate_gpt51_request_cost — Test cost estimation for typical GPT-5.1 requests
    • test_large_gpt51_request_cost — Test cost estimation for large GPT-5.1 requests
    • test_gpt51_hyphen_alias_cost — Test cost estimation works for hyphen alias input
  • TestGPT51Integration (1 tests)
    • test_gpt51_in_full_catalog_response — Test GPT-5.1 appears in full catalog response

tests/routes/test_health.py

49 tests

  • TestBasicHealthCheck (2 tests)
    • test_health_check_no_auth_required — Health check doesn't require authentication
    • test_health_check_response_format — Health check returns valid JSON
  • TestQuickHealthCheck (5 tests)
    • test_quick_health_returns_200 — Quick health check returns 200 OK
    • test_quick_health_returns_healthy_status — Quick health check returns healthy status
    • test_quick_health_has_timestamp — Quick health check includes timestamp
    • test_quick_health_no_auth_required — Quick health check doesn't require authentication
    • test_quick_health_response_is_minimal — Quick health check returns minimal response (no database info)
  • TestSystemHealth (4 tests)
    • test_system_health_success — Successfully get system health metrics
    • test_system_health_requires_auth — System health requires authentication
    • test_system_health_no_data_available — System health handles no data gracefully
    • test_system_health_error_captured_to_sentry — Test that system health errors are captured to Sentry
  • TestProvidersHealth (2 tests)
    • test_get_all_providers_health — Get health for all providers
    • test_filter_providers_by_gateway — Filter providers by gateway parameter
  • TestModelsHealth (3 tests)
    • test_get_all_models_health — Get health for all models
    • test_filter_models_by_provider — Filter models by provider parameter
    • test_filter_models_by_status — Filter models by status parameter
  • TestSpecificModelHealth (2 tests)
    • test_get_model_health_success — Get health for specific model
    • test_get_model_health_not_found — Model not found returns 404
  • TestSpecificProviderHealth (2 tests)
    • test_get_provider_health_success — Get health for specific provider
    • test_get_provider_health_not_found — Provider not found returns 404
  • TestHealthSummary (1 tests)
    • test_get_health_summary — Get comprehensive health summary
  • TestHealthCheck (2 tests)
    • test_perform_health_check — Trigger background health check
    • test_perform_immediate_health_check — Perform immediate health check
  • TestUptimeMetrics (3 tests)
    • test_get_uptime_metrics — Get uptime metrics for status page
    • test_get_uptime_metrics_error_captured_to_sentry — Test that uptime metrics errors are captured to Sentry but return graceful degradation
    • test_get_uptime_metrics_no_data_available — Test that uptime metrics handles no data gracefully
  • TestHealthDashboard (2 tests)
    • test_get_health_dashboard — Get complete health dashboard
    • test_get_health_dashboard_error_captured_to_sentry — Test that dashboard errors are captured to Sentry
  • TestHealthStatus (1 tests)
    • test_get_health_status — Get simple health status
  • TestMonitoringControls (3 tests)
    • test_get_monitoring_status — Get monitoring service status
    • test_start_health_monitoring — Start health monitoring service
    • test_stop_health_monitoring — Stop health monitoring service
  • TestHealthErrorHandling (1 tests)
    • test_system_health_error_handling — Handle errors in system health gracefully
  • TestHealthEdgeCases (3 tests)
    • test_empty_models_list — Handle empty models list
    • test_empty_providers_list — Handle empty providers list
    • test_invalid_model_id_special_chars — Handle special characters in model ID
  • TestModelHealthMonitorScheduling (2 tests)
    • test_get_models_to_check_returns_full_gateway_catalog
    • test_perform_health_checks_batches_models
  • TestDatabaseHealth (5 tests)
    • test_database_health_success — Test database health returns success with initialization status
    • test_database_health_failure — Test database health returns failure with error details
    • test_database_health_captures_to_sentry — Test database health errors are captured to Sentry
    • test_database_health_timeout — Test database health returns degraded status when query times out
    • test_database_health_timeout_constant_exists — Test that HEALTH_CHECK_TIMEOUT_SECONDS constant is defined
  • TestHealthAllEndpoint (2 tests)
    • test_get_all_health_success — Successfully get all health information
    • test_get_all_health_degraded — Return degraded status when providers are unhealthy
  • TestHealthModelsStatsEndpoint (2 tests)
    • test_get_models_stats_success — Successfully get model health stats
    • test_get_models_stats_empty — Handle empty model health data
  • TestHealthProvidersStatsEndpoint (2 tests)
    • test_get_providers_stats_success — Successfully get provider health stats
    • test_get_providers_stats_empty — Handle empty provider health data

tests/routes/test_health_timeline_import.py

4 tests

  • test_health_timeline_imports_successfully — Test that health_timeline module imports without errors.
  • test_health_timeline_uses_correct_supabase_import — Test that health_timeline uses get_supabase_client (not get_supabase_admin).
  • test_supabase_config_exports — Test that supabase_config exports the correct functions.
  • test_health_timeline_endpoints_defined — Test that health timeline endpoints are properly defined.

tests/routes/test_images.py

27 tests

  • TestImageGenerationSuccess (3 tests)
    • test_generate_image_deepinfra_success — Test successful image generation with DeepInfra
    • test_generate_multiple_images — Test generating multiple images
    • test_generate_image_fal_success — Test successful image generation with Fal.ai
  • TestImageGenerationAuth (2 tests)
    • test_generate_image_no_auth_header — Test request without Authorization header
    • test_generate_image_invalid_api_key — Test request with invalid API key
  • TestImageGenerationCredits (2 tests)
    • test_generate_image_insufficient_credits — Test request with insufficient credits
    • test_generate_image_insufficient_credits_multiple — Test request for multiple images with insufficient credits
  • TestImageGenerationValidation (4 tests)
    • test_generate_image_missing_prompt — Test request without required prompt
    • test_generate_image_empty_prompt — Test request with empty prompt
    • test_generate_image_invalid_size — Test request with invalid size
    • test_generate_image_invalid_n — Test request with invalid n value
  • TestImageGenerationProviders (2 tests)
    • test_default_provider_is_deepinfra — Test that DeepInfra is the default provider
    • test_unsupported_provider_error — Test error handling for unsupported providers
  • TestImageGenerationResponseProcessing (2 tests)
    • test_response_includes_gateway_usage — Test that response includes gateway usage metadata
    • test_response_timing_tracked — Test that request timing is tracked
  • TestImageGenerationErrorHandling (3 tests)
    • test_provider_error_handling — Test handling of provider errors
    • test_credit_deduction_failure_returns_402 — Test that credit deduction failures return 402 Payment Required.
    • test_unexpected_billing_error_returns_500 — Test that unexpected billing errors return 500 and don't give away free images.
  • TestImageGenerationPricing (9 tests)
    • test_get_image_cost_known_model — Test cost calculation for known model
    • test_get_image_cost_multiple_images — Test cost calculation for multiple images
    • test_get_image_cost_unknown_model_uses_provider_default — Test that unknown models use provider default pricing and flag as fallback
    • test_get_image_cost_unknown_provider_uses_conservative_default — Test that unknown providers use conservative high default to avoid revenue loss
    • test_get_image_cost_fal_flux_models — Test pricing for Fal flux models
    • test_get_image_cost_resolution_multipliers — Test resolution-aware pricing multipliers
    • test_get_image_cost_unknown_size_uses_default_multiplier — Test that unknown sizes default to 1.0 multiplier
    • test_get_image_cost_none_size_uses_default_multiplier — Test backwards compatibility: None size uses 1.0 multiplier
    • test_get_image_cost_resolution_with_multiple_images — Test resolution multiplier applies correctly to multiple images

tests/routes/test_instrumentation.py

20 tests

  • TestInstrumentationHealth (3 tests)
    • test_health_returns_status — Test health endpoint returns expected structure.
    • test_health_includes_loki_config — Test health endpoint includes Loki configuration.
    • test_health_includes_tempo_config — Test health endpoint includes Tempo configuration.
  • TestTraceContext (1 tests)
    • test_trace_context_returns_ids — Test trace context returns trace and span IDs.
  • TestOtelInitialize (3 tests)
    • test_initialize_requires_auth — Test initialize endpoint requires admin API key.
    • test_initialize_with_valid_auth — Test initialize endpoint with valid admin API key.
    • test_initialize_already_initialized — Test initialize endpoint when OTel is already initialized.
  • TestOtelReinitialize (2 tests)
    • test_reinitialize_requires_auth — Test reinitialize endpoint requires admin API key.
    • test_reinitialize_shuts_down_existing — Test reinitialize shuts down existing provider before reinitializing.
  • TestOtelStatus (2 tests)
    • test_status_requires_auth — Test status endpoint requires admin API key.
    • test_status_returns_config_info — Test status endpoint returns configuration information.
  • TestStartupRetryLogic (2 tests)
    • test_retry_logic_eventually_succeeds — Test that retry logic eventually succeeds when Tempo becomes available.
    • test_retry_logic_gives_up_after_max_attempts — Test that retry logic gives up after max attempts.
  • TestEndpointReachabilityCheck (4 tests)
    • test_check_endpoint_reachable_valid_endpoint — Test reachability check with valid endpoint.
    • test_check_endpoint_reachable_dns_failure — Test reachability check with DNS failure.
    • test_check_endpoint_reachable_connection_refused — Test reachability check with connection refused.
    • test_check_endpoint_reachable_timeout — Test reachability check with timeout.
  • TestRailwayEndpointHandling (3 tests)
    • test_railway_internal_dns_port_handling — Test that Railway internal DNS URLs get port 4318 added.
    • test_railway_public_url_removes_port — Test that Railway public URLs have ports removed.
    • test_railway_public_url_ensures_https — Test that Railway public URLs use HTTPS.

tests/routes/test_messages.py

44 tests

  • TestAnthropicTransformer (13 tests)
    • test_transform_anthropic_to_openai_basic — Test basic message transformation
    • test_transform_anthropic_to_openai_with_system — Test transformation with system message
    • test_transform_anthropic_to_openai_with_params — Test transformation with all parameters
    • test_transform_anthropic_to_openai_content_blocks — Test transformation with content blocks
    • test_transform_anthropic_to_openai_single_text_block — Test transformation with single text block (should unwrap)
    • test_transform_anthropic_to_openai_image_blocks — Test transformation with image content blocks
    • test_transform_openai_to_anthropic_basic — Test OpenAI to Anthropic response transformation
    • test_transform_openai_to_anthropic_finish_reasons — Test finish reason mapping
    • test_transform_openai_to_anthropic_tool_calls — Test OpenAI to Anthropic transformation with tool_calls
    • test_extract_text_from_string_content — Test extracting text from string content
    • test_extract_text_from_content_blocks — Test extracting text from content blocks
    • test_extract_text_from_mixed_blocks — Test extracting text from mixed content blocks
    • test_extract_text_from_empty_blocks — Test extracting text from empty content
  • TestMessagesEndpointSuccess (1 tests)
    • test_messages_endpoint_basic_success — Test successful Claude API message completion
  • TestMessagesEndpointAuth (2 tests)
    • test_messages_endpoint_no_auth_header — Test request without Authorization header
    • test_messages_endpoint_invalid_api_key — Test request with invalid API key
  • TestMessagesEndpointCredits (1 tests)
    • test_messages_endpoint_insufficient_credits — Test request with insufficient credits
  • TestMessagesEndpointRateLimiting (1 tests)
    • test_messages_endpoint_rate_limit_exceeded — Test rate limit exceeded
  • TestMessagesEndpointPlanLimits (1 tests)
    • test_messages_endpoint_plan_limit_exceeded — Test plan limit exceeded
  • TestMessagesEndpointTrialValidation (1 tests)
    • test_messages_endpoint_trial_expired — Test expired trial access
  • TestMessagesEndpointValidation (3 tests)
    • test_messages_endpoint_missing_max_tokens — Test request without required max_tokens
    • test_messages_endpoint_empty_messages — Test request with empty messages array
    • test_messages_endpoint_invalid_role — Test request with invalid message role
  • TestMessagesEndpointFailover (1 tests)
    • test_messages_endpoint_provider_failover_success — Test successful failover to backup provider
  • TestAnthropicTransformerNewFeatures (9 tests)
    • test_transform_system_as_array — Test transformation with system as array of content blocks
    • test_transform_document_content_block — Test transformation of document content blocks
    • test_transform_tool_result_content_block — Test transformation of tool_result content blocks
    • test_transform_tool_choice_types — Test transformation of different tool_choice types
    • test_transform_tools_to_openai_format — Test transformation of Anthropic tool definitions to OpenAI format
    • test_transform_assistant_tool_use_message — Test transformation of assistant messages with tool_use blocks
    • test_transform_openai_to_anthropic_with_stop_sequence — Test response transformation detecting stop sequences
    • test_transform_openai_to_anthropic_with_thinking — Test response transformation with reasoning/thinking content
    • test_transform_openai_to_anthropic_refusal — Test response transformation mapping content_filter to refusal
  • TestMessagesRequestSchema (8 tests)
    • test_valid_request_with_string_system — Test valid request with string system prompt
    • test_valid_request_with_array_system — Test valid request with array system prompt
    • test_valid_request_with_service_tier — Test valid request with service_tier parameter
    • test_valid_request_with_thinking_config — Test valid request with thinking configuration
    • test_thinking_config_budget_validation — Test thinking budget_tokens validation (must be >= 1024)
    • test_temperature_validation — Test temperature must be between 0.0 and 1.0
    • test_content_block_types — Test ContentBlock supports all required types
    • test_tool_choice_models — Test tool choice model types
  • TestMessagesResponseSchema (3 tests)
    • test_response_schema_structure — Test MessagesResponse has correct structure
    • test_usage_response_with_cache_fields — Test UsageResponse with cache-related fields
    • test_stop_reason_values — Test all valid stop_reason values

tests/routes/test_monitoring.py

29 tests

  • TestHealthEndpoints (2 tests)
    • test_get_all_provider_health — Test getting health scores for all providers
    • test_get_provider_health — Test getting health score for specific provider
  • TestErrorEndpoints (2 tests)
    • test_get_provider_errors — Test getting recent errors for a provider
    • test_get_provider_errors_with_limit — Test error endpoint with custom limit
  • TestStatsEndpoints (2 tests)
    • test_get_realtime_stats — Test getting real-time statistics
    • test_get_hourly_stats — Test getting hourly stats for a provider
  • TestCircuitBreakerEndpoints (2 tests)
    • test_get_all_circuit_breakers — Test getting all circuit breaker states
    • test_get_provider_circuit_breakers — Test getting circuit breakers for specific provider
  • TestProviderComparisonEndpoint (1 tests)
    • test_get_provider_comparison — Test provider comparison endpoint
  • TestLatencyEndpoints (3 tests)
    • test_get_latency_percentiles — Test getting latency percentiles
    • test_get_latency_percentiles_custom — Test latency percentiles with custom percentiles
    • test_get_latency_trends — Test getting latency trends
  • TestAnomalyEndpoints (1 tests)
    • test_get_anomalies — Test anomaly detection endpoint
  • TestBusinessMetricsEndpoints (4 tests)
    • test_get_trial_analytics — Test trial analytics endpoint
    • test_get_cost_analysis — Test cost analysis endpoint
    • test_get_error_rates — Test error rates endpoint
    • test_get_token_efficiency — Test token efficiency endpoint
  • TestSentryTunnelEndpoint (12 tests)
    • test_sentry_tunnel_empty_body — Test that empty body returns 400
    • test_sentry_tunnel_invalid_envelope — Test that invalid envelope format returns 400
    • test_sentry_tunnel_no_dsn — Test that envelope without DSN returns 400
    • test_sentry_tunnel_blocked_host — Test that non-Sentry hosts are blocked
    • test_sentry_tunnel_ssrf_prevention_suffix_attack — Test that SSRF attacks via suffix matching are blocked (e.g., evil-sentry.io)
    • test_sentry_tunnel_ssrf_prevention_malicious_subdomain — Test that SSRF attacks via malicious subdomains are blocked
    • test_sentry_tunnel_null_hostname — Test that malformed DSN with null hostname is rejected
    • test_sentry_tunnel_non_dict_json — Test that non-dict JSON envelope returns 400
    • test_sentry_tunnel_string_json — Test that string JSON envelope returns 400
    • test_sentry_tunnel_number_json — Test that number JSON envelope returns 400
    • test_sentry_tunnel_null_json — Test that null JSON envelope returns 400
    • test_sentry_tunnel_valid_envelope — Test that valid Sentry envelope is forwarded

tests/routes/test_nosana_routes.py

20 tests

  • TestNosanaCreditsEndpoints (1 tests)
    • test_get_credit_balance — Test getting credit balance
  • TestNosanaDeploymentEndpoints (3 tests)
    • test_list_deployments — Test listing deployments
    • test_get_deployment — Test getting deployment details
    • test_create_deployment — Test creating a deployment
  • TestNosanaQuickDeployEndpoints (3 tests)
    • test_deploy_llm_inference — Test deploying LLM inference
    • test_deploy_image_generation — Test deploying image generation
    • test_deploy_whisper — Test deploying Whisper transcription
  • TestNosanaJobsEndpoints (4 tests)
    • test_create_job — Test creating a job
    • test_get_job — Test getting job details
    • test_extend_job — Test extending job duration
    • test_stop_job — Test stopping a job
  • TestNosanaMarketsEndpoints (3 tests)
    • test_list_markets — Test listing markets
    • test_get_market — Test getting market details
    • test_get_market_resources — Test getting market resource requirements
  • TestNosanaConfigEndpoint (1 tests)
    • test_get_config — Test getting Nosana configuration
  • TestNosanaRouteValidation (5 tests)
    • test_deployment_create_model_validation — Test deployment creation model validation
    • test_deployment_create_timeout_bounds — Test deployment timeout bounds
    • test_deployment_replicas_bounds — Test deployment replicas bounds
    • test_job_create_model_validation — Test job creation model validation
    • test_llm_inference_job_create_validation — Test LLM inference job creation validation

tests/routes/test_notifications.py

2 tests

  • TestNotificationsRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_optimization_monitor.py

2 tests

  • TestOptimizationMonitorRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_partner_trials.py

12 tests

  • TestPartnerTrialsPublicEndpoints (4 tests)
    • test_get_partner_config_success — Test getting partner config for valid partner
    • test_get_partner_config_not_found — Test getting config for non-existent partner
    • test_check_partner_code_valid — Test checking a valid partner code
    • test_check_partner_code_invalid — Test checking a user referral code (not a partner code)
  • TestPartnerTrialsAuthenticatedEndpoints (6 tests)
    • test_start_partner_trial_success — Test starting a partner trial
    • test_start_partner_trial_invalid_partner — Test starting trial with invalid partner code
    • test_get_partner_trial_status_active — Test getting active partner trial status
    • test_get_partner_trial_status_no_trial — Test getting status when no partner trial exists
    • test_get_daily_limit_partner_user — Test getting daily limit for partner trial user
    • test_get_daily_limit_unlimited — Test getting daily limit for paid subscriber (unlimited)
  • TestPartnerTrialsAdminEndpoints (2 tests)
    • test_get_partner_analytics — Test getting partner analytics
    • test_force_expire_trial — Test force expiring a trial

tests/routes/test_payments.py

17 tests

  • TestStripeWebhooks (3 tests)
    • test_webhook_success — Test successful webhook processing
    • test_webhook_missing_signature — Test webhook without signature
    • test_webhook_invalid_signature — Test webhook with invalid signature
  • TestCheckoutSessions (3 tests)
    • test_create_checkout_session_success — Test successfully creating checkout session
    • test_create_checkout_session_validation_error — Test checkout session with validation error
    • test_get_checkout_session_success — Test retrieving checkout session
  • TestPaymentIntents (2 tests)
    • test_create_payment_intent_success — Test successfully creating payment intent
    • test_get_payment_intent_success — Test retrieving payment intent
  • TestCreditPackages (1 tests)
    • test_get_credit_packages_success — Test getting available credit packages
  • TestRefunds (2 tests)
    • test_create_refund_as_admin_success — Test admin creating refund
    • test_create_refund_as_non_admin_fails — Test non-admin cannot create refund
  • TestPaymentHistory (5 tests)
    • test_get_payment_history_success — Test getting payment history
    • test_get_payment_history_with_pagination — Test payment history with pagination
    • test_get_payment_details_success — Test getting specific payment details
    • test_get_payment_details_not_found — Test getting non-existent payment
    • test_get_payment_details_wrong_user — Test accessing another user's payment
  • TestPaymentsIntegration (1 tests)
    • test_complete_payment_flow — Test complete payment workflow

tests/routes/test_ping.py

2 tests

  • TestPingRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_plans.py

2 tests

  • TestPlansRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_provider_credits.py

9 tests

  • TestGetProviderCreditBalances (3 tests)
    • test_get_balances_as_admin — Test retrieving all provider credit balances as admin
    • test_get_balances_with_warning_status — Test retrieving balance with warning status
    • test_get_balances_unauthorized — Test that non-admin users cannot access credit balances
  • TestGetSpecificProviderBalance (3 tests)
    • test_get_openrouter_balance — Test retrieving OpenRouter specific balance
    • test_get_unsupported_provider — Test retrieving balance for unsupported provider
    • test_get_balance_with_error — Test handling errors during balance retrieval
  • TestClearProviderCreditCache (3 tests)
    • test_clear_all_cache — Test clearing all provider credit caches
    • test_clear_specific_provider_cache — Test clearing cache for specific provider
    • test_clear_cache_unauthorized — Test that non-admin users cannot clear cache

tests/routes/test_ranking.py

2 tests

  • TestRankingRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_rate_limits.py

15 tests

  • TestRateLimitsRoutes (3 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully
    • test_default_config_defined — Test that default rate limit config is defined
  • TestAdminRateLimitsConfig (3 tests)
    • test_get_config_default — Get default rate limit configuration
    • test_get_config_for_specific_key — Get rate limit config for specific API key
    • test_set_config_success — Successfully set rate limit configuration
  • TestAdminRateLimitsReset (2 tests)
    • test_reset_config_success — Successfully reset rate limit to defaults
    • test_reset_config_missing_key — Reset config requires api_key parameter
  • TestAdminRateLimitsUpdate (1 tests)
    • test_update_rate_limits_success — Successfully update rate limits
  • TestAdminRateLimitsDelete (1 tests)
    • test_delete_rate_limits_success — Successfully delete rate limits
  • TestAdminRateLimitsUsers (2 tests)
    • test_get_users_rate_limits — Get rate limits for all users
    • test_get_users_rate_limits_with_filters — Get rate limits with filters
  • TestAdminRateLimitsAlerts (1 tests)
    • test_get_alerts_success — Get rate limit alerts
  • TestAdminRateLimitsSystem (1 tests)
    • test_get_system_stats — Get system-wide rate limit stats
  • TestRateLimitsAuthorization (1 tests)
    • test_admin_endpoints_require_admin — Admin rate limit endpoints require admin authentication

tests/routes/test_referral.py

23 tests

  • TestReferralStats (3 tests)
    • test_get_referral_stats_success — Successfully get referral stats
    • test_get_referral_stats_requires_auth — Referral stats requires authentication
    • test_get_referral_stats_invalid_api_key — Invalid API key returns 401
  • TestReferralValidation (3 tests)
    • test_validate_referral_code_success — Successfully validate a referral code
    • test_validate_invalid_referral_code — Invalid referral code returns error
    • test_validate_referral_requires_code — Validate referral requires code parameter
  • TestReferralCodeGeneration (2 tests)
    • test_referral_code_format — Referral code should follow format
    • test_referral_code_uniqueness — Referral codes should be unique
  • TestSelfReferralPrevention (1 tests)
    • test_cannot_use_own_referral_code — User cannot use their own referral code
  • TestReferralRewards (3 tests)
    • test_referrer_reward_calculation — Calculate referrer reward correctly
    • test_first_purchase_bonus — Additional bonus on first purchase
    • test_total_referral_earnings — Calculate total referral earnings
  • TestReferralLimits (3 tests)
    • test_referral_code_usage_limit — Referral code should have usage limit
    • test_referral_code_exhausted — Exhausted referral code should be invalid
    • test_user_can_only_use_one_code — User can only use one referral code
  • TestReferralAnalytics (1 tests)
    • test_referral_list_includes_details — Referral stats should include referral details
  • TestReferralEdgeCases (4 tests)
    • test_referral_code_case_insensitive — Referral codes should be case insensitive
    • test_referral_code_with_spaces — Referral code with spaces should be trimmed
    • test_empty_referral_code — Empty referral code should be rejected
    • test_very_long_referral_code — Very long referral code should be rejected
  • TestInviteLink (1 tests)
    • test_invite_link_format — Invite link should have correct format
  • TestReferralSecurity (2 tests)
    • test_referral_code_sql_injection — SQL injection in referral code should be prevented
    • test_referral_code_xss_prevention — XSS in referral code should be prevented

tests/routes/test_responses.py

4 tests

  • test_chat_completions_endpoint — Test the legacy /v1/chat/completions endpoint
  • test_responses_endpoint — Test the new /v1/responses endpoint
  • test_responses_with_json_format — Test /v1/responses with JSON response format
  • test_streaming_responses — Test /v1/responses with streaming

tests/routes/test_roles.py

27 tests

  • TestUpdateUserRole (6 tests)
    • test_update_role_success — Test successfully updating user's role
    • test_update_role_without_reason — Test updating role without providing reason
    • test_update_role_invalid_role — Test updating to invalid role
    • test_update_role_failure — Test role update failure
    • test_update_role_exception — Test role update with exception
    • test_update_role_missing_fields — Test validation error for missing required fields
  • TestGetUserRoleInfo (4 tests)
    • test_get_user_role_info_success — Test retrieving user's role and permissions
    • test_get_user_role_info_not_found — Test retrieving role for non-existent user
    • test_get_user_role_info_no_permissions — Test user with role but no permissions
    • test_get_user_role_info_exception — Test exception handling
  • TestRoleAuditLog (5 tests)
    • test_get_audit_log_all — Test retrieving all audit log entries
    • test_get_audit_log_by_user — Test retrieving audit log for specific user
    • test_get_audit_log_with_limit — Test retrieving audit log with custom limit
    • test_get_audit_log_empty — Test retrieving empty audit log
    • test_get_audit_log_exception — Test exception handling
  • TestListUsersByRole (5 tests)
    • test_list_users_by_role_success — Test listing users with specific role
    • test_list_users_by_role_with_limit — Test listing users with custom limit
    • test_list_users_invalid_role — Test listing users with invalid role
    • test_list_users_empty — Test listing users when none have role
    • test_list_users_exception — Test exception handling
  • TestGetRolePermissions (4 tests)
    • test_get_role_permissions_success — Test retrieving permissions for a role
    • test_get_role_permissions_invalid_role — Test getting permissions for invalid role
    • test_get_role_permissions_empty — Test getting permissions when role has none
    • test_get_role_permissions_exception — Test exception handling
  • TestRoleAuthorization (1 tests)
    • test_all_endpoints_require_admin — Test that all role endpoints require admin access
  • TestRoleManagementIntegration (2 tests)
    • test_role_update_creates_audit_entry — Test that updating role creates audit log entry
    • test_role_update_affects_user_list — Test that role updates affect user list by role

tests/routes/test_root.py

2 tests

  • TestRootRoutes (2 tests)
    • test_router_exists — Test that router is defined
    • test_module_imports — Test that module imports successfully

tests/routes/test_share.py

16 tests

  • test_create_share_link_success — Test successfully creating a share link
  • test_create_share_link_unauthenticated — Test creating share link without authentication
  • test_create_share_link_rate_limited — Test creating share link when rate limited
  • test_create_share_link_session_not_found — Test creating share link for non-existent session
  • test_create_share_link_with_expiry — Test creating share link with expiration date
  • test_get_my_share_links_success — Test getting user's share links
  • test_get_my_share_links_empty — Test getting share links when none exist
  • test_get_my_share_links_unauthenticated — Test getting share links without authentication
  • test_get_shared_chat_success — Test getting shared chat by token (public endpoint)
  • test_get_shared_chat_not_found — Test getting non-existent shared chat
  • test_get_shared_chat_expired — Test getting expired shared chat
  • test_delete_share_link_success — Test deleting a share link
  • test_delete_share_link_not_found — Test deleting non-existent share link
  • test_delete_share_link_unauthenticated — Test deleting share link without authentication
  • test_create_share_link_missing_session_id — Test creating share link without session_id
  • test_create_share_link_invalid_expiry_format — Test creating share link with invalid expiry format

tests/routes/test_share_import.py

6 tests

  • test_share_imports_successfully — Test that share module imports without errors.
  • test_share_uses_correct_security_imports — Test that share uses get_optional_api_key (not get_api_key_optional).
  • test_security_deps_exports — Test that security.deps exports the correct functions.
  • test_share_endpoints_defined — Test that share endpoints are properly defined.
  • test_share_router_prefix — Test that share router has the correct prefix.
  • test_share_router_tags — Test that share router has the correct tags.

tests/routes/test_simplismart_models_loading.py

5 tests

  • TestSimplismartModelsLoading (5 tests)
    • test_simplismart_client_returns_models_with_source_gateway — Verify SimpliSmart client returns models with source_gateway field.
    • test_catalog_endpoint_returns_simplismart_models — Verify /models endpoint returns SimpliSmart models when gateway=simplismart.
    • test_catalog_endpoint_includes_simplismart_in_all_gateway — Verify /models endpoint includes SimpliSmart models when gateway=all.
    • test_simplismart_models_have_required_fields — Verify SimpliSmart models have all required fields for frontend filtering.
    • test_v1_models_endpoint_returns_simplismart — Verify /v1/models endpoint also returns SimpliSmart models.

tests/routes/test_stream_generator.py

4 tests

  • TestStreamGeneratorAnonymous (2 tests)
    • test_anonymous_stream_does_not_call_enforce_plan_limits — Test that anonymous streaming does not call enforce_plan_limits.
    • test_anonymous_stream_completes_successfully — Test that anonymous streaming completes without errors.
  • TestStreamGeneratorAuthenticated (2 tests)
    • test_authenticated_stream_calls_enforce_plan_limits — Test that authenticated streaming calls enforce_plan_limits.
    • test_authenticated_stream_plan_limit_exceeded — Test that plan limit exceeded error is properly returned.

tests/routes/test_subscription_management_routes.py

15 tests

  • TestGetCurrentSubscriptionEndpoint (2 tests)
    • test_get_subscription_success — Test successful subscription retrieval
    • test_get_subscription_no_subscription — Test getting subscription when user has none
  • TestUpgradeSubscriptionEndpoint (3 tests)
    • test_upgrade_success — Test successful subscription upgrade
    • test_upgrade_no_subscription — Test upgrading when user has no subscription
    • test_upgrade_invalid_proration_behavior — Test upgrade with invalid proration behavior
  • TestDowngradeSubscriptionEndpoint (2 tests)
    • test_downgrade_success — Test successful subscription downgrade
    • test_downgrade_no_subscription — Test downgrading when user has no subscription
  • TestCancelSubscriptionEndpoint (4 tests)
    • test_cancel_at_period_end_success — Test canceling subscription at end of billing period
    • test_cancel_immediately_success — Test canceling subscription immediately
    • test_cancel_default_behavior — Test cancel with default behavior (cancel at period end)
    • test_cancel_no_subscription — Test canceling when user has no subscription
  • TestSubscriptionEndpointsAuthentication (4 tests)
    • test_get_subscription_no_auth — Test getting subscription without authentication
    • test_upgrade_no_auth — Test upgrading without authentication
    • test_downgrade_no_auth — Test downgrading without authentication
    • test_cancel_no_auth — Test canceling without authentication

tests/routes/test_system.py

39 tests

  • TestCacheStatus (4 tests)
    • test_get_cache_status_with_data — Test cache status with data
    • test_get_cache_status_empty_cache — Test cache status with empty cache
    • test_get_cache_status_stale_cache — Test cache status with stale cache
    • test_get_cache_status_error_handling — Test cache status error handling
  • TestCacheRefresh (5 tests)
    • test_refresh_gateway_cache_success — Test successful cache refresh
    • test_refresh_gateway_cache_skip_if_valid — Test cache refresh skipped if cache is still valid
    • test_refresh_gateway_cache_force — Test forced cache refresh
    • test_refresh_gateway_cache_invalid_gateway — Test refresh with invalid gateway
    • test_refresh_gateway_cache_deepinfra_not_supported — Test that DeepInfra doesn't support bulk refresh
  • TestCacheClear (3 tests)
    • test_clear_specific_gateway_cache — Test clearing cache for specific gateway
    • test_clear_all_caches — Test clearing all caches
    • test_clear_cache_error_handling — Test cache clear error handling
  • TestGatewayHealth (5 tests)
    • test_check_all_gateways_healthy — Test all gateways healthy
    • test_check_all_gateways_some_unhealthy — Test with some gateways unhealthy
    • test_check_all_gateways_timeout — Test gateway timeout handling
    • test_check_single_gateway_success — Test checking single gateway
    • test_check_single_gateway_not_found — Test checking non-existent gateway
  • TestModelzCacheManagement (6 tests)
    • test_get_modelz_cache_status_success — Test getting Modelz cache status
    • test_get_modelz_cache_status_error — Test Modelz cache status error handling
    • test_refresh_modelz_cache_success — Test refreshing Modelz cache
    • test_refresh_modelz_cache_error — Test Modelz cache refresh error handling
    • test_clear_modelz_cache_success — Test clearing Modelz cache
    • test_clear_modelz_cache_error — Test Modelz cache clear error handling
  • TestSystemIntegration (2 tests)
    • test_cache_workflow_status_refresh_status — Test complete cache workflow: check status, refresh, check again
    • test_health_check_with_cache_info — Test health check includes cache information
  • TestGatewayHealthDashboard (6 tests)
    • test_gateway_dashboard_endpoint_and_cache_pass — Test gateway marked healthy when both endpoint and cache pass
    • test_gateway_dashboard_endpoint_pass_cache_fail — Test gateway marked healthy when endpoint passes but cache fails (OR logic)
    • test_gateway_dashboard_endpoint_fail_cache_pass — Test gateway marked healthy when cache passes but endpoint fails (cache-only gateway)
    • test_gateway_dashboard_both_fail — Test gateway marked unhealthy when both endpoint and cache fail
    • test_gateway_dashboard_unconfigured — Test gateway marked unconfigured when no API key is configured
    • test_gateway_dashboard_html_rendering_with_or_logic — Test HTML dashboard rendering applies OR logic correctly for health status
  • TestCacheInvalidation (8 tests)
    • test_invalidate_cache_specific_gateway — Test invalidating cache for a specific gateway
    • test_invalidate_cache_gateway_case_insensitive — Test that gateway name is converted to lowercase
    • test_invalidate_cache_type_models — Test invalidating all model caches
    • test_invalidate_cache_type_providers — Test invalidating providers cache
    • test_invalidate_cache_type_pricing — Test invalidating pricing cache
    • test_invalidate_cache_all — Test invalidating all caches when no parameters provided
    • test_invalidate_cache_error_handling — Test error handling in cache invalidation
    • test_invalidate_cache_response_format — Test that response has correct format

tests/routes/test_transaction_analytics.py

19 tests

  • TestGetTransactionAnalytics (8 tests)
    • test_get_transaction_analytics_success — Test successfully fetching transaction analytics
    • test_get_transaction_analytics_different_windows — Test fetching analytics with different time windows
    • test_get_transaction_analytics_default_window — Test default window parameter
    • test_get_transaction_analytics_missing_cookie — Test that missing OPENROUTER_COOKIE returns 503
    • test_get_transaction_analytics_auth_failure — Test handling OpenRouter authentication failure
    • test_get_transaction_analytics_api_error — Test handling OpenRouter API errors
    • test_get_transaction_analytics_timeout — Test handling request timeout
    • test_get_transaction_analytics_request_error — Test handling request errors
  • TestGetTransactionSummary (8 tests)
    • test_get_transaction_summary_success — Test successfully generating transaction summary
    • test_get_transaction_summary_aggregation — Test summary data aggregation
    • test_get_transaction_summary_token_stats — Test token statistics calculation
    • test_get_transaction_summary_usage_stats — Test usage (cost) statistics calculation
    • test_get_transaction_summary_empty_data — Test summary with empty data
    • test_get_transaction_summary_with_reasoning_tokens — Test summary correctly handles reasoning tokens
    • test_get_transaction_summary_rounding — Test numerical rounding in summary
    • test_get_transaction_summary_error_propagation — Test error propagation from get_transaction_analytics
  • TestTransactionAnalyticsIntegration (3 tests)
    • test_raw_and_summary_consistency — Test raw data and summary are consistent
    • test_multiple_window_queries — Test querying multiple time windows
    • test_public_access_no_auth_required — Test that endpoints are public (no authentication required)

tests/routes/test_users.py

31 tests

  • TestUserBalance (4 tests)
    • test_get_balance_active_user — Test getting balance for active (non-trial) user
    • test_get_balance_trial_user — Test getting balance for trial user
    • test_get_balance_invalid_api_key — Test balance retrieval with invalid API key
    • test_get_balance_error_handling — Test balance error handling
  • TestUserMonitor (4 tests)
    • test_user_monitor_success — Test successful user monitoring
    • test_user_monitor_no_rate_limits — Test user monitoring when no rate limits configured
    • test_user_monitor_failed_metrics — Test monitoring when metrics retrieval fails
    • test_user_monitor_invalid_key — Test monitoring with invalid API key
  • TestRateLimits (3 tests)
    • test_get_rate_limits_configured — Test getting configured rate limits
    • test_get_rate_limits_default — Test getting default rate limits when none configured
    • test_get_rate_limits_invalid_key — Test rate limits with invalid API key
  • TestUserProfile (8 tests)
    • test_get_profile_success — Test successfully getting user profile
    • test_get_profile_with_max_tier_tiered_credits — Test profile response includes tiered credit fields for MAX tier user
    • test_get_profile_tier_display_names — Test that tier display names are correctly mapped for all tiers
    • test_get_profile_not_found — Test getting profile when profile retrieval fails
    • test_update_profile_success — Test successfully updating user profile
    • test_update_profile_no_fields — Test updating profile with no fields provided
    • test_update_profile_failed — Test profile update failure
    • test_update_profile_multiple_fields — Test updating multiple profile fields
  • TestAccountDeletion (4 tests)
    • test_delete_account_success — Test successfully deleting account
    • test_delete_account_wrong_confirmation — Test account deletion with wrong confirmation
    • test_delete_account_failed — Test account deletion failure
    • test_delete_account_invalid_key — Test account deletion with invalid API key
  • TestCreditTransactions (5 tests)
    • test_get_transactions_success — Test successfully getting credit transactions
    • test_get_transactions_with_filters — Test getting transactions with filters
    • test_get_transactions_pagination — Test transaction pagination
    • test_get_transactions_invalid_key — Test transactions with invalid API key
    • test_get_transactions_error_handling — Test transaction retrieval error handling
  • TestUserIntegration (3 tests)
    • test_complete_user_info_flow — Test getting complete user information
    • test_profile_update_workflow — Test complete profile update workflow
    • test_trial_user_journey — Test trial user accessing balance

tests/schemas/

10 files, 113 tests

tests/schemas/test_admin.py

2 tests

  • TestAdminSchemas (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/schemas/test_api_keys.py

2 tests

  • TestApiKeysSchemas (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/schemas/test_common.py

2 tests

  • TestCommonSchemas (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/schemas/test_coupons.py

2 tests

  • TestCouponsSchemas (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/schemas/test_error_schemas.py

29 tests

  • TestErrorContext (10 tests)
    • test_minimal_context — Test creating minimal context with no fields.
    • test_model_not_found_context — Test context for model not found error.
    • test_insufficient_credits_context — Test context for insufficient credits error.
    • test_rate_limit_context — Test context for rate limit error.
    • test_parameter_validation_context — Test context for parameter validation error.
    • test_provider_error_context — Test context for provider error.
    • test_internal_error_context — Test context for internal error.
    • test_api_key_context — Test context for API key error.
    • test_serialization — Test context serialization to dict.
    • test_allowed_values_list — Test that allowed_values accepts list.
  • TestErrorDetail (8 tests)
    • test_minimal_error_detail — Test creating minimal valid error detail.
    • test_full_error_detail — Test fully populated error detail.
    • test_missing_required_fields_raises_error — Test that missing required fields raises ValidationError.
    • test_suggestions_is_list — Test that suggestions must be a list.
    • test_status_is_integer — Test that status must be an integer.
    • test_invalid_status_type_raises_error — Test that invalid status type raises ValidationError.
    • test_serialization_excludes_none — Test that None values can be excluded from serialization.
    • test_context_nested_serialization — Test that nested context is properly serialized.
  • TestErrorResponse (11 tests)
    • test_minimal_error_response — Test creating minimal valid error response.
    • test_full_error_response — Test fully populated error response.
    • test_missing_error_field_raises_error — Test that missing error field raises ValidationError.
    • test_serialization_structure — Test that serialization has correct structure.
    • test_json_serialization — Test that error response can be serialized to JSON.
    • test_nested_context_in_response — Test that nested context is properly included.
    • test_example_model_not_found_response — Test example model not found error response.
    • test_example_insufficient_credits_response — Test example insufficient credits error response.
    • test_example_rate_limit_response — Test example rate limit error response.
    • test_timestamp_validation — Test that timestamp format is validated.
    • test_urls_are_strings — Test that URL fields accept strings.

tests/schemas/test_notification.py

2 tests

  • TestNotificationSchemas (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/schemas/test_payment_credit_validation.py

11 tests

  • test_credit_value_minimum_100_percent — Test that credit value must be at least 100% of payment amount.
  • test_credit_value_less_than_payment_rejected — Test that credit value less than payment amount is rejected.
  • test_credit_value_bonus_packages_allowed — Test that bonus packages (credit > payment) are allowed.
  • test_credit_value_exceeds_3x_rejected — Test that credit value exceeding 3x payment is rejected.
  • test_credit_value_exactly_3x_allowed — Test that credit value exactly 3x payment is allowed.
  • test_credit_value_none_allowed — Test that None credit_value is allowed (uses payment amount as credit).
  • test_various_payment_amounts_with_valid_credits — Test various payment amounts with valid credit values.
  • test_edge_case_just_below_minimum — Test edge case just below 100% credit value.
  • test_edge_case_just_above_maximum — Test edge case just above 3x credit value.
  • test_fractional_credit_values — Test that fractional credit values are handled correctly.
  • test_promotional_packages_realistic_scenarios — Test realistic promotional package scenarios.

tests/schemas/test_proxy.py

59 tests

  • TestMessageSchema (15 tests)
    • test_basic_message — Test basic message creation
    • test_message_with_tool_calls — Test assistant message with tool_calls
    • test_tool_response_message — Test tool response message
    • test_function_message — Test function message with name
    • test_multimodal_content — Test message with multimodal content array
    • test_valid_roles — Test all valid message roles
    • test_invalid_role — Test invalid message role raises error
    • test_user_message_requires_content — Test that user messages must have content
    • test_system_message_requires_content — Test that system messages must have content
    • test_developer_message_requires_content — Test that developer messages must have content
    • test_developer_message_basic — Test basic developer message creation (OpenAI developer role)
    • test_tool_message_requires_content — Test that tool messages must have content (the response)
    • test_function_message_requires_content — Test that function messages must have content
    • test_assistant_message_allows_null_content_with_tool_calls — Test that assistant messages can have null content when tool_calls is present
    • test_assistant_message_requires_content_or_tool_calls — Test that assistant messages need either content or tool_calls
  • TestProxyRequestOpenAIAlignment (25 tests)
    • test_minimal_request — Test minimal valid request
    • test_all_sampling_parameters — Test all sampling parameters
    • test_penalty_parameters — Test frequency and presence penalty parameters
    • test_penalty_bounds — Test penalty parameter bounds (-2.0 to 2.0)
    • test_temperature_bounds — Test temperature parameter bounds (0 to 2)
    • test_streaming_parameters — Test streaming parameters
    • test_tool_choice_string — Test tool_choice with string value
    • test_tool_choice_object — Test tool_choice with object value (specific function)
    • test_parallel_tool_calls — Test parallel_tool_calls parameter
    • test_response_format_json_object — Test response_format with json_object type
    • test_response_format_json_schema — Test response_format with json_schema type
    • test_logprobs_parameters — Test logprobs and top_logprobs parameters
    • test_top_logprobs_bounds — Test top_logprobs bounds (0 to 20)
    • test_logit_bias — Test logit_bias parameter
    • test_seed_parameter — Test seed parameter for deterministic sampling
    • test_user_parameter — Test user parameter for end-user identification
    • test_service_tier_parameter — Test service_tier parameter
    • test_stop_string — Test stop parameter with single string
    • test_stop_list — Test stop parameter with list of strings
    • test_stop_max_sequences — Test stop parameter enforces max 4 sequences
    • test_gateway_provider_parameter — Test gateway-specific provider parameter
    • test_extra_fields_allowed — Test that extra fields are allowed for forward compatibility
    • test_empty_messages_rejected — Test that empty messages list is rejected
    • test_n_minimum_value — Test n parameter minimum value (1)
    • test_complete_request — Test complete request with all parameters
  • TestStreamOptionsSchema (2 tests)
    • test_stream_options_include_usage — Test StreamOptions with include_usage
    • test_stream_options_default — Test StreamOptions with default values
  • TestProxyRequestTools (4 tests)
    • test_proxy_request_without_tools — Test ProxyRequest without tools field
    • test_proxy_request_with_tools — Test ProxyRequest with tools field
    • test_proxy_request_with_empty_tools_list — Test ProxyRequest with empty tools list
    • test_proxy_request_tools_extra_fields — Test ProxyRequest accepts tools via extra fields (backward compatibility)
  • TestResponseRequestTools (3 tests)
    • test_response_request_without_tools — Test ResponseRequest without tools field
    • test_response_request_with_tools — Test ResponseRequest with tools field
    • test_response_request_with_multiple_tools — Test ResponseRequest with multiple tools
  • TestMessageContentValidation (10 tests)
    • test_user_message_rejects_empty_string — Test that user messages reject empty string content
    • test_user_message_rejects_whitespace_only — Test that user messages reject whitespace-only content
    • test_system_message_rejects_empty_string — Test that system messages reject empty string content
    • test_developer_message_rejects_empty_string — Test that developer messages reject empty string content
    • test_developer_message_rejects_whitespace_only — Test that developer messages reject whitespace-only content
    • test_tool_message_rejects_empty_string — Test that tool messages reject empty string content
    • test_assistant_message_rejects_empty_without_tool_calls — Test that assistant messages reject empty content without tool_calls
    • test_assistant_message_allows_null_with_tool_calls — Test that assistant messages allow null content with tool_calls
    • test_user_message_allows_non_empty_string — Test that user messages accept valid non-empty content
    • test_multimodal_content_empty_list_rejected — Test that empty list content is rejected for user messages

tests/schemas/test_trials.py

2 tests

  • TestTrialsSchemas (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/schemas/test_users.py

2 tests

  • TestUsersSchemas (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/security/

6 files, 113 tests

tests/security/test_admin_auth_coverage.py

3 tests

  • test_model_sync_router_has_auth_dependency — CRITICAL: model_sync.py must have require_admin at router level.
  • test_no_admin_route_without_auth — Scan all route files for /admin paths and verify they have auth.
  • test_bandit_false_positives_suppressed — Verify nosec comments are present on known false positives.

tests/security/test_admin_security.py

1 tests

  • test_admin_endpoints — Test that all admin endpoints require proper authentication

tests/security/test_db_security.py

31 tests

  • TestCreateSecureAPIKey (6 tests)
    • test_create_secure_api_key_success — Test successful secure API key creation
    • test_create_secure_api_key_with_expiration — Test API key creation with expiration
    • test_create_secure_api_key_with_ip_allowlist — Test API key creation with IP allowlist
    • test_create_secure_api_key_duplicate_name — Test API key creation with duplicate name fails
    • test_create_secure_api_key_insert_fails — Test API key creation when insert fails
    • test_create_secure_api_key_with_custom_permissions — Test API key creation with custom scope permissions
  • TestValidateSecureAPIKey (8 tests)
    • test_validate_secure_api_key_success — Test successful API key validation
    • test_validate_secure_api_key_not_found — Test validation with non-existent key
    • test_validate_secure_api_key_inactive — Test validation with inactive key
    • test_validate_secure_api_key_expired — Test validation with expired key
    • test_validate_secure_api_key_ip_not_allowed — Test validation with disallowed IP
    • test_validate_secure_api_key_domain_not_allowed — Test validation with disallowed domain
    • test_validate_secure_api_key_request_limit_exceeded — Test validation when request limit is exceeded
    • test_validate_plain_text_key — Test validation of plain text (legacy) key
  • TestRotateAPIKey (3 tests)
    • test_rotate_api_key_success — Test successful API key rotation
    • test_rotate_api_key_not_found — Test rotation with non-existent key
    • test_rotate_api_key_with_new_name — Test API key rotation with new name
  • TestGetAuditLogs (6 tests)
    • test_get_audit_logs_all — Test getting all audit logs
    • test_get_audit_logs_by_user — Test getting audit logs filtered by user
    • test_get_audit_logs_by_action — Test getting audit logs filtered by action
    • test_get_audit_logs_with_date_range — Test getting audit logs within date range
    • test_get_audit_logs_empty — Test getting audit logs when none exist
    • test_get_audit_logs_error_handling — Test audit logs error handling
  • TestBulkRotateUserKeys (4 tests)
    • test_bulk_rotate_user_keys_success — Test successful bulk rotation
    • test_bulk_rotate_user_keys_no_keys — Test bulk rotation when user has no keys
    • test_bulk_rotate_user_keys_with_environment — Test bulk rotation filtered by environment
    • test_bulk_rotate_user_keys_partial_failure — Test bulk rotation when some keys fail
  • TestCheckKeyNameUniqueness (4 tests)
    • test_check_key_name_uniqueness_unique — Test that unique name returns True
    • test_check_key_name_uniqueness_duplicate — Test that duplicate name returns False
    • test_check_key_name_uniqueness_with_exclusion — Test uniqueness check excluding a specific key
    • test_check_key_name_uniqueness_error_handling — Test that errors return False (safe default)

tests/security/test_deps.py

29 tests

  • test_get_api_key_missing_credentials
  • test_get_api_key_missing_token
  • test_get_api_key_valid_logs_usage
  • test_get_api_key_valueerror_mapped
  • test_get_api_key_unexpected
  • test_get_api_key_no_violation_log_when_disabled — Verify that log_security_violations=False prevents security violation logging.
  • test_get_api_key_logs_violation_by_default — Verify that log_security_violations=True (default) logs security violations.
  • test_get_current_user_happy
  • test_get_current_user_404
  • test_require_admin_via_flag
  • test_require_admin_via_role
  • test_require_admin_denied_logs_violation
  • test_get_optional_api_key_no_credentials_returns_none
  • test_get_optional_api_key_invalid_returns_none
  • test_get_optional_api_key_valid_returns_key
  • test_get_optional_api_key_does_not_log_violations — Verify that get_optional_api_key does NOT log security violations.
  • test_get_optional_user_no_credentials_returns_none
  • test_get_optional_user_invalid_returns_none
  • test_get_optional_user_valid_returns_user
  • test_get_optional_user_does_not_log_violations — Verify that get_optional_user does NOT log security violations.
  • test_require_active_subscription_ok
  • test_require_active_subscription_forbidden
  • test_check_credits_ok
  • test_check_credits_402
  • test_get_user_id
  • test_verify_key_permissions_none_required_returns_key
  • test_verify_key_permissions_ok
  • test_verify_key_permissions_user_missing
  • test_verify_key_permissions_forbidden

tests/security/test_injection.py

28 tests

  • TestSQLInjectionPrevention (5 tests)
    • test_sql_injection_in_username — SQL injection attempt in username parameter
    • test_sql_injection_in_email — SQL injection attempt in email parameter
    • test_sql_injection_in_search — SQL injection attempt in search parameters
    • test_sql_injection_union_attack — SQL injection UNION attack attempt
    • test_sql_injection_in_api_key_lookup — SQL injection in API key lookup
  • TestXSSPrevention (4 tests)
    • test_xss_in_username — XSS attempt in username
    • test_xss_in_chat_message — XSS attempt in chat message
    • test_xss_in_json_response — Verify JSON responses don't contain unescaped HTML
    • test_xss_svg_attack — XSS via SVG payload
  • TestCommandInjection (3 tests)
    • test_command_injection_in_model_name — Command injection via model name
    • test_command_injection_backticks — Command injection using backticks
    • test_command_injection_pipe — Command injection using pipe operator
  • TestPathTraversal (3 tests)
    • test_path_traversal_in_filename — Path traversal attempt in filename parameter
    • test_path_traversal_encoded — Path traversal with URL encoding
    • test_path_traversal_windows_style — Path traversal with Windows path separators
  • TestLDAPInjection (2 tests)
    • test_ldap_injection_in_username — LDAP injection attempt
    • test_ldap_wildcard_injection — LDAP wildcard injection
  • TestHeaderInjection (1 tests)
    • test_crlf_injection_in_header — CRLF injection attempt in custom header
  • TestJSONInjection (2 tests)
    • test_json_injection_nested_objects — Attempt to inject malicious JSON structures
    • test_json_with_null_bytes — JSON with null bytes
  • TestInputValidation (3 tests)
    • test_oversized_input — Reject or handle oversized inputs
    • test_special_unicode_characters — Handle special Unicode characters safely
    • test_null_character_injection — Null character injection attempt
  • TestAPIKeyValidation (2 tests)
    • test_api_key_format_validation — Validate API key format
    • test_api_key_length_validation — API key length should be validated
  • TestErrorMessagesSecurity (3 tests)
    • test_database_errors_not_exposed — Database errors should not expose internal details
    • test_stack_traces_not_exposed — Stack traces should not be exposed to clients
    • test_file_paths_not_exposed — File paths should not be exposed in errors

tests/security/test_security.py

21 tests

  • TestAPIKeyHashing (2 tests)
    • test_hash_api_key_requires_salt — Hashing should require API_GATEWAY_SALT
    • test_hash_api_key_rejects_short_salt — Hashing should reject salt shorter than 16 characters
  • TestAPIKeyGeneration (7 tests)
    • test_generate_api_key_success — Successfully generate an API key
    • test_generate_api_key_unique — Generated API keys should be unique
    • test_generate_api_key_test_environment — Generate test environment API key
    • test_generate_api_key_staging_environment — Generate staging environment API key
    • test_generate_api_key_development_environment — Generate development environment API key
    • test_generate_api_key_custom_length — Generate API key with custom length
    • test_generate_api_key_format — API key should follow expected format
  • TestIPAllowlist (1 tests)
    • test_validate_ip_mixed_formats — Validate IP with mixed exact and CIDR formats
  • TestSecurityConstants (2 tests)
    • test_api_key_prefix — API keys should use gw_ prefix
    • test_salt_minimum_length — Salt should meet minimum length requirements
  • TestSecurityEdgeCases (3 tests)
    • test_hash_empty_string — Hash empty string should work or raise appropriate error
    • test_validate_ip_invalid_ip — Validate with invalid IP format
    • test_validate_domain_invalid_url — Validate with invalid URL format
  • TestAuditLoggerFormat (6 tests)
    • test_audit_logger_exists — AuditLogger should be available via get_audit_logger()
    • test_audit_logger_has_handlers — AuditLogger should have configured handlers
    • test_audit_logger_format_has_level_before_audit — Log format should have levelname before [AUDIT] for Railway compatibility
    • test_audit_logger_log_methods_exist — AuditLogger should have all required logging methods
    • test_audit_logger_uses_info_level_for_normal_events — Normal events should use INFO level
    • test_audit_logger_uses_warning_level_for_security_violations — Security violations should use WARNING level

tests/services/

113 files, 2213 tests

tests/services/test_aihubmix_client.py

9 tests

  • TestGetAihubmixClient (3 tests)
    • test_get_client_success — Test successful client initialization
    • test_get_client_missing_api_key — Test error when API key is missing
    • test_get_client_missing_app_code — Test error when APP-Code is missing
  • TestMakeAihubmixRequest (3 tests)
    • test_make_request_success — Test successful non-streaming request
    • test_make_streaming_request_success — Test successful streaming request
    • test_make_request_error — Test request error handling
  • TestProcessAihubmixResponse (3 tests)
    • test_process_response_success — Test successful response processing
    • test_process_response_no_usage — Test processing response without usage data
    • test_process_response_error — Test error handling in response processing

tests/services/test_aihubmix_normalization.py

11 tests

  • TestAiHubMixNormalizationWithPricing (11 tests)
    • test_normalize_model_with_pricing — Test normalizing a model with pricing from AiHubMix API
    • test_normalize_model_filters_zero_pricing — Test that models with zero pricing are filtered out
    • test_normalize_model_filters_empty_pricing — Test that models with empty pricing are filtered out
    • test_normalize_model_missing_id — Test that models without ID are filtered out
    • test_normalize_model_with_small_pricing — Test normalizing a model with small pricing values
    • test_normalize_model_context_length — Test that context length is properly extracted
    • test_normalize_model_default_context_length — Test that default context length is used when not provided
    • test_normalize_model_with_model_id_field — Test normalizing a model with 'model_id' instead of 'id' field
    • test_normalize_model_with_desc_field — Test normalizing a model with 'desc' instead of 'description' field
    • test_missing_id_logs_debug_not_warning — Test that models missing both 'id' and 'model_id' log at DEBUG level, not WARNING
    • test_model_id_field_works_without_warnings — Test that models with 'model_id' field are processed without warnings

tests/services/test_aimo_resilience.py

20 tests

  • TestAIMOTimeoutConfiguration (5 tests)
    • test_aimo_fetch_timeout_configured — Verify AIMO fetch timeout is configurable and reasonable.
    • test_aimo_connect_timeout_configured — Verify AIMO connect timeout is configured.
    • test_aimo_max_retries_configured — Verify AIMO max retries is configured.
    • test_aimo_http_fallback_enabled_by_default — Verify HTTP fallback is enabled by default.
    • test_aimo_base_urls_configured — Verify multiple AIMO base URLs are available.
  • TestAIMOStaleWhileRevalidate (6 tests)
    • test_fresh_cached_models_returns_fresh_data — Fresh cache within TTL should be returned.
    • test_fresh_cached_models_returns_none_when_expired — Fresh cache beyond TTL should return None.
    • test_stale_while_revalidate_returns_fresh_data — Should return data within fresh TTL.
    • test_stale_while_revalidate_returns_stale_data — Should return stale data within stale_ttl window.
    • test_stale_while_revalidate_returns_none_when_both_expired — Should return None when both fresh and stale caches expired.
    • test_stale_while_revalidate_returns_none_with_no_cache — Should return None when no cached data exists.
  • TestAIMORetryLogic (5 tests)
    • test_successful_fetch_on_first_try — Should return data on successful first attempt.
    • test_retry_on_timeout — Should retry on timeout exception.
    • test_http_fallback_when_https_fails — Should try HTTP fallback when HTTPS fails.
    • test_uses_stale_cache_when_all_retries_fail — Should return stale cache when all retries fail.
    • test_returns_empty_list_when_no_cache_and_all_fail — Should return empty list when no cache and all retries fail.
  • TestRedirectHandling (2 tests)
    • test_follows_http_308_redirect — Should follow HTTP 308 redirects by using follow_redirects=True.
    • test_redirect_preserves_headers — Should preserve authorization headers during redirect.
  • TestTimeoutConfiguration (2 tests)
    • test_timeout_uses_httpx_timeout_object — Verify implementation uses httpx.Timeout with separate connect timeout.
    • test_fetch_timeout_is_short — Verify fetch timeout is significantly shorter than original 20s.

tests/services/test_akash_client.py

7 tests

  • TestAkashClient (7 tests)
    • test_get_akash_client — Test getting Akash client
    • test_get_akash_client_no_key — Test getting Akash client without API key
    • test_make_akash_request_openai — Test making request to Akash
    • test_make_akash_request_openai_stream — Test making streaming request to Akash
    • test_process_akash_response — Test processing Akash response
    • test_process_akash_response_no_usage — Test processing Akash response without usage data
    • test_make_akash_request_with_kwargs — Test making request to Akash with additional parameters

tests/services/test_alibaba_cloud_client.py

14 tests

  • test_list_alibaba_models_retries_with_china_region — Ensure we fall back to the China endpoint when the default fails with auth errors.
  • test_chat_request_prefers_cached_region — Subsequent requests should re-use the inferred working region to avoid extra failures.
  • test_get_client_uses_region_specific_keys — Ensure region-specific API keys override the default key.
  • test_region_attempt_order_uses_available_key_for_all_regions — When only one region-specific key is set, all regions should still be attempted.
  • test_failover_with_misconfigured_region_key — Test that failover works when user sets international key but it only works for China.
  • test_is_quota_error_detects_quota_exceeded — Test that _is_quota_error detects various quota error messages.
  • test_is_quota_error_does_not_match_auth_errors — Test that _is_quota_error doesn't match auth errors.
  • test_quota_error_raises_quota_exceeded_error — Test that quota errors are converted to QuotaExceededError.
  • test_quota_error_does_not_retry_other_regions — Test that quota errors don't trigger region failover.
  • TestAlibabaQuotaErrorCaching (5 tests)
    • test_quota_error_caches_failure_state — Test that QuotaExceededError triggers caching of failure state.
    • test_quota_error_backoff_skips_api_calls — Test that subsequent calls during backoff period skip API calls.
    • test_quota_error_backoff_expires — Test that backoff expires after the configured duration.
    • test_successful_fetch_clears_quota_error — Test that a successful fetch clears the quota error state.
    • test_quota_error_backoff_not_overridden_by_cache_ttl — Test that quota error backoff (15 min) is not overridden by cache TTL (1 hour).

tests/services/test_analytics.py

4 tests

  • TestTrialAnalytics (4 tests)
    • test_get_trial_analytics_returns_dict — Test that get_trial_analytics returns a dictionary
    • test_get_trial_analytics_has_required_keys — Test that result has all required keys
    • test_get_trial_analytics_default_values — Test that default values are zero (TODO implementation)
    • test_get_trial_analytics_value_types — Test that values have correct types

tests/services/test_anonymous_rate_limiter.py

23 tests

  • TestModelWhitelist (6 tests)
    • test_free_model_allowed — Test that whitelisted free models are allowed
    • test_non_free_model_rejected — Test that non-free models are rejected
    • test_free_suffix_but_not_whitelisted — Test that models with :free suffix but not whitelisted are rejected
    • test_model_case_sensitivity — Test model matching is case-insensitive
    • test_empty_and_none_models — Test handling of empty and None models
    • test_model_without_free_suffix — Test that models without :free suffix are rejected even if similar
  • TestAnonymousRateLimiting (4 tests)
    • test_rate_limit_allows_first_request — Test that first request from IP is allowed
    • test_rate_limit_blocks_after_limit — Test that requests are blocked after daily limit
    • test_rate_limit_decrements_remaining — Test that remaining count decreases properly
    • test_different_ips_tracked_separately — Test that different IPs have separate rate limits
  • TestValidateAnonymousRequest (3 tests)
    • test_valid_request_allowed — Test that valid request with free model and under limit is allowed
    • test_non_free_model_rejected — Test that non-free model request is rejected
    • test_rate_limit_exceeded_rejected — Test that request over rate limit is rejected
  • TestRecordAnonymousRequest (2 tests)
    • test_record_increments_count — Test that recording a request increments the counter
    • test_record_returns_remaining — Test that recording returns correct remaining count
  • TestIPHashing (3 tests)
    • test_ip_hashing_consistency — Test that same IP produces same hash
    • test_ip_hashing_different_ips — Test that different IPs produce different hashes
    • test_ip_hash_length — Test that hash is truncated to expected length
  • TestAnonymousStats (2 tests)
    • test_stats_empty — Test stats when no usage
    • test_stats_with_usage — Test stats after some usage
  • TestRedisIntegration (3 tests)
    • test_redis_increment — Test Redis INCR operation
    • test_redis_get — Test Redis GET operation
    • test_redis_fallback_on_error — Test fallback to memory when Redis errors

tests/services/test_anthropic_client.py

14 tests

  • TestAnthropicClient (11 tests)
    • test_get_anthropic_client — Test getting Anthropic client
    • test_get_anthropic_client_no_key — Test getting Anthropic client without API key
    • test_make_anthropic_request — Test making request to Anthropic
    • test_make_anthropic_request_with_kwargs — Test making request to Anthropic with additional parameters
    • test_make_anthropic_request_error — Test handling errors from Anthropic
    • test_make_anthropic_request_stream — Test making streaming request to Anthropic
    • test_make_anthropic_request_stream_with_kwargs — Test making streaming request to Anthropic with additional parameters
    • test_make_anthropic_request_stream_error — Test handling streaming errors from Anthropic
    • test_process_anthropic_response — Test processing Anthropic response
    • test_process_anthropic_response_no_usage — Test processing Anthropic response without usage data
    • test_process_anthropic_response_multiple_choices — Test processing Anthropic response with multiple choices
  • TestAnthropicModels (3 tests)
    • test_claude_3_5_sonnet_request — Test making request with Claude 3.5 Sonnet
    • test_claude_3_opus_request — Test making request with Claude 3 Opus
    • test_claude_3_haiku_request — Test making request with Claude 3 Haiku

tests/services/test_anthropic_models.py

11 tests

  • TestFetchModelsFromAnthropic (5 tests)
    • test_fetch_models_no_api_key — Test fetching models without API key returns None
    • test_fetch_models_success — Test successful model fetching from API
    • test_fetch_models_pagination — Test model fetching with pagination
    • test_fetch_models_filters_non_claude — Test that non-Claude models are filtered out
    • test_fetch_models_http_error — Test handling of HTTP errors
  • TestNormalizeAnthropicModel (6 tests)
    • test_normalize_basic_model — Test normalizing a basic model from API response
    • test_normalize_model_with_vision — Test that Claude 3+ models have vision support
    • test_normalize_model_max_output_3_5 — Test that Claude 3.5 models have 8192 max output
    • test_normalize_model_max_output_3_0 — Test that Claude 3.0 models have 4096 max output
    • test_normalize_model_missing_id — Test that models without ID return None
    • test_normalize_model_fallback_name — Test fallback to model ID when display_name is missing

tests/services/test_auth_rate_limiting.py

23 tests

  • TestAuthRateLimitConfig (2 tests)
    • test_default_config_values — Test default configuration values
    • test_custom_config — Test custom configuration values
  • TestAuthRateLimiter (13 tests)
    • test_login_rate_limit_allows_under_limit — Test that requests under limit are allowed
    • test_login_rate_limit_blocks_over_limit — Test that requests over limit are blocked
    • test_register_rate_limit — Test registration rate limiting
    • test_password_reset_rate_limit — Test password reset rate limiting
    • test_api_key_create_rate_limit — Test API key creation rate limiting
    • test_different_ips_have_separate_limits — Test that different IPs are rate limited independently
    • test_different_limit_types_are_independent — Test that different rate limit types are tracked independently
    • test_get_remaining — Test getting remaining attempts
    • test_reset_clears_limit — Test that reset clears the rate limit
    • test_mask_identifier_ipv4 — Test IP address masking for IPv4
    • test_mask_identifier_short_string — Test identifier masking for short strings
    • test_mask_identifier_long_string — Test identifier masking for long strings
    • test_mask_identifier_empty — Test identifier masking for empty/None values
  • TestGetClientIP (6 tests)
    • test_get_ip_from_x_real_ip — Test extracting IP from X-Real-IP header (highest priority)
    • test_get_ip_from_x_forwarded_for_rightmost — Test extracting rightmost IP from X-Forwarded-For header (proxy-added)
    • test_get_ip_from_x_forwarded_for_single — Test extracting IP from X-Forwarded-For header with single IP
    • test_get_ip_from_client_host — Test extracting IP from client.host when no X-Forwarded-For
    • test_get_ip_unknown_when_no_client — Test returning 'unknown' when no client info available
    • test_spoofed_x_forwarded_for_uses_proxy_added_ip — Test that spoofed X-Forwarded-For headers don't bypass rate limiting
  • TestGlobalRateLimiter (2 tests)
    • test_get_auth_rate_limiter_returns_singleton — Test that get_auth_rate_limiter returns the same instance
    • test_check_auth_rate_limit_convenience_function — Test the convenience function for checking rate limits

tests/services/test_backend_error_fixes.py

26 tests

  • TestModelHealthMonitorFixes (3 tests)
    • test_latency_parsing_with_invalid_format — Test that invalid latency formats are handled gracefully without bare except.
    • test_latency_parsing_with_none_value — Test that None latency values are handled without crashing.
    • test_latency_parsing_with_valid_value — Test that valid latency values are parsed correctly.
  • TestAuthorizationHeaderParsingFixes (5 tests)
    • test_malformed_authorization_header_without_token — Test that authorization header with just 'Bearer' doesn't crash.
    • test_malformed_authorization_header_empty_token — Test that authorization header with empty token is handled.
    • test_valid_authorization_header — Test that valid authorization header is parsed correctly.
    • test_authorization_header_with_extra_spaces — Test that authorization header with extra spaces is handled.
    • test_unified_responses_auth_header_parsing — Test that unified_responses endpoint (chat.py:2780) handles malformed headers.
  • TestXForwardedForParsingFixes (7 tests)
    • test_empty_forwarded_for_header — Test that empty X-Forwarded-For header doesn't crash.
    • test_single_ip_forwarded_for — Test that single IP in X-Forwarded-For is parsed correctly.
    • test_multiple_ips_forwarded_for — Test that first IP from multiple IPs is extracted.
    • test_forwarded_for_with_spaces — Test that X-Forwarded-For with extra spaces is handled.
    • test_malformed_forwarded_for_ipv6 — Test that IPv6 addresses in X-Forwarded-For are handled.
    • test_middleware_forwarded_for_parsing — Test that middleware X-Forwarded-For parsing is defensive.
    • test_middleware_empty_forwarded_for — Test middleware handles empty X-Forwarded-For gracefully.
  • TestUnsafeListAccessFixes (7 tests)
    • test_empty_choices_list_finish_reason — Test that empty choices list doesn't crash when accessing finish_reason.
    • test_none_choices_finish_reason — Test that None choices value doesn't crash.
    • test_valid_choices_finish_reason — Test that valid choices list works correctly.
    • test_empty_choices_list_assistant_content — Test that empty choices list doesn't crash when extracting assistant content.
    • test_none_choices_assistant_content — Test that None choices doesn't crash.
    • test_valid_choices_assistant_content — Test that valid choices extracts content correctly.
    • test_choices_with_empty_message — Test that choices with empty message dict doesn't crash.
  • TestSilentExceptionSwallowingFixes (2 tests)
    • test_url_parsing_exception_is_logged — Test that URL parsing exceptions are now logged instead of silently ignored.
    • test_valid_url_parsing — Test that valid URLs are parsed correctly.
  • TestEndToEndErrorHandling (2 tests)
    • test_messages_endpoint_resilience — Test that messages endpoint handles edge cases without crashing.
    • test_chat_endpoint_resilience — Test that chat endpoint handles edge cases without crashing.

tests/services/test_background_tasks.py

11 tests

  • TestBackgroundTasks (11 tests)
    • test_log_activity_async_success — log_activity_async should execute database logging
    • test_log_activity_async_error_handling — log_activity_async should handle errors gracefully
    • test_log_activity_background_no_event_loop — log_activity_background should work without event loop
    • test_log_activity_background_with_event_loop — log_activity_background should create task when event loop exists
    • test_log_activity_background_error_handling — log_activity_background should handle errors gracefully
    • test_get_pending_tasks_count — get_pending_tasks_count should return count of pending tasks
    • test_log_activity_background_with_all_parameters — log_activity_background should handle all parameters
    • test_log_activity_async_with_all_parameters — log_activity_async should handle all parameters
    • test_log_activity_background_with_minimal_parameters — log_activity_background should work with minimal parameters
    • test_log_activity_async_with_none_metadata — log_activity_async should handle None metadata
    • test_log_activity_background_multiple_calls — log_activity_background should handle multiple calls

tests/services/test_braintrust_service.py

17 tests

  • TestBraintrustService (11 tests)
    • test_initialize_without_api_key — Test initialization fails gracefully without API key.
    • test_initialize_with_invalid_api_key — Test initialization warns with invalid API key format.
    • test_is_available_before_init — Test is_available returns False before initialization.
    • test_create_span_returns_noop_when_unavailable — Test create_span returns NoopSpan when Braintrust is unavailable.
    • test_noop_span_log_method — Test NoopSpan.log() doesn't raise exceptions.
    • test_noop_span_end_method — Test NoopSpan.end() doesn't raise exceptions.
    • test_noop_span_context_manager — Test NoopSpan works as a context manager.
    • test_noop_span_set_attributes — Test NoopSpan.set_attributes() doesn't raise exceptions.
    • test_flush_when_not_initialized — Test flush() doesn't raise when not initialized.
    • test_get_logger_when_not_initialized — Test get_logger() returns None when not initialized.
    • test_check_braintrust_available_alias — Test check_braintrust_available is alias for is_available.
  • TestBraintrustServiceWithMocking (4 tests)
    • test_initialize_calls_init_logger_with_correct_params — Test that initialize calls init_logger with correct parameters.
    • test_create_span_uses_logger_start_span — Test that create_span uses logger.start_span() for proper project association.
    • test_flush_calls_logger_flush — Test that flush() calls logger.flush().
    • test_create_span_returns_noop_on_exception — Test that create_span returns NoopSpan if an exception occurs.
  • TestBraintrustRealIntegration (2 tests)
    • test_real_initialization — Test real Braintrust initialization.
    • test_real_span_creation_and_logging — Test real span creation and logging.

tests/services/test_bug_fix_generator.py

32 tests

  • TestBugFixGeneratorInitialization (5 tests)
    • test_init_without_api_key — Test that initialization fails without ANTHROPIC_API_KEY.
    • test_init_with_api_key — Test successful initialization with API key.
    • test_init_with_invalid_key_format — Test warning when API key doesn't start with sk-ant-.
    • test_init_with_github_token — Test initialization with GitHub token.
    • test_init_with_custom_model — Test initialization with custom ANTHROPIC_MODEL.
  • TestAPIKeyValidation (4 tests)
    • test_validate_api_key_success — Test successful API key validation.
    • test_validate_api_key_401_unauthorized — Test validation fails with 401 Unauthorized.
    • test_validate_api_key_400_bad_request — Test validation fails with 400 Bad Request.
    • test_validate_api_key_timeout — Test validation handles timeout gracefully.
  • TestPromptSanitization (7 tests)
    • test_sanitize_text_short_text — Test sanitizing normal-length text.
    • test_sanitize_text_long_text — Test sanitizing text exceeding max length.
    • test_sanitize_text_with_null_bytes — Test removing null bytes from text.
    • test_sanitize_text_empty — Test sanitizing empty text.
    • test_sanitize_text_none — Test sanitizing None.
    • test_prepare_prompt_normal_length — Test preparing normal-length prompt.
    • test_prepare_prompt_exceeds_max — Test preparing prompt that exceeds max length.
  • TestErrorAnalysis (4 tests)
    • test_analyze_error_success — Test successful error analysis.
    • test_analyze_error_no_content — Test error analysis with no content in response.
    • test_analyze_error_400_bad_request — Test error analysis with 400 error.
    • test_analyze_error_sanitizes_long_messages — Test that error analysis sanitizes long error messages.
  • TestFixGeneration (4 tests)
    • test_generate_fix_success — Test successful fix generation.
    • test_generate_fix_analysis_fails — Test fix generation skips when analysis fails.
    • test_generate_fix_invalid_json — Test fix generation handles invalid JSON response.
    • test_generate_fix_no_changes — Test fix generation with empty changes array.
  • TestRetryLogic (3 tests)
    • test_retry_on_timeout — Test that requests are retried on timeout.
    • test_retry_max_attempts — Test that retry stops after max attempts.
    • test_no_retry_on_400 — Test that 400 errors are not retried.
  • TestRequestLogging (2 tests)
    • test_request_logging_success — Test that successful requests are logged.
    • test_request_logging_error — Test that errors are logged with details.
  • TestBugFixDataClass (1 tests)
    • test_bug_fix_to_dict — Test BugFix.to_dict() serialization.
  • TestEdgeCases (2 tests)
    • test_close_generator — Test closing the generator properly.
    • test_multiple_fixes_tracking — Test that generated fixes are tracked.

tests/services/test_butter_client.py

22 tests

  • TestButterCompatibleProviders (4 tests)
    • test_compatible_providers_return_true — Test that known compatible providers are identified correctly.
    • test_excluded_providers_return_false — Test that excluded providers are identified correctly.
    • test_unknown_provider_returns_false — Test that unknown providers default to not compatible.
    • test_case_insensitive_provider_matching — Test that provider matching is case-insensitive.
  • TestShouldUseButterCache (5 tests)
    • test_system_disabled_returns_false — Test that disabled system returns False.
    • test_anonymous_user_returns_false — Test that anonymous users don't use caching.
    • test_user_preference_disabled_returns_false — Test that users with caching disabled return False.
    • test_incompatible_provider_returns_false — Test that incompatible providers return False.
    • test_all_conditions_met_returns_true — Test that all conditions met returns True.
  • TestCacheHitDetection (3 tests)
    • test_fast_response_is_cache_hit — Test that fast responses are detected as cache hits.
    • test_slow_response_is_cache_miss — Test that slow responses are detected as cache misses.
    • test_custom_threshold — Test that custom threshold works.
  • TestButterCacheTimer (4 tests)
    • test_timer_measures_elapsed_time — Test that timer correctly measures elapsed time.
    • test_timer_detects_cache_hit — Test that timer correctly detects cache hit based on latency.
    • test_timer_detects_cache_miss — Test that timer correctly detects cache miss based on latency.
    • test_timer_custom_threshold — Test that timer respects custom threshold.
  • TestButterRequestMetadata (2 tests)
    • test_cache_hit_metadata — Test metadata generation for cache hit.
    • test_cache_miss_metadata — Test metadata generation for cache miss.
  • TestGetUserCachePreference (4 tests)
    • test_returns_true_when_enabled — Test that True is returned when caching is enabled.
    • test_returns_false_when_disabled — Test that False is returned when caching is disabled.
    • test_returns_true_for_missing_preference — Test that True is returned when preference is not set (enabled by default).
    • test_returns_false_for_none_user — Test that False is returned for None user.

tests/services/test_canopywave_client.py

17 tests

  • TestCanopywaveClient (17 tests)
    • test_get_canopywave_client_raises_without_api_key — Test that get_canopywave_client raises ValueError when API key is not configured
    • test_get_canopywave_client_returns_pooled_client — Test that get_canopywave_client returns a pooled client when API key is configured
    • test_fetch_models_from_canopywave_success — Test successful model fetching from Canopy Wave API
    • test_fetch_models_from_canopywave_handles_embedding_models — Test that embedding models are properly handled
    • test_fetch_models_from_canopywave_http_error — Test that HTTP errors are handled gracefully
    • test_fetch_models_from_canopywave_network_error — Test that network errors are handled gracefully
    • test_fetch_models_from_canopywave_skips_models_without_id — Test that models without an ID are skipped
    • test_fetch_models_from_canopywave_handles_missing_pricing — Test that missing pricing information is handled with defaults
    • test_fetch_models_from_canopywave_handles_context_in_capabilities — Test that context_length in capabilities is properly extracted
    • test_make_canopywave_request_openai — Test making a request to Canopy Wave using OpenAI client
    • test_make_canopywave_request_openai_stream — Test making a streaming request to Canopy Wave
    • test_process_canopywave_response — Test processing a Canopy Wave response
    • test_is_canopywave_model_returns_true_for_valid_model — Test that is_canopywave_model returns True for a valid model
    • test_is_canopywave_model_returns_false_for_invalid_model — Test that is_canopywave_model returns False for an invalid model
    • test_is_canopywave_model_handles_errors — Test that is_canopywave_model handles errors gracefully
    • test_fetch_models_handles_function_calling_capability — Test that function_calling capability is mapped to tools feature
    • test_fetch_models_handles_web_search_capability — Test that web_search capability is properly extracted

tests/services/test_cerebras_client.py

14 tests

  • TestCerebrasClient (14 tests)
    • test_module_imports — Test that module imports successfully
    • test_default_models_defined — Test that default models are defined
    • test_default_models_have_required_fields — Test that default models have required fields
    • test_default_supported_parameters — Test default supported parameters
    • test_get_cerebras_client_no_api_key — Test client raises ValueError without API key
    • test_normalize_cerebras_model — Test model normalization
    • test_normalize_cerebras_model_missing_id — Test normalization uses name as fallback ID when id is missing
    • test_cleanup_model_id — Test model ID cleanup
    • test_normalize_pricing — Test pricing normalization
    • test_extract_supported_parameters — Test parameter extraction
    • test_coerce_sequence — Test sequence coercion
    • test_fallback_cerebras_models — Test fallback to static models
    • test_fetch_models_from_cerebras_normalizes_sdk_response — Ensure fetch_models_from_cerebras returns normalized entries when SDK responds.
    • test_fetch_models_from_cerebras_uses_fallback_on_error — Ensure fallback catalog is used when client initialization fails.

tests/services/test_circuit_breaker_improvements.py

12 tests

  • TestCircuitBreakerHalfOpenBehavior (2 tests)
    • test_half_open_allows_multiple_failures_before_reopening — Test that HALF_OPEN state allows 2 failures before reopening
    • test_half_open_closes_after_success — Test that HALF_OPEN transitions to CLOSED after success_threshold successes
  • TestConsecutiveOpensTracking (2 tests)
    • test_consecutive_opens_increments_on_failed_recovery — Test that consecutive_opens increments when recovery fails (circuit reopens from HALF_OPEN)
    • test_consecutive_opens_resets_on_successful_recovery — Test that consecutive_opens resets to 0 after successful recovery
  • TestCircuitBreakerErrorHandling (2 tests)
    • test_circuit_breaker_error_raised_when_open — Test that CircuitBreakerError is raised when circuit is OPEN
    • test_circuit_breaker_error_contains_retry_info — Test that CircuitBreakerError message contains retry information
  • TestDetailedErrorFactory (2 tests)
    • test_provider_unavailable_creates_503_error — Test that provider_unavailable returns 503 status
    • test_provider_unavailable_contains_helpful_info — Test that error contains helpful information for users
  • TestOpenRouterCircuitConfig (1 tests)
    • test_openrouter_config_has_optimized_values — Test that OPENROUTER_CIRCUIT_CONFIG has the optimized parameters
  • TestRedisStatePersistence (2 tests)
    • test_consecutive_opens_persisted_to_redis — Test that consecutive_opens is saved to Redis
    • test_consecutive_opens_loaded_from_redis — Test that consecutive_opens is loaded from Redis
  • TestCircuitBreakerIntegration (1 tests)
    • test_chat_handler_catches_circuit_breaker_error — Test that chat handler catches CircuitBreakerError and returns 503

tests/services/test_cloudflare_workers_ai_client.py

26 tests

  • TestCloudflareWorkersAIClient (15 tests)
    • test_get_cloudflare_workers_ai_client — Test getting Cloudflare Workers AI client
    • test_get_cloudflare_workers_ai_client_no_token — Test getting Cloudflare Workers AI client without API token
    • test_get_cloudflare_workers_ai_client_no_account_id — Test getting Cloudflare Workers AI client without Account ID
    • test_make_cloudflare_workers_ai_request_openai — Test making request to Cloudflare Workers AI
    • test_make_cloudflare_workers_ai_request_openai_with_kwargs — Test making request to Cloudflare Workers AI with additional parameters
    • test_make_cloudflare_workers_ai_request_openai_error — Test handling errors from Cloudflare Workers AI
    • test_make_cloudflare_workers_ai_request_openai_stream — Test making streaming request to Cloudflare Workers AI
    • test_make_cloudflare_workers_ai_request_openai_stream_with_kwargs — Test making streaming request to Cloudflare Workers AI with additional parameters
    • test_make_cloudflare_workers_ai_request_openai_stream_error — Test handling streaming errors from Cloudflare Workers AI
    • test_process_cloudflare_workers_ai_response — Test processing Cloudflare Workers AI response
    • test_process_cloudflare_workers_ai_response_no_usage — Test processing Cloudflare Workers AI response without usage data
    • test_process_cloudflare_workers_ai_response_multiple_choices — Test processing Cloudflare Workers AI response with multiple choices
    • test_fetch_models_from_cloudflare_workers_ai — Test fetching models from Cloudflare Workers AI
    • test_default_cloudflare_workers_ai_models — Test the default Cloudflare Workers AI models catalog
    • test_default_cloudflare_workers_ai_models_have_source_gateway — Test that all default Cloudflare Workers AI models have source_gateway field
  • TestFetchModelsFromCloudflareAPI (11 tests)
    • test_fetch_models_with_properties_as_list — Test parsing models when properties is a list of property objects
    • test_fetch_models_with_properties_as_dict — Test parsing models when properties is a dict (legacy format)
    • test_fetch_models_with_missing_properties — Test parsing models when properties field is missing
    • test_fetch_models_with_task_as_string — Test parsing models when task is a string instead of a dict
    • test_fetch_models_with_invalid_context_length_value — Test parsing models when max_total_tokens has an invalid value
    • test_fetch_models_without_api_token — Test that API returns empty list when token is missing
    • test_fetch_models_without_account_id — Test that API returns empty list when account ID is missing
    • test_fetch_models_with_null_task — Test parsing models when task is null
    • test_fetch_models_with_dict_properties_string_value — Test parsing models when properties dict has string value for max_total_tokens
    • test_fetch_models_with_dict_properties_invalid_value — Test parsing models when properties dict has invalid value for max_total_tokens
    • test_fetch_models_with_non_dict_items_in_result — Test parsing models when result array contains non-dict items (lists, strings, etc.)

tests/services/test_code_classifier.py

26 tests

  • TestCodeTaskClassifier (18 tests)
    • test_classify_simple_code — Test classification of simple code tasks.
    • test_classify_code_explanation — Test classification of code explanation tasks.
    • test_classify_code_generation — Test classification of code generation tasks.
    • test_classify_debugging — Test classification of debugging tasks.
    • test_classify_refactoring — Test classification of refactoring tasks.
    • test_classify_architecture — Test classification of architecture tasks.
    • test_classify_agentic — Test classification of agentic coding tasks.
    • test_high_confidence_classification — Test that clear prompts get high confidence.
    • test_low_confidence_classification — Test that ambiguous prompts get lower confidence.
    • test_default_category_for_ambiguous — Test that ambiguous prompts default to code_generation.
    • test_classification_time_under_threshold — Test that classification completes within target time.
    • test_context_multi_file — Test that multi-file context boosts architecture/agentic scores.
    • test_context_error_trace — Test that error trace context boosts debugging score.
    • test_context_long_conversation — Test that long conversation context boosts refactoring/architecture.
    • test_is_code_related_positive — Test that code-related prompts are detected.
    • test_is_code_related_negative — Test that non-code prompts are not detected as code-related.
    • test_extract_context_from_messages — Test context extraction from conversation messages.
    • test_extract_context_multi_part_messages — Test context extraction with multi-part messages.
  • TestModuleFunctions (3 tests)
    • test_get_classifier_singleton — Test that get_classifier returns the same instance.
    • test_classify_code_task_function — Test the convenience classify_code_task function.
    • test_is_code_related_function — Test the convenience is_code_related function.
  • TestEdgeCases (5 tests)
    • test_empty_prompt — Test classification of empty prompt.
    • test_very_long_prompt — Test classification of very long prompt.
    • test_unicode_prompt — Test classification with unicode characters.
    • test_none_context — Test classification with None context.
    • test_empty_context — Test classification with empty context.

tests/services/test_code_router.py

39 tests

  • TestParseRouterModelString (8 tests)
    • test_parse_router_code_auto — Test parsing 'router:code' as auto mode.
    • test_parse_router_code_price — Test parsing 'router:code:price' mode.
    • test_parse_router_code_quality — Test parsing 'router:code:quality' mode.
    • test_parse_router_code_agentic — Test parsing 'router:code:agentic' mode.
    • test_parse_non_router_model — Test parsing regular model strings.
    • test_parse_router_auto — Test parsing general 'router:' prefix (not code router).
    • test_parse_case_insensitive — Test that parsing is case-insensitive.
    • test_parse_unknown_mode — Test parsing unknown mode falls back to auto.
  • TestCodeRouter (16 tests)
    • test_route_simple_task — Test routing a simple code task.
    • test_route_debugging_task — Test routing a debugging task.
    • test_route_architecture_task — Test routing an architecture task.
    • test_route_agentic_task — Test routing an agentic task.
    • test_route_price_mode — Test routing in price optimization mode.
    • test_route_quality_mode — Test routing in quality optimization mode.
    • test_route_agentic_mode — Test routing in agentic mode.
    • test_route_auto_mode — Test routing in auto mode (default).
    • test_quality_gate_debugging — Test that debugging tasks respect quality gate.
    • test_quality_gate_architecture — Test that architecture tasks respect quality gate.
    • test_quality_gate_agentic_task — Test that agentic tasks respect quality gate.
    • test_routing_result_structure — Test that routing result has expected structure.
    • test_routing_latency — Test that routing completes within target time.
    • test_savings_estimate_structure — Test savings estimate structure.
    • test_route_with_context — Test routing with context.
    • test_route_with_user_default_model — Test routing with user's default model for comparison.
  • TestModuleFunctions (6 tests)
    • test_get_router_singleton — Test that get_router returns the same instance.
    • test_route_code_prompt_function — Test the convenience route_code_prompt function.
    • test_get_routing_metadata — Test get_routing_metadata formats correctly.
    • test_get_model_tiers — Test get_model_tiers returns valid data.
    • test_get_fallback_model — Test get_fallback_model returns valid model.
    • test_get_baselines — Test get_baselines returns baseline configs.
  • TestEdgeCases (5 tests)
    • test_empty_prompt — Test routing with empty prompt.
    • test_very_long_prompt — Test routing with very long prompt.
    • test_unicode_prompt — Test routing with unicode characters.
    • test_none_context — Test routing with None context.
    • test_invalid_mode_defaults_to_auto — Test that invalid mode is handled gracefully.
  • TestTierSelection (4 tests)
    • test_tier_1_for_architecture — Test that architecture always gets tier 1.
    • test_tier_1_for_agentic_mode — Test that agentic mode always selects tier 1.
    • test_tier_respected_in_price_mode — Test that quality gates are respected even in price mode.
    • test_quality_mode_bumps_tier — Test that quality mode bumps up the tier.

tests/services/test_connection_pool.py

6 tests

  • TestConnectionPool (6 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports
    • test_reuses_client_when_api_key_unchanged — The same provider/base/api_key triple should reuse the cached client.
    • test_rebuilds_client_when_api_key_changes — Rotating API keys should evict the old client and close it.
    • test_get_groq_pooled_client — Test that Groq pooled client is configured correctly.
    • test_get_groq_pooled_client_no_key — Test that Groq pooled client raises error without API key.

tests/services/test_credit_handler.py

21 tests

  • TestRecordCreditMetrics (2 tests)
    • test_records_success_metrics — Test that success metrics are recorded correctly.
    • test_records_retry_metrics — Test that retry metrics are recorded for attempt > 1.
  • TestRecordMissedDeduction (2 tests)
    • test_records_missed_deduction — Test that missed deduction is recorded.
    • test_skips_zero_cost — Test that zero cost doesn't record metric.
  • TestRecordBackgroundTaskFailure (1 tests)
    • test_records_failure — Test that background task failure is recorded.
  • TestSendCriticalBillingAlert (2 tests)
    • test_sends_alert_for_significant_cost — Test that Sentry alert is sent for costs >= $0.01.
    • test_skips_message_for_small_cost — Test that Sentry breadcrumb and alert are sent for all costs.
  • TestHandleCreditsAndUsage (8 tests)
    • test_trial_user_no_deduction — Test that trial users don't get credits deducted.
    • test_paid_user_deduction_success — Test that paid users get credits deducted successfully.
    • test_retry_on_transient_failure — Test that credit deduction retries on transient failure.
    • test_no_retry_on_validation_error — Test that ValueError (insufficient credits) doesn't retry.
    • test_all_retries_exhausted — Test behavior when all retries are exhausted.
    • test_trial_override_for_paid_user_with_stale_flag — Test that paid users with stale is_trial flag still get charged.
    • test_no_duplicate_deduction_when_usage_logging_fails — Test that deduct_credits is only called once even if record_usage fails.
    • test_no_duplicate_deduction_when_rate_limit_update_fails — Test that deduct_credits is only called once even if rate limit update fails.
  • TestHandleCreditsAndUsageWithFallback (3 tests)
    • test_returns_success_on_normal_operation — Test successful operation returns (cost, True).
    • test_returns_failure_and_logs_on_error — Test that failures return (cost, False) and log for reconciliation.
    • test_uses_fallback_cost_if_pricing_fails — Test that fallback cost estimation is used if pricing lookup fails.
  • TestRetryConfiguration (3 tests)
    • test_max_retries_is_reasonable — Test that max retries is between 1 and 5.
    • test_retry_delays_are_exponential — Test that retry delays increase.
    • test_retry_delays_match_max_retries — Test that we have enough delay values for all retries.

tests/services/test_credit_precheck.py

20 tests

  • TestGetModelMaxTokens (5 tests)
    • test_exact_match_gpt4 — Test exact match for GPT-4.
    • test_exact_match_claude — Test exact match for Claude models.
    • test_partial_match_gpt4o — Test partial match for versioned GPT-4o.
    • test_unknown_model_defaults — Test unknown model returns default.
    • test_llama_models — Test Llama model detection.
  • TestCalculateMaximumCost (3 tests)
    • test_with_explicit_max_tokens — Test cost calculation with explicit max_tokens.
    • test_without_max_tokens_uses_model_default — Test cost calculation without max_tokens uses model default.
    • test_zero_max_tokens_uses_model_default — Test cost calculation with zero max_tokens uses model default.
  • TestCheckCreditSufficiency (4 tests)
    • test_sufficient_credits — Test when user has sufficient credits.
    • test_insufficient_credits — Test when user has insufficient credits.
    • test_trial_user_always_allowed — Test that trial users are always allowed (no credit check).
    • test_exact_credits_match — Test when user has exactly enough credits.
  • TestEstimateAndCheckCredits (5 tests)
    • test_complete_flow_with_sufficient_credits — Test complete pre-check flow with sufficient credits.
    • test_complete_flow_with_insufficient_credits — Test complete pre-check flow with insufficient credits.
    • test_trial_user_bypasses_check — Test that trial users bypass credit checks entirely.
    • test_uses_model_default_when_no_max_tokens — Test that model defaults are used when max_tokens not specified.
    • test_expensive_model_high_max_tokens — Test expensive scenario: GPT-4 with high max_tokens.
  • TestIntegrationScenarios (3 tests)
    • test_scenario_user_starts_expensive_request — Scenario: User with $0.01 tries to start a GPT-4 request
    • test_scenario_user_reduces_max_tokens — Scenario: User sees error, reduces max_tokens to lower cost.
    • test_scenario_streaming_vs_non_streaming — Scenario: Same credit check for streaming and non-streaming.

tests/services/test_daily_usage_limiter.py

15 tests

  • TestDailyResetTime (2 tests)
    • test_get_daily_reset_time_future — Test that reset time is always in the future
    • test_get_daily_reset_time_tomorrow_if_past_midnight — Test that reset time is tomorrow if we're past midnight
  • TestDailyUsageTracking (3 tests)
    • test_get_daily_usage_no_transactions — Test get_daily_usage when user has no transactions
    • test_get_daily_usage_with_transactions — Test get_daily_usage with multiple transactions
    • test_get_daily_usage_error_handling — Test that errors are handled gracefully
  • TestDailyUsageLimitCheck (5 tests)
    • test_check_daily_usage_limit_within_limit — Test check when usage is within limit
    • test_check_daily_usage_limit_would_exceed — Test check when request would exceed limit
    • test_check_daily_usage_limit_warning_threshold — Test warning level at 80% usage
    • test_check_daily_usage_limit_critical_threshold — Test warning level at 95% usage
    • test_check_daily_usage_limit_exceeded — Test when limit is already exceeded
  • TestEnforceDailyUsageLimit (2 tests)
    • test_enforce_daily_usage_limit_allowed — Test that enforcement passes when within limit
    • test_enforce_daily_usage_limit_exceeded_raises_exception — Test that enforcement raises exception when limit exceeded
  • TestDailyUsageLimitIntegration (2 tests)
    • test_full_workflow_within_limit — Test full workflow when staying within limit
    • test_full_workflow_exceeds_limit — Test full workflow when exceeding limit
  • TestDisabledLimits (1 tests)
    • test_check_with_limits_disabled — Test that checks pass when limits are disabled

tests/services/test_db_first_catalog.py

10 tests

  • test_get_cached_full_catalog_returns_cache_when_available — Test cache hit returns cached data without DB query
  • test_get_cached_full_catalog_fetches_from_db_on_miss — Test cache miss fetches from database
  • test_get_cached_provider_catalog_fetches_from_db_on_miss — Test provider catalog cache miss fetches from database
  • test_get_cached_models_all_gateway — Test get_cached_models with 'all' gateway
  • test_get_cached_models_single_provider — Test get_cached_models with single provider
  • test_get_cached_models_handles_errors — Test get_cached_models handles errors gracefully
  • test_cache_invalidation_exists — Verify that cache invalidation logic exists.
  • test_full_flow_db_first — Integration test: Full flow with DB-first architecture.
  • test_get_cached_models_empty_gateway — Test handling of empty gateway parameter
  • test_get_cached_models_none_gateway — Test handling of None gateway parameter

tests/services/test_deepinfra_normalization.py

9 tests

  • TestDeepInfraNormalization (9 tests)
    • test_normalize_text_to_image_model — Test normalizing an image generation model
    • test_normalize_text_generation_model — Test normalizing a text generation model
    • test_normalize_model_without_type — Test normalizing a model without explicit type (defaults to text->text)
    • test_normalize_model_with_id_field — Test normalizing a model that uses 'id' instead of 'model_name'
    • test_normalize_model_without_pricing — Test normalizing a model without pricing information
    • test_normalize_deprecated_model_without_replacement — Test normalizing a deprecated model without a replacement
    • test_normalize_model_with_multimodal_type — Test normalizing a multimodal model
    • test_normalize_tts_model — Test normalizing a text-to-speech model
    • test_normalize_stt_model — Test normalizing a speech-to-text model

tests/services/test_email_verification.py

30 tests

  • TestEmailVerificationResult (8 tests)
    • test_disposable_email_is_bot — Disposable emails should be marked as bot.
    • test_risky_low_score_is_bot — Risky emails with low score should be marked as bot.
    • test_risky_high_score_not_bot — Risky emails with high score should not be marked as bot.
    • test_deliverable_email_not_bot — Deliverable emails should not be marked as bot.
    • test_unknown_email_not_bot — Unknown emails should not be marked as bot (benefit of doubt).
    • test_undeliverable_should_block — Undeliverable emails should be blocked.
    • test_invalid_domain_should_block — Invalid domain emails should be blocked.
    • test_deliverable_should_not_block — Deliverable emails should not be blocked.
  • TestEmailVerificationService (9 tests)
    • test_service_disabled_by_default — Service should be disabled by default.
    • test_service_enabled_with_api_key — Service should be enabled when API key and enabled flag are set.
    • test_service_disabled_without_api_key — Service should be disabled if enabled but no API key.
    • test_verify_returns_unknown_when_disabled — When disabled, verify should return unknown result.
    • test_verify_parses_deliverable_response — Service should correctly parse deliverable response.
    • test_verify_parses_disposable_response — Service should correctly identify disposable emails.
    • test_verify_handles_rate_limit — Service should handle rate limit gracefully.
    • test_verify_handles_insufficient_credits — Service should handle insufficient credits gracefully.
    • test_verify_handles_timeout — Service should handle timeout gracefully.
  • TestEmailVerificationEdgeCases (5 tests)
    • test_empty_email — Service should handle empty email gracefully.
    • test_email_without_at_symbol — Service should handle malformed email gracefully.
    • test_email_case_normalization — Service should normalize email to lowercase.
    • test_email_whitespace_trimming — Service should trim whitespace from email.
    • test_did_you_mean_suggestion — Result should include did_you_mean suggestion if provided.
  • TestEmailVerificationCaching (8 tests)
    • test_result_to_cache_dict — Result should correctly convert to cache dict.
    • test_result_from_cache_dict — Result should correctly restore from cache dict.
    • test_cache_roundtrip — Result should survive roundtrip through cache dict.
    • test_verify_caches_successful_result — Successful verification should be cached.
    • test_verify_returns_cached_result — Verification should return cached result if available.
    • test_verify_does_not_cache_api_errors — API errors should not be cached.
    • test_verify_does_not_cache_timeouts — Timeouts should not be cached.
    • test_verify_works_when_redis_unavailable — Verification should work when Redis is unavailable.

tests/services/test_enhanced_notification.py

10 tests

  • TestEmailValidation (7 tests)
    • test_valid_email_passes_validation — Test that valid emails pass validation
    • test_privy_fallback_email_rejected — Test that Privy fallback emails are rejected
    • test_empty_email_rejected — Test that empty/None emails are rejected
    • test_invalid_email_format_rejected — Test that invalid email formats are rejected
    • test_send_welcome_email_skips_privy_fallback — Test that welcome email is skipped for Privy fallback addresses
    • test_send_email_notification_skips_privy_fallback — Test that email notification is skipped for Privy fallback addresses
    • test_send_email_notification_sends_valid_email — Test that valid emails are sent normally
  • TestEdgeCases (3 tests)
    • test_email_ending_with_privy_user_substring — Test emails that contain but don't end with @privy.user
    • test_similar_but_valid_domain — Test emails with similar-looking but valid domains
    • test_password_reset_validates_email_before_token_creation — Test that password reset validates email BEFORE creating token

tests/services/test_error_monitor_loki.py

6 tests

  • TestLokiErrorLogging (6 tests)
    • test_loki_empty_error_not_logged — Test that empty Loki errors are suppressed and not logged at ERROR level.
    • test_loki_none_error_not_logged — Test that 'None' string errors from Loki are suppressed.
    • test_loki_real_error_still_logged — Test that real Loki errors are still logged at ERROR level.
    • test_loki_successful_fetch — Test that successful Loki fetches work correctly.
    • test_loki_disabled_no_error — Test that disabled Loki doesn't trigger errors.
    • test_loki_whitespace_error_suppressed — Test that whitespace-only errors are suppressed.

tests/services/test_fal_image_client.py

11 tests

  • TestFalModelsCatalog (11 tests)
    • test_load_fal_models_catalog — Test loading Fal.ai models catalog
    • test_get_fal_models — Test getting all Fal.ai models
    • test_get_fal_models_by_type_text_to_image — Test filtering models by text-to-image type
    • test_get_fal_models_by_type_image_to_video — Test filtering models by image-to-video type
    • test_get_fal_models_by_type_text_to_video — Test filtering models by text-to-video type
    • test_get_fal_models_by_type_empty — Test filtering with non-existent type returns empty list
    • test_validate_fal_model_valid — Test validating existing model IDs
    • test_validate_fal_model_invalid — Test validating non-existent model IDs
    • test_model_types_coverage — Test that catalog covers all expected model types
    • test_catalog_caching — Test that catalog is cached after first load
    • test_model_structure — Test that all models have consistent structure

tests/services/test_featherless_client.py

15 tests

  • TestGetFeatherlessClient (2 tests)
    • test_get_featherless_client_success — Test successful client initialization
    • test_get_featherless_client_missing_key — Test client initialization with missing API key
  • TestMakeFeatherlessRequest (2 tests)
    • test_make_featherless_request_openai_forwards_args — Test that request forwards all arguments correctly
    • test_make_featherless_request_openai_error — Test request error handling
  • TestMakeFeatherlessRequestStream (2 tests)
    • test_make_featherless_request_openai_stream — Test streaming request
    • test_make_featherless_request_openai_stream_error — Test streaming request error handling
  • TestProcessFeatherlessResponse (3 tests)
    • test_process_featherless_response_happy — Test processing a normal response
    • test_process_featherless_response_no_usage — Test processing response without usage data
    • test_process_featherless_response_error — Test processing error handling
  • TestSanitizeMessagesForFeatherless (6 tests)
    • test_removes_null_tool_calls_from_dict — Test that null tool_calls are removed from dict messages
    • test_preserves_valid_tool_calls — Test that valid tool_calls arrays are preserved
    • test_preserves_messages_without_tool_calls — Test that messages without tool_calls field are unchanged
    • test_does_not_mutate_original_messages — Test that original messages are not modified
    • test_removes_invalid_tool_calls_type — Test that non-array tool_calls are removed
    • test_handles_empty_list — Test that empty message list is handled

tests/services/test_fireworks_client.py

5 tests

  • TestFireworksClient (5 tests)
    • test_get_fireworks_client — Test getting Fireworks client
    • test_get_fireworks_client_no_key — Test getting Fireworks client without API key
    • test_make_fireworks_request_openai — Test making request to Fireworks
    • test_make_fireworks_request_openai_stream — Test making streaming request to Fireworks
    • test_process_fireworks_response — Test processing Fireworks response

tests/services/test_general_router.py

20 tests

  • test_route_general_prompt_convenience_function — Test the convenience function route_general_prompt.
  • TestNormalizeModelString (5 tests)
    • test_normalize_general_router_hyphenated — Test normalizing gatewayz-general aliases.
    • test_normalize_code_router_hyphenated — Test normalizing gatewayz-code aliases.
    • test_normalize_already_normalized — Test that already normalized strings pass through unchanged.
    • test_normalize_regular_model — Test that regular model IDs pass through unchanged.
    • test_normalize_case_insensitive — Test case insensitivity.
  • TestParseRouterModelString (8 tests)
    • test_parse_router_general_balanced — Test parsing router:general (balanced mode).
    • test_parse_router_general_quality — Test parsing router:general:quality.
    • test_parse_router_general_cost — Test parsing router:general:cost.
    • test_parse_router_general_latency — Test parsing router:general:latency.
    • test_parse_invalid_mode — Test parsing invalid mode falls back to balanced.
    • test_parse_non_router_string — Test parsing non-router strings.
    • test_parse_code_router_string — Test that code router strings are not matched.
    • test_parse_case_insensitive — Test case insensitivity.
  • TestGeneralRouterRoute (4 tests)
    • test_route_with_notdiamond_enabled — Test routing when NotDiamond is enabled and working.
    • test_route_with_notdiamond_disabled — Test routing when NotDiamond is disabled.
    • test_route_with_unavailable_model — Test routing when NotDiamond recommends unavailable model.
    • test_route_with_notdiamond_exception — Test routing when NotDiamond throws exception.
  • TestRoutingMetadata (2 tests)
    • test_get_routing_metadata_success — Test metadata formatting for successful routing.
    • test_get_routing_metadata_fallback — Test metadata formatting for fallback routing.

tests/services/test_google_models_config.py

2 tests

  • TestGoogleModelsConfig (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/services/test_google_oauth2_jwt.py

17 tests

  • test_base64url_encode_simple — Test base64url encoding of simple strings
  • test_base64url_encode_json — Test base64url encoding of JSON
  • test_base64url_encode_decode_roundtrip — Test that base64url encoding can be decoded back
  • test_build_jwt_assertion_structure — Test JWT assertion has correct structure
  • test_build_jwt_assertion_with_custom_subject — Test JWT assertion with custom subject claim
  • test_build_jwt_assertion_expiry — Test JWT assertion expiry calculation
  • test_build_jwt_assertion_invalid_key — Test JWT assertion building with invalid private key
  • test_exchange_jwt_for_access_token_success — Test successful JWT exchange for access token
  • test_exchange_jwt_for_access_token_401_error — Test JWT exchange with 401 unauthorized error
  • test_exchange_jwt_for_access_token_500_error — Test JWT exchange with 500 server error
  • test_exchange_jwt_for_access_token_network_error — Test JWT exchange with network error
  • test_get_access_token_from_service_account_success — Test getting access token from service account JSON
  • test_get_access_token_from_service_account_invalid_json — Test with invalid service account JSON
  • test_get_access_token_from_service_account_missing_email — Test with service account JSON missing client_email
  • test_get_access_token_from_service_account_missing_key — Test with service account JSON missing private_key
  • test_get_access_token_from_service_account_custom_scope — Test with custom OAuth2 scope
  • test_full_oauth2_jwt_flow_integration — Test the complete OAuth2 JWT flow end-to-end

tests/services/test_google_oauth2_jwt_id_token_fallback.py

7 tests

  • TestIdTokenFallback (7 tests)
    • test_exchange_jwt_returns_access_token_when_present — Test normal flow: access_token is present in response
    • test_exchange_jwt_falls_back_to_id_token — Test fallback: id_token is used when access_token is missing
    • test_exchange_jwt_prefers_access_token_over_id_token — Test that access_token is preferred when both are present
    • test_exchange_jwt_raises_when_no_token_present — Test that error is raised when neither token is present
    • test_exchange_jwt_handles_http_error — Test that HTTP errors are properly handled
    • test_exchange_jwt_sends_correct_request — Test that the correct request is sent to Google's token endpoint
    • test_id_token_with_real_google_response_structure — Test with a realistic Google OAuth2 response containing id_token

tests/services/test_google_vertex_client.py

40 tests

  • TestTransformGoogleVertexModelId (4 tests)
    • test_transform_simple_model_id — Test transforming a simple model ID returns the model name
    • test_transform_full_resource_name — Test that full resource names are extracted to simple model name
    • test_transform_google_prefix — Test that google/ prefix is stripped from model IDs
    • test_transform_various_models — Test transforming various model IDs
  • TestBuildVertexContent (6 tests)
    • test_build_simple_text_content — Test building content from simple text messages
    • test_build_multimodal_content — Test building multimodal content with images
    • test_build_base64_image_content_strips_data_url_prefix — Test that base64 image data URLs are properly parsed to extract raw base64 data.
    • test_build_base64_image_content_jpeg — Test that JPEG base64 images are correctly parsed with proper MIME type.
    • test_build_base64_image_content_webp — Test that WebP base64 images are correctly parsed with proper MIME type.
    • test_build_system_message — Test that system messages are mapped to model role
  • TestProcessGoogleVertexResponse (3 tests)
    • test_process_successful_rest_response — Test processing a successful REST API response
    • test_process_multiple_content_parts — Test processing response with multiple content parts
    • test_process_gemini_flash_lite_response — Test processing response from gemini-2.5-flash-lite-preview-09-2025
  • TestMakeGoogleVertexRequest (3 tests)
    • test_make_request_with_parameters — Test making a request with various parameters
    • test_make_streaming_request — Test making a streaming request
    • test_make_request_gemini_flash_lite — Test making a request to gemini-2.5-flash-lite (maps to preview version)
  • TestGoogleVertexRestTransport (2 tests)
    • test_rest_request_success — Ensure REST transport returns normalized response.
    • test_rest_request_http_error — REST transport should raise ValueError on HTTP error.
  • TestGoogleVertexModelIntegration (2 tests)
    • test_gemini_model_detection — Test that gemini models are properly detected when credentials are available
    • test_model_id_transformation_consistency — Test that model IDs are transformed consistently
  • TestFetchModelsFromGoogleVertex (6 tests)
    • test_fetch_models_returns_list — Test that fetch_models_from_google_vertex returns a list of models
    • test_fetch_models_includes_gemini_models — Test that returned models include known Gemini models
    • test_fetch_models_normalized_format — Test that returned models have the expected normalized format
    • test_fetch_models_updates_cache — Test that fetch_models_from_google_vertex updates the cache
    • test_fetch_models_includes_provider_model_id — Test that returned models include provider_model_id for database sync.
    • test_fetch_models_raw_google_vertex_includes_provider_model_id — Test that raw_google_vertex metadata includes provider_model_id.
  • TestModelLocationRouting (8 tests)
    • test_gemini_3_uses_global_endpoint — Test that Gemini 3 models use the global endpoint
    • test_gemini_2_uses_regional_endpoint — Test that Gemini 2.x models use the configured regional endpoint
    • test_gemini_1_uses_regional_endpoint — Test that Gemini 1.x models use the configured regional endpoint
    • test_gemma_models_use_regional_endpoint — Test that Gemma models use the configured regional endpoint
    • test_rest_request_uses_global_for_gemini_3 — Ensure REST transport constructs URL with global endpoint for Gemini 3
    • test_rest_request_uses_regional_for_gemini_2 — Ensure REST transport constructs URL with regional endpoint for Gemini 2
    • test_rest_request_strips_google_prefix_from_model_id — Ensure google/ prefix is stripped from model ID to prevent 404 errors.
    • test_sdk_request_uses_global_for_gemini_3 — Ensure SDK transport initializes with global location for Gemini 3
  • TestMaxOutputTokensValidation (3 tests)
    • test_max_tokens_capped_at_65536 — Test that max_tokens > 65536 is clamped to 65536 for Vertex AI.
    • test_max_tokens_minimum_is_16 — Test that max_tokens < 16 is raised to 16 for Vertex AI.
    • test_valid_max_tokens_unchanged — Test that valid max_tokens values are not modified.
  • TestNoCandidatesErrorHandling (3 tests)
    • test_no_candidates_without_prompt_feedback — Test that 'no candidates' error includes model version when no promptFeedback is present
    • test_no_candidates_with_block_reason — Test that 'no candidates' error includes block reason when promptFeedback is present
    • test_no_candidates_with_safety_ratings — Test that safety rating details are included in error when blocked

tests/services/test_google_vertex_multi_endpoint.py

6 tests

  • TestGoogleVertexMultiEndpoint (6 tests)
    • test_first_endpoint_success — Test successful fetch from first endpoint (regional with project)
    • test_fallback_to_second_endpoint — Test fallback to second endpoint when first fails with 404
    • test_fallback_to_global_endpoint — Test fallback to global endpoint when regional endpoints fail
    • test_all_endpoints_fail_returns_none — Test that None is returned when all endpoints fail
    • test_exception_handling_falls_through — Test that exceptions during fetch are handled and fallback occurs
    • test_correct_endpoint_urls — Test that the correct endpoint URLs are constructed

tests/services/test_google_vertex_tools.py

19 tests

  • TestOpenAIToVertexToolsTranslation (9 tests)
    • test_translate_single_tool — Test translating a single OpenAI tool to Vertex format
    • test_translate_multiple_tools — Test translating multiple OpenAI tools to Vertex format
    • test_translate_tool_without_description — Test translating a tool without description
    • test_translate_tool_without_parameters — Test translating a tool without parameters
    • test_translate_empty_tools_list — Test translating an empty tools list
    • test_translate_none_tools — Test translating None tools
    • test_skip_non_function_tools — Test that non-function tools are skipped with warning
    • test_skip_tool_without_name — Test that tools without name are skipped
    • test_skip_tool_without_function_definition — Test that tools without function definition are skipped
  • TestToolChoiceTranslation (8 tests)
    • test_translate_tool_choice_none — Test translating tool_choice='none'
    • test_translate_tool_choice_auto — Test translating tool_choice='auto'
    • test_translate_tool_choice_required — Test translating tool_choice='required'
    • test_translate_tool_choice_specific_function — Test translating tool_choice with specific function
    • test_translate_tool_choice_null — Test translating tool_choice=None
    • test_translate_unknown_tool_choice_string — Test translating unknown tool_choice string defaults to AUTO
    • test_translate_tool_choice_function_missing_name — Test translating tool_choice function without name
    • test_translate_tool_choice_function_is_none — Test translating tool_choice when function value is None
  • TestGoogleVertexToolsSupport (2 tests)
    • test_tools_extracted_from_kwargs — Test that tools are extracted from kwargs
    • test_tools_request_completes_successfully — Test that requests with tools complete successfully

tests/services/test_groq_client.py

11 tests

  • TestGroqClient (11 tests)
    • test_get_groq_client — Test getting Groq client
    • test_get_groq_client_no_key — Test getting Groq client without API key
    • test_make_groq_request_openai — Test making request to Groq
    • test_make_groq_request_openai_with_kwargs — Test making request to Groq with additional parameters
    • test_make_groq_request_openai_error — Test handling errors from Groq
    • test_make_groq_request_openai_stream — Test making streaming request to Groq
    • test_make_groq_request_openai_stream_with_kwargs — Test making streaming request to Groq with additional parameters
    • test_make_groq_request_openai_stream_error — Test handling streaming errors from Groq
    • test_process_groq_response — Test processing Groq response
    • test_process_groq_response_no_usage — Test processing Groq response without usage data
    • test_process_groq_response_multiple_choices — Test processing Groq response with multiple choices

tests/services/test_helicone_client.py

22 tests

  • TestGetHeliconeClient (3 tests)
    • test_get_helicone_client — Test getting Helicone client with valid API key
    • test_get_helicone_client_no_key — Test getting Helicone client without API key raises error
    • test_get_helicone_client_empty_key — Test getting Helicone client with empty API key raises error
  • TestMakeHeliconeRequest (3 tests)
    • test_make_helicone_request — Test making request to Helicone AI Gateway
    • test_make_helicone_request_with_kwargs — Test making request with additional parameters
    • test_make_helicone_request_error — Test handling errors from Helicone
  • TestMakeHeliconeRequestStream (3 tests)
    • test_make_helicone_request_stream — Test making streaming request to Helicone
    • test_make_helicone_request_stream_with_kwargs — Test making streaming request with additional parameters
    • test_make_helicone_request_stream_error — Test handling errors during streaming
  • TestProcessHeliconeResponse (3 tests)
    • test_process_helicone_response — Test processing Helicone response
    • test_process_helicone_response_no_usage — Test processing response without usage data
    • test_process_helicone_response_multiple_choices — Test processing response with multiple choices
  • TestFetchModelPricing (5 tests)
    • test_fetch_pricing_no_api_key — Test fetching pricing without API key returns None
    • test_fetch_pricing_placeholder_key — Test fetching pricing with placeholder key returns None
    • test_fetch_pricing_during_catalog_build — Test fetching pricing during catalog build returns None
    • test_fetch_pricing_from_api — Test fetching pricing from Helicone API
    • test_fetch_pricing_fallback_to_provider — Test fetching pricing falls back to provider lookup
  • TestGetProviderPricing (4 tests)
    • test_get_provider_pricing_during_catalog_build — Test getting provider pricing during catalog build returns None
    • test_get_provider_pricing_found — Test getting provider pricing when found
    • test_get_provider_pricing_not_found — Test getting provider pricing when not found
    • test_get_provider_pricing_with_prefix — Test getting provider pricing with provider prefix
  • TestHeliconeTimeout (1 tests)
    • test_helicone_timeout_values — Test that timeout values are correctly configured

tests/services/test_huggingface_400_error_handling.py

4 tests

  • TestHuggingFace400ErrorHandling (4 tests)
    • test_400_error_with_detailed_logging — Test that 400 errors are logged with detailed information including payload
    • test_400_error_streaming_with_detailed_logging — Test that streaming 400 errors are logged with detailed information
    • test_non_400_error_standard_logging — Test that non-400 errors use standard error logging
    • test_400_error_without_text_attribute — Test 400 error handling when response has no text attribute

tests/services/test_huggingface_client.py

15 tests

  • TestHuggingFaceClient (7 tests)
    • test_get_huggingface_client — Test getting Hugging Face client
    • test_get_huggingface_client_no_key — Test getting Hugging Face client without API key
    • test_make_huggingface_request_openai — Test making request to Hugging Face
    • test_make_huggingface_request_openai_stream — Test making streaming request to Hugging Face
    • test_make_huggingface_request_openai_foreign_namespace — Models scoped to other providers should not be routed through Hugging Face
    • test_make_huggingface_request_openai_stream_foreign_namespace — Streaming helper should also reject foreign provider models
    • test_process_huggingface_response — Test processing Hugging Face response
  • TestModelIdCaseResolution (8 tests)
    • test_resolve_model_id_case_with_cached_model — Test that lowercase model IDs are resolved to correct case from cache
    • test_resolve_model_id_case_preserves_suffix — Test that :hf-inference suffix is preserved after case resolution
    • test_resolve_model_id_case_not_in_cache — Test that model IDs not in cache are returned as-is
    • test_resolve_model_id_case_empty_cache — Test that model IDs are returned as-is when cache is empty
    • test_resolve_model_id_case_already_correct — Test that correctly-cased model IDs are returned unchanged
    • test_prepare_model_resolves_case_and_adds_suffix — Test that _prepare_model resolves case and adds suffix
    • test_prepare_model_does_not_double_suffix — Test that _prepare_model doesn't add suffix if already present
    • test_request_uses_correct_case — Test that requests use the correctly-cased model ID

tests/services/test_huggingface_models.py

2 tests

  • TestHuggingfaceModels (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/services/test_huggingface_tools.py

4 tests

  • TestHuggingFaceToolsSupport (4 tests)
    • test_tools_in_allowed_params — Test that 'tools' is in ALLOWED_PARAMS
    • test_build_payload_includes_tools — Test that _build_payload includes tools when provided
    • test_build_payload_without_tools — Test that _build_payload works without tools
    • test_build_payload_filters_other_params — Test that _build_payload only includes allowed params

tests/services/test_image_generation_client.py

2 tests

  • TestImageGenerationClient (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/services/test_json_parsing_robustness.py

14 tests

  • TestHuggingFaceModelsJSONParsing (4 tests)
    • test_fetch_models_with_invalid_json_response — Test that fetch_huggingface_models handles invalid JSON gracefully
    • test_search_models_with_invalid_json — Test that search_huggingface_models handles malformed JSON
    • test_get_model_info_with_invalid_json — Test that get_huggingface_model_info handles JSON errors
    • test_fetch_models_with_html_error_response — Test handling when API returns HTML error page instead of JSON
  • TestProvidersJSONParsing (2 tests)
    • test_fetch_providers_with_invalid_json — Test that fetch_providers handles malformed JSON response
    • test_fetch_providers_with_empty_response — Test handling of empty response body
  • TestModelsJSONParsing (2 tests)
    • test_fetch_openrouter_models_with_invalid_json — Test that fetch_openrouter_models handles JSON parse errors
    • test_fetch_models_with_corrupted_json_response — Test handling of corrupted/truncated JSON
  • TestJSONParsingWithDifferentContentTypes (2 tests)
    • test_xml_content_type_returned_instead_of_json — Test when API incorrectly returns XML but claims JSON
    • test_plain_text_error_message_as_json — Test handling when API returns plain text error
  • TestJSONParsingErrorLogging (2 tests)
    • test_json_error_is_logged_with_context — Verify JSON parse errors are logged with useful context
    • test_providers_json_error_logging — Verify provider JSON errors include response details
  • TestJSONParsingEdgeCases (2 tests)
    • test_null_json_response — Test when response.json() returns None
    • test_json_with_unexpected_structure — Test when JSON structure differs from expected

tests/services/test_model_availability.py

7 tests

  • TestModelAvailability (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports
  • TestIsModelAvailable (5 tests)
    • test_unknown_model_returns_available — Test that unknown models (not in availability cache) return True.
    • test_available_model_returns_available — Test that models with AVAILABLE status return True
    • test_model_with_open_circuit_returns_unavailable — Test that models with OPEN circuit breaker return False
    • test_model_in_maintenance_returns_unavailable — Test that models in maintenance mode return False
    • test_unavailable_status_returns_unavailable — Test that models with UNAVAILABLE status return False

tests/services/test_model_availability_comprehensive.py

37 tests

  • TestCircuitBreakerStateTransitions (8 tests)
    • test_initial_state_is_closed — Test circuit breaker starts in CLOSED state
    • test_can_execute_when_closed — Test requests can execute when CLOSED
    • test_transitions_to_open_after_failures — Test CLOSED -> OPEN after failure threshold
    • test_cannot_execute_when_open — Test requests blocked when OPEN
    • test_transitions_to_half_open_after_recovery_timeout — Test OPEN -> HALF_OPEN after recovery timeout
    • test_half_open_to_closed_after_successes — Test HALF_OPEN -> CLOSED after success threshold
    • test_half_open_to_open_on_failure — Test HALF_OPEN -> OPEN on failure
    • test_success_decrements_failure_count_when_closed — Test success reduces failure count in CLOSED state
  • TestAvailabilityConfig (2 tests)
    • test_default_config_values — Test default configuration values
    • test_custom_config_values — Test custom configuration values
  • TestModelAvailabilityDataclass (1 tests)
    • test_create_model_availability — Test creating ModelAvailability instance
  • TestModelAvailabilityService (16 tests)
    • test_service_initialization — Test service initializes correctly
    • test_fallback_mappings_loaded — Test fallback mappings are loaded correctly
    • test_get_fallback_models — Test getting fallback models
    • test_get_model_availability_not_found — Test getting availability for unknown model
    • test_get_model_availability_with_gateway — Test getting availability with specific gateway
    • test_get_available_models_empty — Test getting available models when cache is empty
    • test_get_available_models_filtered — Test getting available models with filters
    • test_is_model_available — Test model availability check
    • test_is_model_available_with_open_circuit — Test availability check with open circuit breaker
    • test_is_model_available_during_maintenance — Test availability check during maintenance
    • test_get_best_available_model_preferred_available — Test best model selection when preferred is available
    • test_get_best_available_model_with_fallback — Test best model selection falls back when preferred unavailable
    • test_get_availability_summary_empty — Test availability summary with empty cache
    • test_get_availability_summary — Test availability summary with data
    • test_set_maintenance_mode — Test setting maintenance mode
    • test_clear_maintenance_mode — Test clearing maintenance mode
  • TestModelAvailabilityServiceAsync (3 tests)
    • test_start_monitoring — Test starting availability monitoring
    • test_start_monitoring_idempotent — Test starting monitoring multiple times is idempotent
    • test_stop_monitoring_when_not_started — Test stopping monitoring when not started
  • TestAvailabilityStatus (5 tests)
    • test_available_status — Test AVAILABLE status
    • test_unavailable_status — Test UNAVAILABLE status
    • test_degraded_status — Test DEGRADED status
    • test_maintenance_status — Test MAINTENANCE status
    • test_unknown_status — Test UNKNOWN status
  • TestGlobalAvailabilityService (2 tests)
    • test_global_service_exists — Test global service is initialized
    • test_global_service_has_fallback_mappings — Test global service has fallback mappings loaded

tests/services/test_model_catalog_sync.py

22 tests

  • TestTransformNormalizedModelToDbSchema (3 tests)
    • test_uses_explicit_provider_model_id — Test that explicit provider_model_id is used when available.
    • test_falls_back_to_model_id_when_provider_model_id_missing — Test that model_id is used when provider_model_id is not specified.
    • test_handles_none_provider_model_id — Test that None provider_model_id falls back to model_id.
  • TestGoogleVertexProviderModelId (3 tests)
    • test_gemini_3_flash_provider_model_id — Test that Gemini 3 Flash models have correct provider_model_id.
    • test_gemini_3_pro_provider_model_id — Test that Gemini 3 Pro models have correct provider_model_id.
    • test_gemini_2_models_same_provider_model_id — Test that Gemini 2.x models have same model_id and provider_model_id.
  • TestExtractModality (3 tests)
    • test_extract_from_architecture — Test modality extraction from architecture field.
    • test_extract_from_top_level — Test modality extraction from top-level field.
    • test_default_modality — Test default modality when not specified.
  • TestExtractPricing (2 tests)
    • test_extract_valid_pricing — Test pricing extraction with valid values.
    • test_extract_missing_pricing — Test pricing extraction with missing pricing.
  • TestExtractCapabilities (2 tests)
    • test_extract_vision_support — Test vision support extraction.
    • test_no_vision_support — Test no vision support.
  • TestProviderFetchFunctionsRegistry (9 tests)
    • test_openai_registered — Test that OpenAI provider is registered.
    • test_anthropic_registered — Test that Anthropic provider is registered.
    • test_clarifai_registered — Test that Clarifai provider is registered.
    • test_simplismart_registered — Test that SimpliSmart provider is registered.
    • test_onerouter_registered — Test that OneRouter provider is registered.
    • test_cloudflare_workers_ai_registered — Test that Cloudflare Workers AI provider is registered.
    • test_modelz_registered — Test that Modelz provider is registered.
    • test_all_core_providers_registered — Test that all core providers have fetch functions registered.
    • test_provider_count — Test that we have the expected number of providers registered.

tests/services/test_model_catalog_validator.py

15 tests

  • TestValidateModelAvailability (9 tests)
    • test_validate_cerebras_model_available — Test validating available Cerebras model
    • test_validate_cerebras_model_unavailable — Test validating unavailable Cerebras model
    • test_validate_huggingface_model_available — Test validating available HuggingFace model
    • test_validate_huggingface_model_not_on_hub — Test model not found on HuggingFace Hub
    • test_validate_huggingface_model_not_on_inference_router — Test model exists on Hub but not available on Inference Router
    • test_validate_openrouter_model_available — Test validating available OpenRouter model
    • test_validate_openrouter_model_unavailable — Test validating unavailable OpenRouter model
    • test_validate_unsupported_provider — Test validation for provider without validation implementation
    • test_validation_caching — Test that validation results are cached
  • TestValidateModelsBatch (4 tests)
    • test_validate_batch_all_available — Test validating batch where all models are available
    • test_validate_batch_some_unavailable — Test validating batch where some models are unavailable
    • test_validate_batch_empty_list — Test validating empty batch
    • test_validate_batch_with_exceptions — Test batch validation handles exceptions gracefully
  • TestValidationCacheManagement (2 tests)
    • test_clear_specific_model_cache — Test clearing cache for specific model
    • test_clear_all_validation_cache — Test clearing all validation cache

tests/services/test_model_health_monitor.py

51 tests

  • TestModelHealthTracking (4 tests)
    • test_get_model_health_returns_none_for_unknown — Get health for unknown model returns None
    • test_get_all_models_health_returns_list — Get all models health returns a list
    • test_get_health_summary_returns_dict — Get health summary returns a dictionary
    • test_get_system_health_initially_none — System health is None before monitoring starts
  • TestHealthStatusManagement (3 tests)
    • test_get_provider_health_returns_none_for_unknown — Get provider health for unknown provider returns None
    • test_get_all_providers_health_returns_list — Get all providers health returns a list
    • test_health_data_structure — Health data internal structure is initialized
  • TestHealthScoreCalculation (3 tests)
    • test_calculate_health_score_all_success — Calculate health score with all successful requests
    • test_calculate_health_score_mixed — Calculate health score with mixed results
    • test_calculate_health_score_all_failures — Calculate health score with all failures
  • TestHealthMetrics (2 tests)
    • test_model_health_metrics_dataclass — Test ModelHealthMetrics dataclass structure
    • test_model_health_metrics_unhealthy — Test ModelHealthMetrics with unhealthy status
  • TestFailureDetection (3 tests)
    • test_detect_high_error_rate — Detect when error rate exceeds threshold
    • test_detect_slow_response — Detect when latency exceeds threshold
    • test_consecutive_failures — Detect consecutive failures
  • TestRecoveryDetection (2 tests)
    • test_health_status_enum_values — Test HealthStatus enum has expected values
    • test_health_status_transitions — Test that health status can transition between states
  • TestHealthAlerts (2 tests)
    • test_alert_on_model_failure — Alert should trigger when model fails
    • test_alert_cooldown — Alerts should have cooldown period
  • TestHealthMonitoringEdgeCases (3 tests)
    • test_get_model_health_with_none_model_id — Handle None model ID gracefully
    • test_get_model_health_with_empty_string — Handle empty string model ID
    • test_get_provider_health_with_none_values — Handle None provider/gateway gracefully
  • TestHealthMonitorPersistence (2 tests)
    • test_health_data_dict_initialized — Health data dictionary is initialized on monitor creation
    • test_model_health_metrics_has_timestamp — ModelHealthMetrics includes last_checked timestamp
  • TestHealthMonitorIntegration (2 tests)
    • test_unhealthy_models_excluded_from_routing — Unhealthy models should be excluded from routing
    • test_health_check_integration — Health monitor should integrate with health check endpoint
  • TestPerformanceMetrics (3 tests)
    • test_calculate_success_rate — Calculate success rate correctly
    • test_calculate_average_latency — Calculate average latency
    • test_calculate_error_rate — Calculate error rate
  • TestSentryErrorCapture (3 tests)
    • test_sentry_capture_on_model_failure — Test that Sentry captures errors when models fail health checks
    • test_sentry_capture_on_exception — Test that Sentry captures exceptions during health checks
    • test_sentry_not_captured_on_success — Test that Sentry does not capture errors when models are healthy
  • TestApiKeyForGateway (2 tests)
    • test_get_api_key_for_known_gateway — Test that API key is returned for known gateways
    • test_get_api_key_for_unknown_gateway — Test that None is returned for unknown gateways
  • TestPerformModelRequestAuthentication (2 tests)
    • test_returns_error_when_no_api_key_configured — Test that request fails when no API key is configured for gateway
    • test_returns_error_for_unknown_gateway — Test that request fails for unknown gateway (no URL configured)
  • TestShouldCaptureError (15 tests)
    • test_should_capture_unknown_errors — Test that unknown errors (no status code) are captured
    • test_should_not_capture_rate_limits — Test that rate limit errors (429) are NOT captured
    • test_should_not_capture_data_policy_errors — Test that data policy restriction errors (404) are NOT captured
    • test_should_capture_other_404_errors — Test that other 404 errors (not data policy) ARE captured
    • test_should_not_capture_service_unavailable — Test that service unavailable errors (503) are NOT captured
    • test_should_capture_other_503_errors — Test that other 503 errors ARE captured
    • test_should_not_capture_max_output_tokens_validation — Test that max_output_tokens validation errors (400) are NOT captured
    • test_should_not_capture_audio_modality_errors — Test that audio modality requirement errors (400) are NOT captured
    • test_should_capture_other_400_errors — Test that other 400 errors ARE captured
    • test_should_not_capture_auth_key_errors — Test that authentication key errors (403) are NOT captured
    • test_should_capture_other_403_errors — Test that other 403 errors ARE captured
    • test_should_capture_server_errors — Test that server errors (500, 502, etc.) ARE captured
    • test_should_capture_client_errors — Test that most client errors ARE captured
    • test_error_filtering_in_check_model_health_rate_limit — Test that rate limit errors are not sent to Sentry during health checks
    • test_error_filtering_in_check_model_health_server_error — Test that server errors ARE sent to Sentry during health checks

tests/services/test_model_transformations.py

34 tests

  • test_openrouter_prefixed_model_keeps_nested_provider
  • test_openrouter_gpt51_hyphen_alias_transforms
  • test_detect_provider_gpt51_alias_without_org — Test that gpt-5-1 alias routes to native OpenAI provider.
  • test_bare_openai_model_names_alias_to_canonical — Test that bare OpenAI model names (without openai/ prefix) are aliased correctly.
  • test_bare_openai_model_names_detect_as_native_openai — Test that bare OpenAI model names are detected as native OpenAI provider.
  • test_bare_anthropic_model_names_alias_to_canonical — Test that bare Anthropic/Claude model names (without anthropic/ prefix) are aliased correctly.
  • test_bare_anthropic_model_names_detect_as_native_anthropic — Test that bare Anthropic/Claude model names are detected as native Anthropic provider.
  • test_openrouter_auto_preserves_prefix
  • test_openrouter_auto_transforms_for_huggingface
  • test_openrouter_auto_transforms_for_cerebras
  • test_detect_provider_from_model_id_fal_ai — Test that fal-ai models are detected as 'fal' provider
  • test_detect_provider_from_model_id_fal_orgs — Test that various Fal-related orgs are detected as 'fal' provider
  • test_detect_provider_from_model_id_existing_providers — Test that existing provider detection still works.
  • test_detect_provider_google_vertex_models — Test that Google Vertex AI models are correctly detected when credentials are available
  • test_detect_provider_at_prefix_models — Test that models with @ prefix are routed to OpenRouter after Portkey removal
  • test_z_ai_glm_with_exacto_suffix — Test that z-ai/glm-4.6:exacto is correctly detected as OpenRouter
  • test_z_ai_glm_prefix_detected_as_openrouter — Test that z-ai/ prefixed models are detected as OpenRouter provider
  • test_z_ai_glm_47_alias_transforms — Test that z-ai/glm-4.7 (non-existent model) is aliased to existing model
  • test_openrouter_colon_suffix_variants — Test that OpenRouter models with colon suffixes are correctly detected
  • test_detect_provider_groq_models — Test that Groq models are correctly detected as 'groq' provider
  • test_transform_groq_model_strips_prefix — Test that groq/ prefix is stripped when transforming for Groq provider
  • test_transform_groq_model_without_prefix — Test that Groq models without prefix pass through correctly
  • test_detect_provider_onerouter_prefixed_models — Test that onerouter/ prefixed models are detected as 'onerouter' provider
  • test_detect_provider_onerouter_versioned_models — Test that OneRouter models with @ version suffix are detected correctly
  • test_transform_onerouter_strips_prefix — Test that onerouter/ prefix is stripped when transforming for OneRouter provider
  • test_transform_onerouter_passthrough_versioned — Test that versioned OneRouter models pass through correctly
  • test_transform_onerouter_simple_names — Test that simple model names get @ version suffix added
  • test_transform_onerouter_unknown_model_passthrough — Test that unknown models pass through as lowercase
  • test_onerouter_model_id_mapping_exists — Test that OneRouter has model ID mappings defined
  • test_fireworks_unknown_model_does_not_construct_invalid_id — Test that unknown models are NOT naively transformed into invalid Fireworks IDs.
  • test_fireworks_known_model_still_transforms — Test that known Fireworks models are still properly transformed.
  • test_fireworks_unknown_model_without_slash_passthrough — Test that unknown models without org prefix pass through as-is.
  • test_fireworks_nonexistent_variant_passthrough — Test various nonexistent model variants pass through without naive construction.
  • test_fireworks_fuzzy_match_still_works — Test that fuzzy matching still works for models with slight variations.

tests/services/test_model_transformations_comprehensive.py

65 tests

  • TestModelAliasResolution (6 tests)
    • test_apply_model_alias_gpt_variants — Test GPT-5.1 alias variants
    • test_apply_model_alias_xai_deprecated — Test XAI deprecated model aliases - now mapped to canonical x-ai/ prefix
    • test_apply_model_alias_case_insensitive — Test that alias lookup is case insensitive
    • test_apply_model_alias_no_match — Test non-aliased models pass through
    • test_apply_model_alias_none_input — Test None input returns None
    • test_apply_model_alias_empty_string — Test empty string returns empty string
  • TestFireworksTransformations (8 tests)
    • test_transform_deepseek_v3 — Test DeepSeek-V3 transformation
    • test_transform_deepseek_v3_alt_org — Test DeepSeek-V3 with alternative org prefix
    • test_transform_llama_3_3_70b — Test Llama 3.3 70B transformation
    • test_transform_llama_3_1_70b — Test Llama 3.1 70B transformation
    • test_transform_qwen_models — Test Qwen model transformations
    • test_transform_already_in_fireworks_format — Test model already in Fireworks format passes through
    • test_transform_unknown_model_passes_through — Test unknown model passes through as-is (Fireworks API will reject if invalid)
    • test_transform_deepseek_r1 — Test DeepSeek-R1 transformation
  • TestOpenRouterTransformations (6 tests)
    • test_transform_openrouter_prefix_stripped — Test openrouter/ prefix is stripped
    • test_transform_openrouter_auto_preserved — Test openrouter/auto is preserved
    • test_transform_openrouter_bodybuilder_preserved — Test openrouter/bodybuilder is preserved
    • test_transform_openrouter_meta_models_preserved — Test all OpenRouter meta-models are preserved
    • test_transform_claude_sonnet_variants — Test Claude Sonnet 4.5 transformation variants
    • test_transform_cerebras_to_openrouter — Test Cerebras model routed through OpenRouter
  • TestOpenRouterAutoFallbacks (4 tests)
    • test_openrouter_auto_fallback_to_cerebras — Test openrouter/auto fallback to Cerebras
    • test_openrouter_auto_fallback_to_huggingface — Test openrouter/auto fallback to HuggingFace
    • test_openrouter_auto_fallback_to_google_vertex — Test openrouter/auto fallback to Google Vertex
    • test_openrouter_auto_fallback_to_alibaba — Test openrouter/auto fallback to Alibaba Cloud
  • TestGroqTransformations (4 tests)
    • test_transform_groq_prefix_stripped — Test groq/ prefix is stripped
    • test_transform_groq_mixtral — Test Groq Mixtral transformation
    • test_transform_groq_gemma — Test Groq Gemma transformation
    • test_transform_groq_without_prefix — Test Groq model without prefix
  • TestGoogleVertexTransformations (5 tests)
    • test_transform_gemini_2_5_flash — Test Gemini 2.5 Flash transformation
    • test_transform_gemini_with_google_prefix — Test Gemini with google/ prefix
    • test_transform_gemini_with_at_prefix — Test Gemini with @google/models/ prefix
    • test_transform_gemini_1_5_pro — Test Gemini 1.5 Pro transformation
    • test_transform_gemini_2_0_flash — Test Gemini 2.0 Flash transformation
  • TestHuggingFaceTransformations (5 tests)
    • test_transform_llama_models — Test Llama model transformations
    • test_transform_deepseek_models — Test DeepSeek model transformations
    • test_transform_qwen_models — Test Qwen model transformations
    • test_transform_mistral_models — Test Mistral model transformations
    • test_hug_alias_same_as_huggingface — Test 'hug' alias behaves same as 'huggingface'
  • TestNearTransformations (1 tests)
    • test_transform_near_prefix_stripped — Test near/ prefix is stripped
  • TestAIMOTransformations (1 tests)
    • test_transform_aimo_prefix_stripped — Test aimo/ prefix is stripped
  • TestMorpheusTransformations (1 tests)
    • test_transform_morpheus_prefix_stripped — Test morpheus/ prefix is stripped
  • TestProviderDetection (6 tests)
    • test_detect_fal_ai_provider — Test Fal.ai provider detection
    • test_detect_fal_related_orgs — Test Fal-related org detection
    • test_detect_openrouter_colon_suffix — Test OpenRouter models with colon suffix
    • test_detect_groq_models — Test Groq model detection
    • test_detect_google_vertex_models — Test Google Vertex AI model detection
    • test_detect_at_prefix_models — Test @ prefix model detection
  • TestModelIdNormalization (3 tests)
    • test_transform_normalizes_to_lowercase — Test that output is normalized to lowercase
    • test_transform_empty_string — Test empty string input
    • test_transform_none_input — Test None input
  • TestGetModelIdMapping (5 tests)
    • test_get_fireworks_mapping — Test getting Fireworks mapping
    • test_get_openrouter_mapping — Test getting OpenRouter mapping
    • test_get_groq_mapping — Test getting Groq mapping
    • test_get_google_vertex_mapping — Test getting Google Vertex AI mapping
    • test_get_unknown_provider_mapping — Test getting mapping for unknown provider returns empty dict
  • TestModelProviderOverrides (2 tests)
    • test_katanemo_override — Test Katanemo model override
    • test_zai_override — Test Z-AI model override
  • TestOpenRouterAutoFallbackConfig (2 tests)
    • test_fallback_providers_defined — Test fallback providers are defined
    • test_fallback_models_valid — Test fallback models are valid model IDs
  • TestEdgeCases (4 tests)
    • test_transform_with_version_variations — Test transformation handles version variations
    • test_transform_preserves_special_characters — Test transformation preserves necessary special characters
    • test_transform_handles_mixed_case — Test transformation handles mixed case input
    • test_transform_with_instruct_suffix — Test transformation handles instruct suffix
  • TestNormalizeModelName (2 tests)
    • test_normalize_basic — Test basic normalization
    • test_normalize_removes_version_separators — Test normalization handles version separators

tests/services/test_models.py

47 tests

  • TestSanitizePricing (11 tests)
    • test_sanitize_none — Test with None input
    • test_sanitize_empty_dict — Test with empty dict
    • test_sanitize_non_dict — Test with non-dict input
    • test_sanitize_positive_values — Test that positive values are preserved
    • test_sanitize_negative_values_returns_none — Test that negative values (dynamic pricing) return None to filter model
    • test_sanitize_mixed_values_with_negative_returns_none — Test with mix of positive and negative values returns None
    • test_sanitize_numeric_negative_values_returns_none — Test with numeric negative values returns None
    • test_sanitize_invalid_values — Test with invalid numeric values
    • test_sanitize_zero_values — Test that zero values are preserved
    • test_sanitize_with_negative_prompt_returns_none — Test that negative pricing in prompt returns None (filters model)
    • test_sanitize_preserves_other_fields_with_valid_pricing — Test that non-pricing fields are preserved when pricing is valid
  • TestLoadFeatherlessCatalogExport (2 tests)
    • test_load_when_no_file_exists — Test when CSV file doesn't exist
    • test_load_file_not_found — Test when file is not found
  • TestGetCachedModels (7 tests)
    • test_get_cached_models_openrouter — Test getting OpenRouter models from cache
    • test_get_cached_models_featherless — Test getting Featherless models
    • test_get_cached_models_groq — Test getting Groq models
    • test_get_cached_models_hug — Test getting Hugging Face models
    • test_get_cached_models_huggingface_alias — Test huggingface alias
    • test_get_cached_models_invalid_gateway — Test with invalid gateway name
    • test_get_cached_models_case_insensitive — Test that gateway names are case-insensitive
  • TestEnhanceModelWithProviderInfo (4 tests)
    • test_enhance_empty_model — Test enhancing empty model
    • test_enhance_model_basic — Test basic model enhancement
    • test_enhance_model_no_providers — Test when no providers available
    • test_enhance_preserves_existing_fields — Test that existing fields are preserved
  • TestEnhanceModelWithHuggingFaceData (3 tests)
    • test_enhance_with_hf_data_success — Test successful HF enhancement
    • test_enhance_with_hf_data_none — Test when HF returns None
    • test_enhance_with_hf_data_no_model — Test with empty model
  • TestGetModelCountByProvider (4 tests)
    • test_count_empty_list — Test counting with empty model list
    • test_count_no_providers — Test with no providers
    • test_count_basic — Test basic counting
    • test_count_with_none_inputs — Test with None inputs
  • TestFetchSpecificModel (4 tests)
    • test_fetch_model_not_found — Test fetching non-existent model
    • test_fetch_model_invalid_format — Test with invalid model format
    • test_fetch_model_none — Test with None input
    • test_fetch_openrouter_auto — Test fetching openrouter/auto model specifically
  • TestFalAiIntegration (7 tests)
    • test_fetch_specific_model_from_fal_success — Test successful fetch from Fal.ai cache
    • test_fetch_specific_model_from_fal_case_insensitive — Test case-insensitive matching
    • test_fetch_specific_model_from_fal_not_found — Test model not found in cache
    • test_fetch_specific_model_from_fal_empty_cache — Test with empty cache
    • test_fetch_specific_model_from_fal_exception — Test exception handling
    • test_detect_model_gateway_fal — Test that Fal models are detected in gateway detection
    • test_detect_model_gateway_fallback_to_openrouter — Test fallback to openrouter when model not found
  • TestOpenRouterIsFreeField (5 tests)
    • test_openrouter_free_model_sets_is_free_true — OpenRouter models with :free suffix should have is_free=True
    • test_openrouter_paid_model_sets_is_free_false — OpenRouter models without :free suffix should have is_free=False
    • test_openrouter_mixed_models_correct_is_free — Test that mixed models (free and paid) are correctly marked
    • test_openrouter_exacto_and_extended_suffixes_not_free — Models with :exacto or :extended suffixes should NOT be marked as free
    • test_openrouter_null_model_id_does_not_crash — Models with null id should not crash and should have is_free=False

tests/services/test_models_import_fix.py

14 tests

  • TestGetAllModelsImport (14 tests)
    • test_get_all_models_can_be_imported — Verify get_all_models can be imported from models module.
    • test_get_all_models_is_alias_of_parallel — Verify get_all_models is an alias of get_all_models_parallel.
    • test_get_all_models_returns_models_including_simplismart — Verify get_all_models returns models including SimpliSmart.
    • test_startup_can_import_get_all_models — Verify the startup module can successfully import get_all_models.
    • test_all_provider_gateways_included — Verify all key providers are supported by get_all_models_parallel.
    • test_gateway_registry_consistency — Verify all gateways from GATEWAY_REGISTRY are included in parallel fetch.
    • test_morpheus_gateway_included — Verify morpheus gateway is included in parallel fetch.
    • test_vercel_ai_gateway_included — Verify vercel-ai-gateway is included in parallel fetch.
    • test_sybil_gateway_included — Verify sybil gateway is included in parallel fetch.
    • test_near_ai_fallback_models_are_correct — Verify Near AI fallback models match the actual models available on the platform.
    • test_get_fallback_models_from_db_function_exists — Verify get_fallback_models_from_db function is available.
    • test_get_fallback_models_from_db_handles_missing_provider — Verify get_fallback_models_from_db returns None for unknown provider.
    • test_get_fallback_models_from_db_converts_models_correctly — Verify get_fallback_models_from_db converts database models to raw format.
    • test_get_fallback_models_from_db_near_pricing_format — Verify Near AI models get correct pricing format with amount and scale.

tests/services/test_morpheus_client.py

21 tests

  • TestMorpheusClient (13 tests)
    • test_morpheus_base_url — Test Morpheus base URL is correctly set
    • test_get_morpheus_client_raises_without_api_key — Test that get_morpheus_client raises error without API key
    • test_get_morpheus_client_returns_pooled_client — Test that get_morpheus_client returns pooled client
    • test_make_morpheus_request_openai — Test making a request through Morpheus
    • test_make_morpheus_request_openai_stream — Test making a streaming request through Morpheus
    • test_process_morpheus_response — Test processing Morpheus response
    • test_process_morpheus_response_with_tool_calls — Test processing Morpheus response with tool calls
    • test_fetch_models_from_morpheus_success — Test fetching models from Morpheus API
    • test_fetch_models_from_morpheus_updates_cache_on_success — Test that cache timestamp is updated after successful fetch
    • test_fetch_models_from_morpheus_updates_cache_on_http_error — Test that cache timestamp is updated even when API fails (prevents repeated calls)
    • test_fetch_models_from_morpheus_no_api_key — Test fetch_models returns empty list without API key
    • test_fetch_models_from_morpheus_updates_cache_when_no_api_key — Test that cache is updated even when API key is missing
    • test_fetch_models_from_morpheus_skips_empty_ids — Test that models with empty or missing IDs are skipped
  • TestMorpheusModelTransformations (3 tests)
    • test_morpheus_provider_detection — Test that morpheus/ prefix models are detected correctly
    • test_morpheus_prefix_stripping — Test that morpheus/ prefix is stripped during transformation
    • test_morpheus_direct_model_passthrough — Test that direct model names pass through
  • TestMorpheusCacheIntegration (3 tests)
    • test_morpheus_cache_exists — Test that Morpheus cache is defined in cache module
    • test_morpheus_cache_in_get_models_cache — Test that Morpheus is included in get_models_cache mapping
    • test_morpheus_cache_clearable — Test that Morpheus cache can be cleared
  • TestMorpheusGatewayRegistry (1 tests)
    • test_morpheus_in_gateway_registry — Test that Morpheus is in the GATEWAY_REGISTRY
  • TestMorpheusConnectionPool (1 tests)
    • test_morpheus_pooled_client_raises_without_key — Test that get_morpheus_pooled_client raises without API key

tests/services/test_multi_provider_registry.py

2 tests

  • TestMultiProviderRegistry (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/services/test_nosana_client.py

29 tests

  • TestNosanaClient (7 tests)
    • test_get_nosana_client — Test getting Nosana client
    • test_get_nosana_client_no_key — Test getting Nosana client without API key
    • test_make_nosana_request_openai — Test making request to Nosana
    • test_make_nosana_request_openai_stream — Test making streaming request to Nosana
    • test_process_nosana_response — Test processing Nosana response
    • test_process_nosana_response_no_usage — Test processing Nosana response without usage data
    • test_make_nosana_request_with_kwargs — Test making request to Nosana with additional parameters
  • TestNosanaJobDefinitionBuilders (8 tests)
    • test_build_container_job_definition_basic — Test building a basic container job definition
    • test_build_container_job_definition_with_all_options — Test building a container job definition with all options
    • test_build_llm_inference_job_definition_vllm — Test building vLLM inference job definition
    • test_build_llm_inference_job_definition_ollama — Test building Ollama inference job definition
    • test_build_llm_inference_job_definition_lmdeploy — Test building LMDeploy inference job definition
    • test_build_llm_inference_job_definition_invalid_framework — Test building LLM inference with invalid framework
    • test_build_stable_diffusion_job_definition — Test building Stable Diffusion job definition
    • test_build_whisper_job_definition — Test building Whisper job definition
  • TestNosanaDeploymentAPI (6 tests)
    • test_get_credits_balance — Test getting credit balance
    • test_list_deployments — Test listing deployments
    • test_create_deployment — Test creating a deployment
    • test_create_deployment_invalid_strategy — Test creating deployment with invalid strategy
    • test_create_deployment_invalid_timeout — Test creating deployment with invalid timeout
    • test_create_deployment_invalid_replicas — Test creating deployment with invalid replicas
  • TestNosanaJobsAPI (3 tests)
    • test_create_job — Test creating a job
    • test_create_job_invalid_timeout — Test creating job with invalid timeout
    • test_extend_job_invalid_timeout — Test extending job with invalid timeout
  • TestNosanaMarketsAPI (3 tests)
    • test_list_markets — Test listing markets
    • test_list_markets_with_filter — Test listing markets with type filter
    • test_list_markets_invalid_type — Test listing markets with invalid type
  • TestNosanaConstants (2 tests)
    • test_deployment_strategies — Test deployment strategies are defined
    • test_deployment_statuses — Test deployment statuses are defined

tests/services/test_notification.py

29 tests

  • TestUserPreferences (8 tests)
    • test_get_user_preferences_exists — Test getting existing user preferences
    • test_get_user_preferences_not_found — Test getting preferences for user without preferences
    • test_get_user_preferences_error_handling — Test error handling when getting preferences
    • test_create_user_preferences_success — Test creating default preferences
    • test_create_user_preferences_error — Test error handling when creating preferences
    • test_update_user_preferences_success — Test updating user preferences
    • test_update_user_preferences_not_found — Test updating non-existent preferences
    • test_update_user_preferences_error — Test error handling when updating preferences
  • TestLowBalanceAlerts (6 tests)
    • test_check_low_balance_alert_triggered — Test low balance alert is triggered when credits < $5
    • test_check_low_balance_alert_not_triggered — Test low balance alert not triggered when credits > $5
    • test_check_low_balance_alert_trial_user — Test low balance alert for trial user includes trial info
    • test_check_low_balance_alert_user_not_found — Test low balance alert when user not found
    • test_check_low_balance_alert_notifications_disabled — Test low balance alert when notifications are disabled
    • test_check_low_balance_alert_recent_notification — Test low balance alert skipped if recent notification sent
  • TestTrialExpiryAlerts (3 tests)
    • test_check_trial_expiry_alert_triggered — Test trial expiry alert triggered 1 day before expiry
    • test_check_trial_expiry_alert_not_trial_user — Test no alert for non-trial users
    • test_check_trial_expiry_alert_too_early — Test no alert when trial has >1 day remaining
  • TestRecentNotificationTracking (3 tests)
    • test_has_recent_notification_found — Test detecting recent notification
    • test_has_recent_notification_not_found — Test no recent notification
    • test_has_recent_notification_error — Test error handling in recent notification check
  • TestNotificationServiceInit (2 tests)
    • test_init_with_environment_variables — Test initialization with environment variables
    • test_init_with_defaults — Test initialization with default values
  • TestNotificationIntegration (1 tests)
    • test_low_balance_alert_with_plan_info — Test low balance alert includes plan information for paid users
  • TestEmailValidation (6 tests)
    • test_valid_email_addresses — Test that valid email addresses pass validation
    • test_privy_placeholder_emails_rejected — Test that Privy placeholder emails are rejected
    • test_did_prefix_emails_rejected — Test that DID-prefixed emails are rejected
    • test_malformed_emails_rejected — Test that malformed emails are rejected
    • test_send_email_skips_invalid_addresses — Test that send_email_notification skips invalid email addresses
    • test_send_email_proceeds_for_valid_addresses — Test that send_email_notification proceeds for valid email addresses

tests/services/test_novita_client.py

20 tests

  • TestGetNovitaSDKClient (3 tests)
    • test_returns_none_when_sdk_not_available — Test that function returns None when SDK is not installed
    • test_raises_error_when_api_key_missing — Test that function raises ValueError when API key is not configured
    • test_returns_client_when_sdk_available_and_key_configured — Test that function returns NovitaClient instance when everything is configured
  • TestFetchModelsFromNovita (3 tests)
    • test_returns_fallback_when_api_key_missing — Test that function returns fallback models when API key is not configured
    • test_fetches_models_from_openai_api — Test that function fetches models from OpenAI-compatible API
    • test_returns_fallback_on_api_error — Test that function returns fallback models when API call fails
  • TestFetchImageModelsFromNovitaSDK (4 tests)
    • test_returns_none_when_sdk_not_available — Test that function returns None when SDK is not installed
    • test_returns_none_when_client_initialization_fails — Test that function returns None when client initialization fails
    • test_fetches_models_using_sdk — Test that function fetches models using SDK's models_v3 method
    • test_returns_none_on_sdk_error — Test that function returns None when SDK call fails
  • TestGenerateImageWithNovitaSDK (5 tests)
    • test_raises_import_error_when_sdk_not_available — Test that function raises ImportError when SDK is not installed
    • test_raises_value_error_when_client_initialization_fails — Test that function raises ValueError when client initialization fails
    • test_generates_image_with_default_parameters — Test that function generates image with default parameters
    • test_generates_image_with_custom_parameters — Test that function generates image with custom parameters
    • test_raises_error_on_generation_failure — Test that function raises error when image generation fails
  • TestDefaultModels (2 tests)
    • test_default_models_are_defined — Test that default models list is defined
    • test_default_models_have_required_fields — Test that default models have required fields
  • TestHelperFunctions (3 tests)
    • test_cleanup_model_id_removes_prefixes — Test that _cleanup_model_id removes various prefixes
    • test_normalize_pricing_handles_none — Test that _normalize_pricing handles None input
    • test_normalize_pricing_extracts_values — Test that _normalize_pricing extracts pricing values

tests/services/test_onerouter_client.py

27 tests

  • TestGetOneRouterClient (2 tests)
    • test_get_onerouter_client_success — Test successful client initialization
    • test_get_onerouter_client_missing_key — Test client initialization with missing API key
  • TestMakeOneRouterRequest (2 tests)
    • test_make_onerouter_request_openai_forwards_args — Test that request forwards all arguments correctly
    • test_make_onerouter_request_openai_error — Test request error handling
  • TestMakeOneRouterRequestStream (2 tests)
    • test_make_onerouter_request_openai_stream — Test streaming request
    • test_make_onerouter_request_openai_stream_error — Test streaming request error handling
  • TestProcessOneRouterResponse (3 tests)
    • test_process_onerouter_response_happy — Test processing a normal response
    • test_process_onerouter_response_no_usage — Test processing response without usage data
    • test_process_onerouter_response_error — Test processing error handling
  • TestParseTokenLimit (6 tests)
    • test_parse_token_limit_int — Test parsing integer value
    • test_parse_token_limit_string — Test parsing string value
    • test_parse_token_limit_string_with_commas — Test parsing string value with commas
    • test_parse_token_limit_none — Test parsing None value
    • test_parse_token_limit_invalid_string — Test parsing invalid string values returns default
    • test_parse_token_limit_float — Test parsing float values
  • TestParsePricing (5 tests)
    • test_parse_pricing_with_dollar_sign — Test parsing pricing with $ sign
    • test_parse_pricing_without_dollar_sign — Test parsing pricing without $ sign
    • test_parse_pricing_zero — Test parsing zero pricing
    • test_parse_pricing_none — Test parsing None value
    • test_parse_pricing_with_commas — Test parsing pricing with commas
  • TestFetchModelsFromOneRouter (7 tests)
    • test_fetch_models_success_with_caching_and_pricing — Test successful model fetch with pricing enrichment from display_models
    • test_fetch_models_context_length_priority — Test that context_length is prioritized over context_window
    • test_fetch_models_skip_empty_model_id — Test that models without id are skipped
    • test_fetch_models_http_error_with_caching — Test HTTP error handling and verify cache is still updated
    • test_fetch_models_generic_error_with_caching — Test generic error handling and verify cache is updated
    • test_fetch_models_uses_correct_endpoints — Test that fetch_models_from_onerouter calls both /v1/models and display_models
    • test_fetch_models_missing_api_key — Test that fetch returns empty list when API key is not configured

tests/services/test_openai_client.py

11 tests

  • TestOpenAIClient (11 tests)
    • test_get_openai_client — Test getting OpenAI client
    • test_get_openai_client_no_key — Test getting OpenAI client without API key
    • test_make_openai_request — Test making request to OpenAI
    • test_make_openai_request_with_kwargs — Test making request to OpenAI with additional parameters
    • test_make_openai_request_error — Test handling errors from OpenAI
    • test_make_openai_request_stream — Test making streaming request to OpenAI
    • test_make_openai_request_stream_with_kwargs — Test making streaming request to OpenAI with additional parameters
    • test_make_openai_request_stream_error — Test handling streaming errors from OpenAI
    • test_process_openai_response — Test processing OpenAI response
    • test_process_openai_response_no_usage — Test processing OpenAI response without usage data
    • test_process_openai_response_multiple_choices — Test processing OpenAI response with multiple choices

tests/services/test_openrouter_client.py

19 tests

  • test_get_openrouter_client_success
  • test_get_openrouter_client_missing_key_raises
  • test_make_openrouter_request_openai_forwards_args
  • test_process_openrouter_response_happy
  • test_process_openrouter_response_no_usage
  • test_make_openrouter_request_normalizes_developer_role — Verify that make_openrouter_request_openai normalizes developer role to system.
  • test_make_openrouter_request_stream_normalizes_developer_role — Verify that make_openrouter_request_openai_stream normalizes developer role to system.
  • test_make_openrouter_request_stream_async_normalizes_developer_role — Verify that make_openrouter_request_openai_stream_async normalizes developer role to system.
  • TestMergeExtraBody (5 tests)
    • test_merge_empty_kwargs — Empty kwargs should get extra_body with provider settings.
    • test_merge_preserves_other_kwargs — Other kwargs like temperature should be preserved.
    • test_merge_preserves_existing_extra_body — Existing extra_body values should be preserved.
    • test_merge_preserves_existing_provider_settings — Existing provider settings should be preserved and merged.
    • test_user_can_override_data_collection — User-provided data_collection should override the default.
  • TestNormalizeMessageRoles (6 tests)
    • test_normalize_empty_messages — Empty messages list should return empty list.
    • test_normalize_preserves_standard_roles — Standard roles (user, assistant, system, tool) should be preserved.
    • test_normalize_transforms_developer_to_system — Developer role should be transformed to system role.
    • test_normalize_preserves_other_message_fields — Other message fields should be preserved when transforming developer role.
    • test_normalize_multiple_developer_messages — Multiple developer messages should all be transformed.
    • test_normalize_handles_non_dict_messages — Non-dict messages should be passed through unchanged.

tests/services/test_partner_trial_service.py

17 tests

  • TestPartnerTrialService (17 tests)
    • test_get_partner_config_valid_partner — Test fetching valid partner configuration
    • test_get_partner_config_case_insensitive — Test that partner code lookup is case insensitive
    • test_get_partner_config_not_found — Test fetching non-existent partner
    • test_get_partner_config_caching — Test that partner configs are cached
    • test_is_partner_code_true — Test is_partner_code returns True for valid partner
    • test_is_partner_code_false — Test is_partner_code returns False for invalid partner
    • test_is_partner_code_empty — Test is_partner_code returns False for empty string
    • test_start_partner_trial_success — Test starting a partner trial successfully
    • test_start_partner_trial_invalid_partner — Test starting trial with invalid partner code raises error
    • test_get_user_daily_limit_partner_user — Test getting daily limit for partner trial user
    • test_get_user_daily_limit_standard_user — Test getting daily limit for standard trial user
    • test_get_user_daily_limit_paid_subscriber — Test getting daily limit for paid subscriber (unlimited)
    • test_get_partner_trial_status — Test getting partner trial status
    • test_get_partner_trial_status_no_trial — Test getting status when user has no partner trial
    • test_get_partner_analytics — Test getting partner analytics
    • test_cache_invalidation — Test cache invalidation
    • test_cache_invalidation_all — Test invalidating entire cache

tests/services/test_payment_failure_downgrade.py

6 tests

  • TestPaymentFailureDowngrade (6 tests)
    • test_invoice_payment_failed_downgrades_tier — Test that invoice payment failure downgrades user to basic tier
    • test_invoice_payment_failed_for_max_tier — Test that MAX tier is also downgraded on payment failure
    • test_invoice_payment_failed_non_subscription — Test that non-subscription invoices are skipped
    • test_payment_failure_updates_both_tables — Test that payment failure updates both users and api_keys_new tables
    • test_payment_failure_logs_warning — Test that payment failure logs appropriate warning
    • test_payment_failure_handles_database_error — Test that database errors during payment failure are handled

tests/services/test_payment_processing.py

34 tests

  • TestStripeServiceInitialization (2 tests)
    • test_init_success — Test successful initialization
    • test_init_missing_api_key — Test initialization fails without API key
  • TestCheckoutSession (7 tests)
    • test_create_checkout_session_success — Test successful checkout session creation
    • test_create_checkout_session_with_discounted_credit_value — Test checkout session with discounted credit_value (e.g., $9 for $10 credits)
    • test_create_checkout_session_large_discounted_package — Test checkout session with larger discounted package ($75 for $100 credits)
    • test_create_checkout_session_persists_payment_intent — Ensure checkout session stores payment_intent when provided by Stripe.
    • test_create_checkout_session_user_not_found — Test checkout session fails for non-existent user
    • test_create_checkout_session_stripe_error — Test checkout session handles Stripe errors
    • test_create_checkout_session_with_privy_did — Test checkout session handles Privy DID emails
  • TestPaymentIntents (2 tests)
    • test_create_payment_intent_success — Test successful payment intent creation
    • test_create_payment_intent_with_specific_payment_methods — Test payment intent with specific payment methods
  • TestWebhooks (11 tests)
    • test_handle_checkout_completed_webhook — Test checkout.session.completed webhook
    • test_handle_webhook_invalid_signature — Test webhook with invalid signature
    • test_checkout_completed_adds_credits — Test checkout completed webhook adds credits to user
    • test_checkout_completed_refetches_metadata_when_missing — Ensure checkout handler refetches the session when metadata is missing
    • test_checkout_completed_recovers_metadata_from_payment_intent — Verify fallback metadata retrieval pulls from the related payment intent.
    • test_checkout_completed_hydrates_metadata_from_payment_intent — Ensure metadata can be recovered from the PaymentIntent when absent on the session.
    • test_checkout_completed_raises_when_metadata_and_id_missing — Ensure handler fails fast when both metadata and session id are missing
    • test_checkout_completed_recovers_missing_metadata — Ensure webhook handler falls back to Supabase when metadata is absent.
    • test_checkout_completed_recovers_via_session_id_when_intent_lookup_fails — Ensure fallback lookup retries with the checkout session ID.
    • test_payment_intent_succeeded_webhook — Test payment_intent.succeeded webhook
    • test_payment_intent_failed_webhook — Test payment_intent.payment_failed webhook
  • TestCreditPackages (1 tests)
    • test_get_credit_packages — Test get credit packages
  • TestRefunds (2 tests)
    • test_create_refund_success — Test successful refund creation
    • test_create_refund_stripe_error — Test refund handles Stripe errors
  • TestSessionRetrieval (2 tests)
    • test_retrieve_checkout_session — Test retrieve checkout session
    • test_retrieve_payment_intent — Test retrieve payment intent
  • TestPaymentIntegration (4 tests)
    • test_complete_payment_flow — Test complete payment flow: create session → webhook → credits added
    • test_checkout_completed_missing_credits_uses_amount_total — Ensure handler falls back to session amount when credits metadata is missing
    • test_checkout_completed_missing_ids_uses_payment_lookup — Ensure handler can recover user/payment IDs from Supabase when metadata is incomplete
    • test_checkout_completed_creates_fallback_payment_when_missing_metadata — Ensure handler creates a fallback payment record when payment_id cannot be recovered.
  • TestCheckoutCompletedSubscriptionStatus (3 tests)
    • test_checkout_completed_sets_inactive_status_for_trial_user — Test that checkout completed sets subscription_status to 'inactive' for trial users.
    • test_checkout_completed_preserves_active_subscription_status — Test that checkout completed does NOT change subscription_status for users with active subscriptions.
    • test_checkout_completed_sets_inactive_for_expired_trial_user — Test that checkout completed sets subscription_status to 'inactive' for expired trial users.

tests/services/test_payments_decimal_precision.py

6 tests

  • test_decimal_credit_calculation — Test that Decimal provides precise credit calculations.
  • test_decimal_precision_edge_cases — Test edge cases where floating-point precision matters.
  • test_decimal_preserves_precision — Test that Decimal preserves precision better than float.
  • test_credit_value_conversion_matches_schema — Test that credit value conversion matches expected schema behavior.
  • test_large_credit_values — Test that large credit values are handled correctly.
  • test_fractional_cents_handled — Test that fractional cents are handled correctly.

tests/services/test_posthog_service.py

6 tests

  • TestPosthogService (6 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports
    • test_initialize_with_exception_autocapture — Test that PostHog initializes with exception autocapture enabled
    • test_capture_exception_success — Test successful exception capture
    • test_capture_exception_default_distinct_id — Test exception capture with default distinct_id
    • test_capture_exception_when_not_initialized — Test that capture_exception handles uninitialized client gracefully

tests/services/test_pricing.py

16 tests

  • test_get_model_pricing_found_by_id
  • test_get_model_pricing_found_by_slug
  • test_get_model_pricing_model_not_found_uses_default
  • test_get_model_pricing_empty_cache_uses_default
  • test_get_model_pricing_handles_missing_prices
  • test_get_model_pricing_exception_returns_default
  • test_get_model_pricing_normalizes_hf_suffix — Test that HuggingFace :hf-inference suffix is stripped for pricing lookup
  • test_get_model_pricing_handles_multiple_provider_suffixes — Test that various provider suffixes are normalized
  • test_calculate_cost_uses_fallback_on_exception
  • test_calculate_cost_free_model_returns_zero — Test that models ending with :free return $0 cost
  • test_calculate_cost_free_model_openrouter_format — Test that OpenRouter free models (ending with :free) return $0 cost
  • test_calculate_cost_free_model_with_zero_tokens — Test that free models with zero tokens return $0
  • test_calculate_cost_non_free_model_normal_pricing — Test that non-free models are charged normally
  • test_calculate_cost_free_model_fallback_on_exception — Test that free models return $0 even in fallback/exception case
  • test_calculate_cost_free_suffix_case_sensitive — Test that :free suffix detection is case sensitive (lowercase only)
  • test_calculate_cost_multiple_free_models — Test various free model formats

tests/services/test_pricing_accuracy.py

15 tests

  • TestProviderAPIFormats (2 tests)
    • test_openrouter_api_returns_per_token_pricing — CRITICAL TEST: Verify OpenRouter returns per-token pricing, not per-1M
    • test_manual_pricing_format — Test that manual_pricing.json uses per-1M format
  • TestPricingNormalization (5 tests)
    • test_normalize_per_1m_to_per_token — Test normalization from per-1M tokens to per-token
    • test_normalize_per_1k_to_per_token — Test normalization from per-1K tokens to per-token
    • test_normalize_already_per_token — Test that per-token pricing passes through unchanged
    • test_normalize_pricing_dict — Test normalizing a full pricing dictionary
    • test_validate_normalized_price — Test price validation
  • TestProviderFormatMapping (3 tests)
    • test_openrouter_format_should_be_per_token — CRITICAL: OpenRouter format should be PER_TOKEN, not PER_1M_TOKENS
    • test_deepinfra_format_mapping — DeepInfra uses per-1M tokens pricing
    • test_aihubmix_format_mapping — AiHubMix uses per-1K tokens pricing
  • TestCreditCalculations (2 tests)
    • test_calculate_cost_with_correct_pricing — Test credit calculation with known pricing
    • test_calculate_cost_with_wrong_normalization — Test showing the impact of wrong normalization
  • TestEndToEndPricing (1 tests)
    • test_gpt4o_mini_pricing_end_to_end — Integration test: Verify GPT-4o-mini pricing end-to-end
  • TestPricingConsistency (1 tests)
    • test_openrouter_pricing_not_double_normalized — Ensure OpenRouter pricing isn't normalized twice
  • TestKnownModelPricing (1 tests)
    • test_known_model_pricing — Test that known models have correct pricing (within 20% tolerance)

tests/services/test_pricing_coverage.py

9 tests

  • test_individual_model_pricing_not_default — Per-model check: warns if an individual model falls back to default pricing.
  • TestGetPricingCoverageReport (6 tests)
    • test_empty_model_list — An empty list should return 100% coverage with zero counts.
    • test_all_models_covered — When every model has real pricing, coverage should be 100%.
    • test_no_models_covered — When no model has real pricing, coverage should be 0%.
    • test_partial_coverage — When some models are covered and some are not, report reflects both.
    • test_uncovered_models_sorted — Uncovered models list should be sorted alphabetically.
    • test_high_value_model_without_pricing_counted_as_uncovered — High-value models (GPT-4, Claude, etc.) that raise ValueError when
  • TestCatalogPricingCoverage (2 tests)
    • test_sample_catalog_coverage_report — Run the coverage report over the sample catalog and warn about gaps.
    • test_high_value_models_have_pricing — High-value models (OpenAI, Anthropic, Google) should always have

tests/services/test_pricing_database.py

12 tests

  • TestDatabasePricingLookup (3 tests)
    • test_get_pricing_from_database_success — Test successful database query returns pricing
    • test_get_pricing_from_database_not_found — Test database query returns None when model not found
    • test_get_pricing_from_database_handles_error — Test database query handles exceptions gracefully
  • TestPricingCache (4 tests)
    • test_cache_stores_and_retrieves_pricing — Test cache stores and retrieves pricing correctly
    • test_cache_expiration — Test cache expires after TTL
    • test_clear_pricing_cache — Test clearing pricing cache
    • test_get_pricing_cache_stats — Test cache statistics
  • TestFallbackMechanism (3 tests)
    • test_fallback_to_cache_on_database_failure — Test fallback to provider API cache when database fails
    • test_fallback_to_default_when_all_fail — Test fallback to default pricing when both database and cache fail
    • test_database_takes_priority_over_cache — Test database is queried before falling back to cache
  • TestModelIDNormalization (2 tests)
    • test_handles_provider_suffixes — Test stripping provider-specific suffixes
    • test_free_model_detection — Test free models return $0 cost

tests/services/test_pricing_lookup.py

71 tests

  • TestPricingLookup (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports
  • TestEnrichModelWithPricing (13 tests)
    • test_enriches_model_with_zero_pricing — Models with zero pricing should be enriched with manual pricing
    • test_skips_enrichment_for_non_zero_pricing — Models with non-zero pricing should not be enriched
    • test_handles_scientific_notation_pricing — Models with scientific notation pricing should not be enriched
    • test_handles_float_zero_pricing — Models with float 0.0 pricing should be enriched
    • test_handles_string_zero_variations — Models with various zero string formats should be enriched
    • test_handles_no_pricing_field — Models without pricing field should be enriched
    • test_handles_empty_pricing — Models with empty pricing dict should be enriched
    • test_handles_none_pricing_values — Models with None pricing values should be enriched
    • test_handles_empty_string_pricing — Models with empty string pricing values should be enriched
    • test_handles_missing_model_id — Models without id should be returned unchanged
    • test_handles_no_manual_pricing_available — Models without manual pricing should keep original pricing
    • test_handles_mixed_zero_and_nonzero_pricing — Models with some non-zero values should not be enriched
    • test_handles_invalid_pricing_values — Models with invalid pricing values should be treated as zero
  • TestGetModelPricing (6 tests)
    • test_returns_pricing_for_existing_model — Should return pricing for model in manual pricing data
    • test_returns_none_for_missing_gateway — Should return None for unknown gateway
    • test_returns_none_for_missing_model — Should return None for unknown model
    • test_case_insensitive_gateway_match — Should match gateway case-insensitively
    • test_case_insensitive_model_match — Should match model case-insensitively
    • test_returns_none_for_empty_pricing_data — Should return None when no pricing data loaded
  • TestGatewayProviders (2 tests)
    • test_gateway_providers_contains_expected_providers — Test that GATEWAY_PROVIDERS contains all expected gateway providers
    • test_gateway_providers_is_set — Test that GATEWAY_PROVIDERS is a set for O(1) lookup
  • TestCrossReferencePricing (8 tests)
    • test_cross_reference_pricing_returns_none_when_catalog_building — Test that cross-reference returns None when catalog is building
    • test_cross_reference_pricing_returns_none_when_no_openrouter_models — Test that cross-reference returns None when OpenRouter cache is empty
    • test_cross_reference_pricing_finds_matching_model — Test that cross-reference finds pricing for matching model
    • test_cross_reference_pricing_handles_provider_prefix — Test that cross-reference works with provider/model format
    • test_cross_reference_pricing_returns_none_for_unknown_model — Test that cross-reference returns None for models not in OpenRouter
    • test_cross_reference_pricing_handles_versioned_model_ids — Test that cross-reference matches versioned OpenRouter model IDs
    • test_cross_reference_pricing_does_not_match_different_model_variants — Test that cross-reference does NOT match different model variants incorrectly
    • test_cross_reference_pricing_matches_correct_model_with_variants — Test that cross-reference matches the correct model when variants exist
  • TestEnrichModelWithPricingGatewayProviders (8 tests)
    • test_gateway_provider_uses_cross_reference_pricing — Gateway provider should use cross-reference pricing when no manual pricing
    • test_gateway_provider_returns_none_when_no_pricing_and_not_building — Gateway provider should return None when no pricing found and not building catalog
    • test_gateway_provider_keeps_model_during_catalog_build — Gateway provider should keep model with zero pricing during catalog build
    • test_gateway_provider_prefers_manual_pricing — Gateway provider should prefer manual pricing over cross-reference
    • test_non_gateway_provider_returns_model_without_pricing — Non-gateway provider should return model unchanged when no pricing found
    • test_gateway_provider_with_existing_non_zero_pricing — Gateway provider with non-zero pricing should not be enriched
    • test_gateway_provider_filters_on_exception — Gateway provider should return None on exception to prevent appearing free
    • test_non_gateway_provider_returns_model_on_exception — Non-gateway provider should return model data on exception
  • TestGatewayProviderZeroPricingFiltering (22 tests)
    • test_gateway_provider_filters_zero_cross_reference_pricing — Gateway provider should filter out models with zero cross-reference pricing
    • test_gateway_provider_accepts_nonzero_cross_reference_pricing — Gateway provider should accept models with non-zero cross-reference pricing
    • test_gateway_provider_filters_zero_string_variants — Gateway provider should filter zero pricing in various formats
    • test_gateway_provider_accepts_partial_nonzero_pricing — Gateway provider should accept models with at least one non-zero prompt/completion
    • test_clarifai_in_gateway_providers — Test that clarifai is now in GATEWAY_PROVIDERS
    • test_onerouter_in_gateway_providers — Test that onerouter is now in GATEWAY_PROVIDERS
    • test_deepinfra_in_gateway_providers — Test that deepinfra is now in GATEWAY_PROVIDERS
    • test_featherless_in_gateway_providers — Test that featherless is now in GATEWAY_PROVIDERS
    • test_deepinfra_filters_models_without_pricing — Test that deepinfra filters out models without valid pricing
    • test_featherless_filters_models_without_pricing — Test that featherless filters out models without valid pricing
    • test_deepinfra_accepts_models_with_manual_pricing — Test that deepinfra accepts models with manual pricing
    • test_featherless_accepts_models_with_manual_pricing — Test that featherless accepts models with manual pricing
    • test_groq_in_gateway_providers — Test that groq is now in GATEWAY_PROVIDERS
    • test_fireworks_in_gateway_providers — Test that fireworks is now in GATEWAY_PROVIDERS
    • test_together_in_gateway_providers — Test that together is now in GATEWAY_PROVIDERS
    • test_akash_in_gateway_providers — Test that akash is now in GATEWAY_PROVIDERS
    • test_cloudflare_workers_ai_in_gateway_providers — Test that cloudflare-workers-ai is now in GATEWAY_PROVIDERS
    • test_groq_filters_models_without_pricing — Test that groq filters out models without valid pricing
    • test_fireworks_filters_models_without_pricing — Test that fireworks filters out models without valid pricing
    • test_together_filters_models_without_pricing — Test that together filters out models without valid pricing
    • test_groq_accepts_models_with_manual_pricing — Test that groq accepts models with manual pricing
    • test_fireworks_accepts_models_with_manual_pricing — Test that fireworks accepts models with manual pricing
  • TestCrossReferencePricingNullHandling (2 tests)
    • test_cross_reference_handles_none_pricing_values — Cross-reference should handle None pricing values correctly
    • test_cross_reference_handles_empty_string_pricing — Cross-reference should handle empty string pricing values correctly
  • TestIsFreeField (8 tests)
    • test_non_openrouter_gateway_sets_is_free_false — Non-OpenRouter gateways should have is_free set to False
    • test_deepinfra_sets_is_free_false — DeepInfra models should have is_free set to False
    • test_featherless_sets_is_free_false — Featherless models should have is_free set to False
    • test_together_sets_is_free_false — Together models should have is_free set to False
    • test_fireworks_sets_is_free_false — Fireworks models should have is_free set to False
    • test_openrouter_does_not_set_is_free — OpenRouter models should not have is_free set by enrich_model_with_pricing
    • test_aihubmix_sets_is_free_false — AiHubMix gateway models should have is_free set to False
    • test_helicone_sets_is_free_false — Helicone gateway models should have is_free set to False

tests/services/test_pricing_normalization.py

43 tests

  • TestNormalizeToPerToken (11 tests)
    • test_normalize_per_1m_to_per_token — Test normalization from per-1M format
    • test_normalize_per_1k_to_per_token — Test normalization from per-1K format
    • test_normalize_already_per_token — Test normalization when already per-token
    • test_normalize_negative_price — Test that negative prices (dynamic) return None
    • test_normalize_zero_price — Test that zero prices are handled
    • test_normalize_none_price — Test that None returns None
    • test_normalize_empty_string — Test that empty string returns None
    • test_normalize_string_price — Test normalization with string input
    • test_normalize_decimal_price — Test normalization with Decimal input
    • test_normalize_large_price — Test normalization with large price (expensive model)
    • test_normalize_very_small_price — Test normalization with very small price (cheap model)
  • TestNormalizePricingDict (4 tests)
    • test_normalize_full_pricing_dict — Test normalizing full pricing dictionary
    • test_normalize_partial_pricing_dict — Test normalizing dict with missing fields
    • test_normalize_empty_pricing_dict — Test normalizing empty dict
    • test_normalize_none_pricing_dict — Test normalizing None (should handle gracefully)
  • TestProviderFormats (5 tests)
    • test_get_openrouter_format — OpenRouter uses per-1M format
    • test_get_deepinfra_format — DeepInfra uses per-1M format
    • test_get_aihubmix_format — AiHubMix uses per-1K format
    • test_get_unknown_provider_format — Unknown providers default to per-1M
    • test_provider_format_case_insensitive — Provider format lookup is case-insensitive
  • TestAutoDetectFormat (4 tests)
    • test_detect_per_token — Detect per-token format (very small values)
    • test_detect_per_1k — Detect per-1K format (medium values)
    • test_detect_per_1m — Detect per-1M format (large values)
    • test_detect_boundary_values — Test detection at boundary values
  • TestConvertBetweenFormats (4 tests)
    • test_convert_1m_to_token — Convert from per-1M to per-token
    • test_convert_1k_to_token — Convert from per-1K to per-token
    • test_convert_token_to_1m — Convert from per-token to per-1M
    • test_convert_1k_to_1m — Convert from per-1K to per-1M
  • TestValidateNormalizedPrice (2 tests)
    • test_validate_correct_per_token — Valid per-token prices should pass
    • test_validate_incorrect_per_token — Prices that are too large should fail
  • TestNormalizePriceFromProvider (3 tests)
    • test_normalize_from_deepinfra — DeepInfra uses per-1M format
    • test_normalize_from_aihubmix — AiHubMix uses per-1K format
    • test_normalize_from_openrouter — OpenRouter uses per-1M format
  • TestCostCalculations (4 tests)
    • test_llama_3_1_8b_cost — Test cost calculation for Llama-3.1-8B (known pricing)
    • test_gpt4_cost — Test cost calculation for GPT-4 (known pricing)
    • test_mixed_input_output_cost — Test cost with different input/output pricing
    • test_large_request_cost — Test cost calculation for large request
  • TestEdgeCases (4 tests)
    • test_invalid_price_string — Invalid price strings should return None
    • test_scientific_notation — Test handling of scientific notation
    • test_very_large_number — Test handling of very large numbers
    • test_very_small_number — Test handling of very small numbers
  • TestRealWorldScenarios (2 tests)
    • test_typical_openrouter_model — Test typical OpenRouter model pricing
    • test_typical_aihubmix_model — Test typical AiHubMix model pricing

tests/services/test_pricing_validation.py

24 tests

  • TestGoogleModelsPricing (7 tests)
    • test_all_models_have_pricing — Test that all Google models have pricing configured.
    • test_pricing_format_not_per_million — Test that pricing is in per-1K format, not per-1M format.
    • test_google_pricing_matches_official — Test that Google models match official pricing.
    • test_gemma_models_are_free — Test that Gemma models are free.
    • test_output_price_higher_than_input — Test that output pricing is higher than input pricing.
    • test_pricing_reasonable_range — Test that pricing is in a reasonable range.
    • test_per_token_format_conversion — Test that when converted to per-token format, prices are very small.
  • TestPricingNormalization (3 tests)
    • test_per_1k_to_per_token_conversion — Test conversion from per-1K to per-token format.
    • test_per_1m_to_per_token_conversion — Test conversion from per-1M to per-token format.
    • test_google_pricing_conversion_examples — Test specific Google pricing conversion examples.
  • TestPriceBoundsValidation (5 tests)
    • test_valid_price_within_bounds — Test that valid prices pass validation
    • test_price_below_minimum_rejected — Test that prices below minimum are rejected
    • test_price_above_maximum_rejected — Test that prices above maximum are rejected
    • test_zero_price_valid_with_warning — Test that zero pricing is valid but triggers warning
    • test_unusually_low_price_warning — Test that unusually low prices trigger warnings
  • TestPriceSpikeDetection (3 tests)
    • test_small_price_change_valid — Test that small price changes are valid
    • test_large_price_spike_rejected — Test that large price spikes are rejected
    • test_spike_detection_with_zero_old_price — Test spike detection when old price is zero
  • TestComprehensivePricingValidation (3 tests)
    • test_valid_pricing_update — Test validation of a valid pricing update
    • test_pricing_update_with_bounds_violation — Test pricing update with bounds violations
    • test_pricing_update_with_spike — Test pricing update with price spike
  • TestValidationWithRealPricing (3 tests)
    • test_openai_gpt4o_pricing — Test validation with actual GPT-4o pricing
    • test_anthropic_claude_opus_pricing — Test validation with actual Claude Opus pricing
    • test_llama_8b_pricing — Test validation with actual Llama 3.1 8B pricing

tests/services/test_professional_email_templates.py

2 tests

  • TestProfessionalEmailTemplates (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/services/test_prometheus_remote_write.py

37 tests

  • TestPrometheusProtobuf (7 tests)
    • test_label_serialization — Test Label message serialization
    • test_label_empty_values — Test Label with empty values
    • test_sample_serialization — Test Sample message serialization
    • test_sample_zero_value — Test Sample with zero value but non-zero timestamp
    • test_timeseries_serialization — Test TimeSeries message serialization
    • test_write_request_serialization — Test WriteRequest message serialization
    • test_write_request_multiple_timeseries — Test WriteRequest with multiple timeseries
  • TestPrometheusRemoteWrite (24 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports
    • test_prometheus_remote_writer_initialization — Test PrometheusRemoteWriter initialization
    • test_push_metrics_when_disabled — Test push_metrics returns False when disabled
    • test_push_metrics_when_no_client — Test push_metrics returns False when client is None
    • test_push_metrics_success — Test successful metrics push with protobuf
    • test_push_metrics_http_error — Test metrics push with HTTP error
    • test_get_stats — Test get_stats returns correct statistics
    • test_get_stats_no_pushes — Test get_stats with no pushes returns 0 success rate
    • test_circuit_breaker_initial_state — Test circuit breaker is initially closed
    • test_circuit_breaker_opens_after_threshold — Test circuit breaker opens after consecutive failures
    • test_circuit_breaker_resets_on_success — Test circuit breaker resets after successful push
    • test_circuit_breaker_closes_after_timeout — Test circuit breaker allows retry after timeout
    • test_push_metrics_skipped_when_circuit_open — Test push_metrics is skipped when circuit is open
    • test_circuit_breaker_opens_on_connection_errors — Test circuit breaker opens after repeated connection errors
    • test_circuit_breaker_opens_on_timeout_errors — Test circuit breaker opens after repeated timeout errors
    • test_init_prometheus_remote_write_disabled — Test init_prometheus_remote_write when Prometheus is disabled
    • test_init_prometheus_remote_write_enabled — Test init_prometheus_remote_write creates and starts writer
    • test_shutdown_prometheus_remote_write_with_writer — Test shutdown_prometheus_remote_write with active writer
    • test_shutdown_prometheus_remote_write_no_writer — Test shutdown_prometheus_remote_write with no writer
    • test_get_prometheus_writer — Test get_prometheus_writer returns the global instance
    • test_serialize_metrics_success — Test successful metrics serialization with new protobuf implementation
    • test_serialize_metrics_empty_registry — Test serialization with empty registry
    • test_get_instance_labels — Test _get_instance_labels returns expected labels
  • TestVarintEncoding (4 tests)
    • test_encode_small_value — Test encoding small values (< 128)
    • test_encode_medium_value — Test encoding medium values (128-16383)
    • test_encode_large_value — Test encoding large values (timestamps)
    • test_encode_negative_value_raises — Test that encoding negative values raises ValueError
  • TestLabelSorting (2 tests)
    • test_labels_are_sorted — Test that labels in TimeSeries are sorted by name
    • test_serialize_metrics_sorts_labels — Test that _serialize_metrics_to_protobuf sorts labels correctly

tests/services/test_prompt_router.py

36 tests

  • TestCapabilityGating (7 tests)
    • test_extract_capabilities_basic — Test basic capability extraction from messages.
    • test_extract_capabilities_with_tools — Test capability extraction with tools.
    • test_extract_capabilities_with_json_mode — Test capability extraction with JSON response format.
    • test_extract_capabilities_with_vision — Test capability extraction with image content.
    • test_filter_by_capabilities_tools — Test filtering models by tool capability.
    • test_filter_by_capabilities_context — Test filtering models by context length.
    • test_filter_by_capabilities_cost — Test filtering models by cost limit.
  • TestPromptClassifier (9 tests)
    • test_classify_simple_question — Test classification of simple Q&A.
    • test_classify_code_generation — Test classification of code generation request.
    • test_classify_code_with_block — Test classification of message with code block.
    • test_classify_math — Test classification of math calculation.
    • test_classify_reasoning — Test classification of complex reasoning.
    • test_classify_summarization — Test classification of summarization request.
    • test_classify_translation — Test classification of translation request.
    • test_classify_multimodal_content — Test classification handles multimodal content.
    • test_classification_performance — Test classification completes within target time.
  • TestModelSelector (6 tests)
    • test_select_model_basic — Test basic model selection.
    • test_select_model_price_optimization — Test model selection with price optimization.
    • test_select_model_quality_optimization — Test model selection with quality optimization.
    • test_select_model_stable_with_conversation_id — Test that same conversation ID produces stable selection.
    • test_select_model_excludes_models — Test that excluded models are not selected.
    • test_select_model_no_candidates — Test selection with no candidates returns default.
  • TestFallbackChain (4 tests)
    • test_build_fallback_chain_basic — Test basic fallback chain building.
    • test_build_fallback_chain_provider_diversity — Test fallback chain prefers different providers.
    • test_build_fallback_chain_respects_capabilities — Test fallback chain only includes capable models.
    • test_build_fallback_chain_max_limit — Test fallback chain respects tier limit.
  • TestPromptRouter (5 tests)
    • test_router_basic_routing — Test basic routing without auto mode.
    • test_router_fail_open_on_no_candidates — Test router fails open when no candidates available.
    • test_router_timeout_protection — Test router returns default on timeout.
    • test_router_with_tools — Test routing with tools requirement.
    • test_router_with_user_preferences — Test routing with user preferences.
  • TestAutoRouteHelpers (2 tests)
    • test_is_auto_route_request — Test detection of auto-route requests.
    • test_parse_auto_route_options — Test parsing of auto-route options.
  • TestPerformanceBenchmarks (3 tests)
    • test_capability_extraction_performance — Benchmark capability extraction.
    • test_classification_performance — Benchmark prompt classification.
    • test_full_router_performance — Benchmark full router pipeline.

tests/services/test_provider_credit_monitor.py

15 tests

  • TestCreditStatusDetermination (4 tests)
    • test_healthy_balance — Test that high balance returns healthy status
    • test_info_balance — Test that moderate balance returns info status
    • test_warning_balance — Test that low balance returns warning status
    • test_critical_balance — Test that very low balance returns critical status
  • TestOpenRouterCreditCheck (6 tests)
    • test_successful_credit_check — Test successful credit balance check
    • test_credit_check_with_warning_balance — Test credit check with low balance triggering warning
    • test_credit_check_with_critical_balance — Test credit check with very low balance triggering critical
    • test_credit_check_without_api_key — Test credit check fails gracefully without API key
    • test_credit_check_http_error — Test credit check handles HTTP errors
    • test_credit_check_caching — Test that credit checks are cached
  • TestCheckAllProviderCredits (1 tests)
    • test_check_all_providers — Test checking all monitored providers
  • TestLowCreditAlerts (2 tests)
    • test_send_critical_alert — Test sending critical credit alert
    • test_send_warning_alert — Test sending warning credit alert
  • TestCreditCacheManagement (2 tests)
    • test_clear_specific_provider_cache — Test clearing cache for specific provider
    • test_clear_all_provider_cache — Test clearing all provider caches

tests/services/test_provider_failover.py

99 tests

  • TestBuildProviderFailoverChain (10 tests)
    • test_chain_with_huggingface_first — Test chain starting with huggingface
    • test_chain_with_openrouter_first — Test chain starting with openrouter
    • test_chain_with_featherless_first — Test chain starting with featherless
    • test_chain_with_none_provider — Test chain with None provider defaults to onerouter
    • test_chain_with_empty_string — Test chain with empty string defaults to onerouter
    • test_chain_with_unknown_provider — Test chain with unknown provider (not in fallback list)
    • test_chain_with_removed_provider — Test chain with removed provider (not eligible for fallback)
    • test_chain_case_insensitive — Test chain building is case insensitive
    • test_chain_no_duplicates — Test chain has no duplicate providers
    • test_fallback_provider_priority_constants — Test fallback provider constants are defined correctly
  • TestEnforceModelFailoverRules (17 tests)
    • test_openrouter_prefix_locked
    • test_openrouter_suffix_locked
    • test_openai_prefix_routes_to_native_then_openrouter — Test that openai/* models route to native OpenAI first, then OpenRouter fallback.
    • test_openai_prefix_without_native_falls_back_to_openrouter — Test that openai/* models fall back to OpenRouter if native OpenAI not in chain.
    • test_anthropic_prefix_routes_to_native_then_openrouter — Test that anthropic/* models route to native Anthropic first, then OpenRouter fallback.
    • test_anthropic_prefix_without_native_falls_back_to_openrouter — Test that anthropic/* models fall back to OpenRouter if native Anthropic not in chain.
    • test_non_locked_model_noop
    • test_payment_failover_does_not_bypass_anthropic_restriction — Test that allow_payment_failover=True does NOT bypass anthropic/ restriction.
    • test_payment_failover_does_not_bypass_openai_restriction — Test that allow_payment_failover=True does NOT bypass openai/ restriction.
    • test_payment_failover_does_not_bypass_suffix_lock — Test that allow_payment_failover=True does NOT bypass suffix-based provider lock.
    • test_payment_failover_no_effect_on_unlocked_models — Test that allow_payment_failover has no effect on models without provider lock
    • test_bare_openai_model_names_route_to_native_first — Test that bare OpenAI model names route to native OpenAI first, then OpenRouter.
    • test_bare_openai_model_names_fallback_to_openrouter — Test that bare OpenAI model names fall back to OpenRouter if native not in chain.
    • test_bare_openai_model_names_keep_restriction_with_payment_failover — Test that bare OpenAI model names keep their restriction even with payment_failover=True.
    • test_bare_anthropic_model_names_route_to_native_first — Test that bare Anthropic/Claude model names route to native Anthropic first, then OpenRouter.
    • test_bare_anthropic_model_names_fallback_to_openrouter — Test that bare Anthropic model names fall back to OpenRouter if native not in chain.
    • test_bare_anthropic_model_names_keep_restriction_with_payment_failover — Test that bare Anthropic model names keep restriction with payment_failover=True.
  • TestShouldFailover (12 tests)
    • test_should_failover_401 — Test 401 Unauthorized triggers failover
    • test_should_failover_403 — Test 403 Forbidden triggers failover
    • test_should_failover_404 — Test 404 Not Found triggers failover
    • test_should_failover_429 — Test 429 Rate Limit does NOT trigger failover - client should retry
    • test_should_failover_502 — Test 502 Bad Gateway triggers failover
    • test_should_failover_503 — Test 503 Service Unavailable triggers failover
    • test_should_failover_504 — Test 504 Gateway Timeout triggers failover
    • test_should_not_failover_200 — Test 200 OK does not trigger failover
    • test_should_not_failover_400 — Test 400 Bad Request does not trigger failover
    • test_should_not_failover_500 — Test 500 Internal Server Error does not trigger failover
    • test_failover_status_codes_constant — Test FAILOVER_STATUS_CODES contains expected codes
    • test_should_failover_402 — Test 402 Payment Required triggers failover (e.g., provider credits exhausted)
  • TestMapProviderErrorHTTPException (2 tests)
    • test_map_http_exception_passthrough — Test HTTPException is passed through unchanged
    • test_map_value_error — Test ValueError is mapped to 400
  • TestMapProviderErrorHTTPX (9 tests)
    • test_map_httpx_timeout_exception — Test httpx.TimeoutException maps to 504
    • test_map_asyncio_timeout_error — Test asyncio.TimeoutError maps to 504
    • test_map_httpx_request_error — Test httpx.RequestError maps to 503
    • test_map_httpx_status_error_429_with_retry_after — Test httpx 429 error preserves Retry-After header
    • test_map_httpx_status_error_429_without_retry_after — Test httpx 429 error without Retry-After header
    • test_map_httpx_status_error_401 — Test httpx 401 error maps to 500 (internal auth issue)
    • test_map_httpx_status_error_404 — Test httpx 404 error indicates model not found
    • test_map_httpx_status_error_4xx — Test httpx 4xx errors map to 400
    • test_map_httpx_status_error_5xx — Test httpx 5xx errors map to 502
  • TestMapProviderErrorOpenAI (13 tests)
    • test_map_api_connection_error — Test APIConnectionError maps to 503
    • test_map_api_timeout_error — Test APITimeoutError maps to 504
    • test_map_rate_limit_error_with_retry_after_header — Test RateLimitError with Retry-After in response headers
    • test_map_rate_limit_error_with_retry_after_body — Test RateLimitError with retry_after in body
    • test_map_authentication_error — Test AuthenticationError maps to 401
    • test_map_permission_denied_error — Test PermissionDeniedError maps to 401
    • test_map_not_found_error — Test NotFoundError indicates model not available
    • test_map_bad_request_error — Test BadRequestError maps to 400
    • test_map_generic_openai_error — Test generic OpenAIError maps to 502
    • test_map_api_status_error_with_custom_status — Test APIStatusError with custom status code
    • test_map_api_status_error_invalid_status — Test APIStatusError with invalid status code defaults to 500
    • test_map_api_status_error_404_generic — Test generic APIStatusError with 404 status provides proper error message
    • test_map_api_status_error_403_maps_to_401 — Test generic APIStatusError with 403 maps to 401 for consistency
  • TestMapProviderErrorCerebras (12 tests)
    • test_map_cerebras_api_connection_error — Test Cerebras APIConnectionError maps to 503
    • test_map_cerebras_rate_limit_error_with_retry_after_header — Test Cerebras RateLimitError with Retry-After in response headers
    • test_map_cerebras_rate_limit_error_with_retry_after_body — Test Cerebras RateLimitError with retry_after in body
    • test_map_cerebras_authentication_error — Test Cerebras AuthenticationError maps to 401
    • test_map_cerebras_permission_denied_error — Test Cerebras PermissionDeniedError maps to 401
    • test_map_cerebras_not_found_error — Test Cerebras NotFoundError indicates model not available
    • test_map_cerebras_bad_request_error — Test Cerebras BadRequestError maps to 400
    • test_map_cerebras_api_status_error_with_custom_status — Test Cerebras APIStatusError with custom status code
    • test_map_cerebras_api_status_error_invalid_status — Test Cerebras APIStatusError with invalid status code defaults to 500
    • test_map_cerebras_api_status_error_5xx — Test Cerebras APIStatusError with 5xx status maps to service error
    • test_map_cerebras_api_status_error_403_generic — Test generic Cerebras APIStatusError with 403 maps to 401
    • test_map_cerebras_api_status_error_404_generic — Test generic Cerebras APIStatusError with 404 provides proper error message
  • TestMapProviderErrorCerebras (12 tests)
    • test_map_cerebras_api_connection_error — Test Cerebras APIConnectionError maps to 503
    • test_map_cerebras_rate_limit_error_with_retry_after_header — Test Cerebras RateLimitError with Retry-After in response headers
    • test_map_cerebras_rate_limit_error_with_retry_after_body — Test Cerebras RateLimitError with retry_after in body
    • test_map_cerebras_authentication_error — Test Cerebras AuthenticationError maps to 401
    • test_map_cerebras_permission_denied_error — Test Cerebras PermissionDeniedError maps to 401
    • test_map_cerebras_not_found_error — Test Cerebras NotFoundError indicates model not available
    • test_map_cerebras_bad_request_error — Test Cerebras BadRequestError maps to 400
    • test_map_cerebras_api_status_error_with_custom_status — Test Cerebras APIStatusError with custom status code
    • test_map_cerebras_api_status_error_invalid_status — Test Cerebras APIStatusError with invalid status code defaults to 500
    • test_map_cerebras_api_status_error_5xx — Test Cerebras APIStatusError with 5xx status maps to service error
    • test_map_cerebras_api_status_error_403_generic — Test generic Cerebras APIStatusError with 403 maps to 401
    • test_map_cerebras_api_status_error_404_generic — Test generic Cerebras APIStatusError with 404 provides proper error message
  • TestMapProviderErrorGeneric (3 tests)
    • test_map_generic_exception — Test generic Exception maps to 502
    • test_map_runtime_error — Test RuntimeError maps to 502
    • test_map_key_error — Test KeyError maps to 502
  • TestProviderFailoverIntegration (3 tests)
    • test_complete_failover_chain_all_providers — Test complete failover chain with all providers
    • test_failover_decision_matrix — Test failover decisions for various error scenarios
    • test_error_mapping_preserves_provider_context — Test error messages include provider and model context
  • TestMapProviderErrorNoCandidates (6 tests)
    • test_no_candidates_error_triggers_failover — Test 'no candidates' ValueError maps to 503 for failover
    • test_block_reason_error_triggers_failover — Test that block reason errors map to 503
    • test_prompt_feedback_error_triggers_failover — Test that promptFeedback errors map to 503
    • test_regular_value_error_does_not_trigger_failover — Test that regular ValueErrors map to 400 (no failover)
    • test_503_is_in_failover_status_codes — Verify that 503 is in the set of status codes that trigger failover
    • test_no_candidates_from_other_provider — Test 'no candidates' detection works for any provider, not just google-vertex

tests/services/test_provider_selector.py

2 tests

  • TestProviderSelector (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/services/test_providers.py

2 tests

  • TestProviders (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/services/test_query_classifier.py

47 tests

  • TestNormalizeText (3 tests)
    • test_lowercase — Test that text is lowercased.
    • test_strip_whitespace — Test that whitespace is stripped.
    • test_combined — Test combined normalization.
  • TestContainsKeywords (4 tests)
    • test_single_keyword_found — Test detection of single keyword.
    • test_multiple_keywords_found — Test detection of multiple keywords.
    • test_no_keywords_found — Test when no keywords are found.
    • test_case_insensitive — Test case-insensitive matching.
  • TestMatchesPatterns (5 tests)
    • test_factual_question_pattern — Test factual question pattern detection.
    • test_what_is_the_best_pattern — Test 'what is the best' pattern.
    • test_code_pattern_function — Test code pattern detection for functions.
    • test_code_pattern_import — Test code pattern detection for imports.
    • test_no_pattern_match — Test when no patterns match.
  • TestIsCodeQuery (5 tests)
    • test_code_block — Test detection of code block.
    • test_python_function — Test detection of Python function.
    • test_javascript_function — Test detection of JavaScript function.
    • test_import_statement — Test detection of import statement.
    • test_not_code — Test non-code query.
  • TestExtractUserQuery (5 tests)
    • test_single_user_message — Test extraction from single user message.
    • test_multiple_messages — Test extraction from conversation with multiple messages.
    • test_multimodal_content — Test extraction from multimodal content.
    • test_no_user_message — Test when there's no user message.
    • test_empty_messages — Test with empty messages list.
  • TestClassifyQuery (12 tests)
    • test_wifi_travel_query_el_salvador — Test classification of wifi/travel query for El Salvador.
    • test_wifi_travel_query_thailand — Test classification of wifi/travel query for Thailand.
    • test_cost_of_living_query — Test classification of cost of living query.
    • test_current_events_query — Test classification of current events query.
    • test_price_query — Test classification of price query.
    • test_weather_query — Test classification of weather query.
    • test_code_query_no_search — Test that code queries don't trigger search.
    • test_import_statement_no_search — Test that import statements don't trigger search.
    • test_simple_greeting_no_search — Test that simple greetings don't trigger search.
    • test_general_knowledge_low_confidence — Test general knowledge questions have lower confidence.
    • test_empty_messages — Test handling of empty messages.
    • test_threshold_adjustment — Test that threshold affects search decision.
  • TestShouldAutoSearch (3 tests)
    • test_enabled_returns_result — Test that enabled=True processes the query.
    • test_disabled_returns_false — Test that enabled=False always returns False.
    • test_remote_work_query — Test remote work related query triggers search.
  • TestRealWorldQueries (1 tests)
    • test_query_classification — Test classification of various real-world queries.
  • TestKeywordSets (4 tests)
    • test_current_info_keywords_not_empty — Test that current info keywords set is not empty.
    • test_location_keywords_not_empty — Test that location keywords set is not empty.
    • test_travel_destinations_not_empty — Test that travel destinations set is not empty.
    • test_all_keywords_lowercase — Test that all keywords are lowercase.
  • TestEdgeCases (5 tests)
    • test_very_long_query — Test handling of very long queries.
    • test_unicode_characters — Test handling of unicode characters.
    • test_mixed_content_messages — Test handling of mixed content in conversation.
    • test_none_content — Test handling of None content.
    • test_empty_string_content — Test handling of empty string content.

tests/services/test_rate_limiting.py

18 tests

  • test_check_rate_limit_happy_path_local
  • test_concurrency_limit_exceeded — Test concurrency limit - currently disabled in source code (see rate_limiting.py lines 94-104)
  • test_burst_limit_local
  • test_minute_request_limit_local
  • test_minute_token_limit_local
  • test_manager_get_key_config_from_db
  • test_manager_get_key_config_default_on_error
  • test_manager_update_key_config_calls_db
  • test_get_rate_limiter_singleton
  • test_get_rate_limit_manager_singleton
  • test_concurrency_increment_decrement_helpers
  • test_severe_rate_limit_configs_exist — Test that SEVERE_RATE_LIMIT_CONFIG and BLOCKED_ACCOUNT_CONFIG exist and have correct values
  • test_severe_rate_limit_for_temporary_email — Test that temporary email domains get SEVERE rate limiting
  • test_blocked_rate_limit_for_blocked_email_domain — Test that blocked email domains get BLOCKED rate limiting (most restrictive)
  • test_no_severe_rate_limit_for_normal_user — Test that normal users don't get severe rate limiting
  • test_severe_rate_limit_graceful_on_missing_user — Test that severe rate limit check handles missing user gracefully
  • test_severe_rate_limit_graceful_on_exception — Test that severe rate limit check handles exceptions gracefully
  • test_blocked_email_domain_check_priority — Test that blocked domains take priority over temporary domains

tests/services/test_rebuild_full_catalog.py

10 tests

  • TestRebuildFullCatalogFromProviders (8 tests)
    • test_assembles_all_providers — All provider models are merged into a single catalog.
    • test_caches_result_in_redis_and_local — Assembled catalog is cached in both Redis and local memory.
    • test_partial_provider_failure — If some providers fail, others still contribute to the catalog.
    • test_all_providers_empty_returns_empty — If every provider returns empty, result is empty and nothing is cached.
    • test_empty_provider_list_returns_empty — If no provider slugs are discovered, return empty immediately.
    • test_db_discovery_failure_falls_back_to_registry — If DB slug query fails, falls back to GATEWAY_REGISTRY keys.
    • test_providers_with_zero_models_handled_gracefully — Providers returning 0 models don't break the assembly.
    • test_does_not_use_provider_slugs_with_overrides — Fallback uses GATEWAY_REGISTRY.keys() not PROVIDER_SLUGS
  • TestRebuildIntegrationWithGetCachedFullCatalog (2 tests)
    • test_cache_hit_does_not_trigger_rebuild — Redis cache hit returns immediately without rebuilding.
    • test_cache_miss_triggers_rebuild — Cache miss triggers rebuild_full_catalog_from_providers.

tests/services/test_redis_integration_basic.py

16 tests

  • test_redis_availability_check — Test that we can check Redis availability
  • TestAuthCacheBasic (4 tests)
    • test_cache_and_retrieve_user_by_api_key — Test caching user data by API key
    • test_cache_miss — Test cache miss returns None
    • test_cache_invalidation — Test cache invalidation works
    • test_cache_user_by_id — Test caching user by ID
  • TestDBCacheBasic (4 tests)
    • test_db_cache_set_and_get — Test basic cache set and get operations
    • test_db_cache_invalidate — Test cache invalidation
    • test_convenience_functions — Test convenience functions work
    • test_cache_stats — Test cache statistics work
  • TestModelCatalogCache (3 tests)
    • test_cache_full_catalog — Test caching full model catalog
    • test_cache_provider_catalog — Test caching provider-specific catalog
    • test_invalidate_full_catalog — Test invalidating full catalog
  • TestCacheWithRedisUnavailable (2 tests)
    • test_cache_fails_gracefully_when_redis_unavailable — Test that cache operations don't crash when Redis is down
    • test_db_cache_fails_gracefully — Test DB cache fails gracefully without Redis
  • TestCacheIntegrationWithDB (2 tests)
    • test_user_caching_integration — Test full user caching flow with DB
    • test_cache_invalidation_on_credit_update — Test cache invalidates when credits are updated

tests/services/test_redis_metrics.py

20 tests

  • TestRecordRequest (4 tests)
    • test_record_successful_request — Test recording a successful request
    • test_record_failed_request — Test recording a failed request with error
    • test_record_request_disabled — Test recording when Redis is disabled
    • test_record_request_exception_handling — Test exception handling during recording
  • TestProviderHealth (3 tests)
    • test_get_provider_health — Test getting provider health score
    • test_get_provider_health_no_data — Test getting health score when no data exists
    • test_get_all_provider_health — Test getting all provider health scores
  • TestRecentErrors (2 tests)
    • test_get_recent_errors — Test getting recent errors
    • test_get_recent_errors_disabled — Test getting errors when Redis is disabled
  • TestHourlyStats (2 tests)
    • test_get_hourly_stats — Test getting hourly statistics
    • test_get_hourly_stats_disabled — Test getting stats when Redis is disabled
  • TestLatencyPercentiles (3 tests)
    • test_get_latency_percentiles — Test calculating latency percentiles
    • test_get_latency_percentiles_no_data — Test percentiles when no data exists
    • test_get_latency_percentiles_disabled — Test percentiles when Redis is disabled
  • TestCircuitBreaker (2 tests)
    • test_update_circuit_breaker — Test updating circuit breaker state
    • test_update_circuit_breaker_disabled — Test updating circuit breaker when Redis is disabled
  • TestCleanup (2 tests)
    • test_cleanup_old_data — Test cleaning up old Redis data
    • test_cleanup_old_data_disabled — Test cleanup when Redis is disabled
  • TestRequestMetricsDataclass (2 tests)
    • test_request_metrics_creation — Test creating RequestMetrics instance
    • test_request_metrics_with_error — Test creating RequestMetrics with error

tests/services/test_request_prioritization.py

23 tests

  • TestRequestPrioritization (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports
  • TestLowLatencyModels (7 tests)
    • test_ultra_low_latency_models_defined — Test that ultra-low-latency models are defined
    • test_low_latency_models_defined — Test that low-latency models are defined
    • test_ultra_low_latency_subset_of_low_latency — Test that ultra-low-latency models are included in low-latency set
    • test_is_low_latency_model — Test is_low_latency_model function
    • test_is_ultra_low_latency_model — Test is_ultra_low_latency_model function
    • test_get_low_latency_models — Test get_low_latency_models returns sorted list
    • test_get_ultra_low_latency_models — Test get_ultra_low_latency_models returns sorted list
  • TestProviderLatencyTiers (6 tests)
    • test_provider_latency_tiers_defined — Test that provider latency tiers are defined
    • test_groq_is_tier_1 — Test that Groq is classified as tier 1 (fastest)
    • test_cerebras_is_tier_1 — Test that Cerebras is classified as tier 1 (fastest)
    • test_fireworks_is_tier_2 — Test that Fireworks is classified as tier 2
    • test_get_provider_latency_tier — Test get_provider_latency_tier function
    • test_get_fastest_providers — Test get_fastest_providers returns sorted list
  • TestLowLatencyAlternatives (4 tests)
    • test_suggest_low_latency_alternative_for_claude — Test suggesting alternatives for Claude models
    • test_suggest_low_latency_alternative_for_gpt — Test suggesting alternatives for GPT models
    • test_suggest_low_latency_alternative_for_reasoning_model — Test suggesting alternatives for reasoning models
    • test_suggest_low_latency_alternative_for_unknown — Test suggesting alternatives for unknown models
  • TestProviderSelection (4 tests)
    • test_high_priority_gets_fastest_first — Test that high priority requests get fastest providers first
    • test_low_priority_gets_slower_first — Test that low priority requests get slower providers first
    • test_all_providers_included — Test that all available providers are included in result
    • test_empty_providers_list — Test handling of empty providers list

tests/services/test_response_cache.py

29 tests

  • TestCacheKeyGeneration (4 tests)
    • test_generate_cache_key_from_request — Generate cache key from request parameters
    • test_same_request_same_key — Same request should generate same cache key
    • test_different_request_different_key — Different requests should generate different keys
    • test_cache_key_ignores_irrelevant_params — Cache key should ignore irrelevant parameters
  • TestCacheOperations (3 tests)
    • test_cache_miss — Cache miss returns None
    • test_cache_hit — Cache hit returns cached response
    • test_cache_set_and_get — Set and get cache entry
  • TestCacheTTL (3 tests)
    • test_cache_expires_after_ttl — Cache entry should expire after TTL
    • test_cache_valid_within_ttl — Cache entry should be valid within TTL
    • test_custom_ttl — Support custom TTL values
  • TestCacheInvalidation (2 tests)
    • test_invalidate_single_entry — Invalidate single cache entry
    • test_clear_all_cache — Clear all cache entries
  • TestCachingStrategies (5 tests)
    • test_streaming_responses_not_cached — Streaming responses should not be cached
    • test_error_responses_not_cached — Error responses should not be cached
    • test_successful_responses_cached — Successful responses should be cached
    • test_cache_respects_max_tokens — Different max_tokens should create different cache keys
    • test_cache_respects_temperature — Different temperature should create different cache keys
  • TestCachePerformance (3 tests)
    • test_cache_reduces_latency — Cache hit should be faster than API call
    • test_cache_size_limited — Cache size should be limited to prevent memory issues
    • test_cache_eviction_lru — Least Recently Used (LRU) eviction strategy
  • TestCacheMonitoring (3 tests)
    • test_cache_hit_ratio_calculation — Calculate cache hit ratio
    • test_cache_memory_usage — Track cache memory usage
    • test_cache_metrics_exposed — Cache metrics should be exposed via API
  • TestCacheEdgeCases (3 tests)
    • test_cache_null_response — Cache should handle null responses
    • test_cache_large_response — Cache should handle large responses
    • test_cache_key_collision_handling — Handle potential cache key collisions
  • TestCacheConfiguration (3 tests)
    • test_default_ttl_configuration — Default TTL should be configurable
    • test_max_cache_size_configuration — Max cache size should be configurable
    • test_cache_enabled_configuration — Cache should be enable/disable via configuration

tests/services/test_roles.py

23 tests

  • TestRequireRole (11 tests)
    • test_require_role_admin_success — Test admin accessing admin resource
    • test_require_role_developer_success — Test developer accessing developer resource
    • test_require_role_user_success — Test user accessing user resource
    • test_require_role_hierarchy_admin_to_dev — Test admin can access developer resources
    • test_require_role_hierarchy_admin_to_user — Test admin can access user resources
    • test_require_role_hierarchy_dev_to_user — Test developer can access user resources
    • test_require_role_user_cannot_access_dev — Test user cannot access developer resources
    • test_require_role_user_cannot_access_admin — Test user cannot access admin resources
    • test_require_role_dev_cannot_access_admin — Test developer cannot access admin resources
    • test_require_role_default_to_user — Test missing role defaults to user level
    • test_require_role_unknown_role — Test unknown role is rejected
  • TestRequireAdmin (3 tests)
    • test_require_admin_success — Test admin can access admin-only resource
    • test_require_admin_developer_rejected — Test developer cannot access admin-only resource
    • test_require_admin_user_rejected — Test user cannot access admin-only resource
  • TestRequireDeveloper (3 tests)
    • test_require_developer_success — Test developer can access developer resource
    • test_require_developer_admin_success — Test admin can access developer resource
    • test_require_developer_user_rejected — Test user cannot access developer resource
  • TestRequirePermission (3 tests)
    • test_require_permission_granted — Test permission granted
    • test_require_permission_denied — Test permission denied
    • test_require_permission_different_resources — Test permission checking for different resources
  • TestCreatePermissionChecker (3 tests)
    • test_create_permission_checker_success — Test permission checker factory
    • test_create_permission_checker_denied — Test permission checker factory with denied permission
    • test_create_permission_checker_multiple_resources — Test creating multiple permission checkers

tests/services/test_simplismart_client.py

49 tests

  • TestSimplismartClient (6 tests)
    • test_get_simplismart_client — Test getting Simplismart client
    • test_get_simplismart_client_no_key — Test getting Simplismart client without API key
    • test_make_simplismart_request_openai — Test making request to Simplismart
    • test_make_simplismart_request_openai_stream — Test making streaming request to Simplismart
    • test_process_simplismart_response — Test processing Simplismart response
    • test_process_simplismart_response_no_usage — Test processing response when usage is not available
  • TestSimplismartModelResolution (11 tests)
    • test_resolve_alias_llama_3_1_8b — Test resolving llama-3.1-8b alias
    • test_resolve_alias_llama_3_1_70b — Test resolving llama-3.1-70b alias
    • test_resolve_alias_llama_3_3_70b — Test resolving llama-3.3-70b alias
    • test_resolve_alias_gemma — Test resolving gemma aliases
    • test_resolve_alias_qwen — Test resolving qwen aliases
    • test_resolve_alias_deepseek — Test resolving deepseek aliases
    • test_resolve_alias_mixtral — Test resolving mixtral aliases
    • test_resolve_alias_devstral — Test resolving devstral aliases
    • test_resolve_full_model_id — Test that full model IDs are returned as-is
    • test_resolve_unknown_model — Test that unknown models are returned as-is
    • test_resolve_case_insensitive — Test that alias resolution is case insensitive
  • TestSimplismartModelCatalog (5 tests)
    • test_fetch_models_from_simplismart — Test fetching models from Simplismart catalog
    • test_fetch_models_has_llama_models — Test that catalog includes Llama models
    • test_fetch_models_has_gemma_models — Test that catalog includes Gemma models
    • test_fetch_models_has_qwen_models — Test that catalog includes Qwen models
    • test_fetch_models_has_deepseek_models — Test that catalog includes DeepSeek models
  • TestIsSimplismartModel (4 tests)
    • test_is_simplismart_model_with_alias — Test checking models with aliases
    • test_is_simplismart_model_with_full_id — Test checking models with full IDs
    • test_is_simplismart_model_case_insensitive — Test case insensitivity
    • test_is_simplismart_model_unknown — Test that unknown models return False
  • TestSimplismartConstants (6 tests)
    • test_base_url — Test that base URL is correct
    • test_model_catalog_not_empty — Test that model catalog is not empty
    • test_model_aliases_not_empty — Test that model aliases are not empty
    • test_all_aliases_resolve_to_catalog_models — Test that all aliases resolve to models in the catalog
    • test_models_have_required_metadata — Test that all models have required metadata
    • test_models_have_pricing — Test that all models have pricing information
  • TestSimplismartPricing (4 tests)
    • test_fetch_models_includes_pricing — Test that fetched models include pricing data
    • test_pricing_values_match_source — Test specific pricing values from https://simplismart.ai/pricing
    • test_new_models_present — Test that new models from pricing page are present
    • test_new_aliases_work — Test that new model aliases resolve correctly
  • TestSimplismartImageModels (6 tests)
    • test_fetch_models_has_flux_models — Test that catalog includes Flux image models
    • test_fetch_models_has_sdxl — Test that catalog includes SDXL
    • test_image_models_have_type — Test that image models have correct type
    • test_image_models_have_per_image_pricing — Test that image models have per_image pricing model
    • test_image_model_pricing_values — Test specific image model pricing from simplismart.ai/pricing
    • test_flux_aliases_resolve_correctly — Test Flux image model aliases
  • TestSimplismartSpeechModels (5 tests)
    • test_fetch_models_has_whisper_models — Test that catalog includes Whisper models
    • test_whisper_models_have_type — Test that Whisper models have correct type
    • test_whisper_models_have_per_minute_pricing — Test that Whisper models have per_minute pricing model
    • test_whisper_model_pricing_values — Test specific Whisper model pricing from simplismart.ai/pricing
    • test_whisper_aliases_resolve_correctly — Test Whisper model aliases
  • TestSimplismartIsModelWithNewTypes (2 tests)
    • test_is_simplismart_model_flux — Test checking Flux image models
    • test_is_simplismart_model_whisper — Test checking Whisper speech models

tests/services/test_startup.py

9 tests

  • TestStartup (9 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports
    • test_lifespan_startup_success — Test successful lifespan startup
    • test_lifespan_startup_critical_env_vars_missing — Test lifespan startup fails with missing critical env vars
    • test_lifespan_startup_supabase_init_fails — Test lifespan startup continues in degraded mode when Supabase init fails
    • test_lifespan_startup_continues_if_fal_cache_fails — Test lifespan startup continues even if optional Fal cache fails
    • test_initialize_services — Test initialize_services function
    • test_shutdown_services — Test shutdown_services function
    • test_shutdown_services_handles_failure — Test shutdown_services handles failures gracefully

tests/services/test_statsig_service.py

15 tests

  • TestStatsigService (3 tests)
    • test_log_event_returns_true_in_fallback_mode — Test log_event returns True in fallback mode (no server key)
    • test_flush_returns_true_when_not_enabled — Test flush returns True when service is not enabled
    • test_get_feature_flag_returns_default_when_not_enabled — Test get_feature_flag returns default value when not enabled
  • TestStatsigServiceInitialization (2 tests)
    • test_initialize_without_server_key — Test initialization gracefully handles missing server key
    • test_initialize_with_missing_sdk — Test initialization handles missing statsig_python_core package
  • TestStatsigServiceShutdown (2 tests)
    • test_shutdown_when_not_enabled — Test shutdown works when service is not enabled
    • test_shutdown_calls_wait — Test shutdown calls .wait() on the statsig SDK
  • TestStatsigServiceFlush (3 tests)
    • test_flush_when_not_enabled — Test flush returns True when service is not enabled
    • test_flush_calls_wait_when_flush_exists — Test flush calls .wait() on the SDK flush method
    • test_flush_handles_exception_gracefully — Test flush returns False when exception occurs
  • TestStatsigServiceBatchingConfig (1 tests)
    • test_initialization_sets_batching_options — Test that initialization configures event batching options
  • TestStatsigServiceSessionTracking (4 tests)
    • test_log_session_start_in_fallback_mode — Test log_session_start returns True in fallback mode
    • test_log_session_start_includes_platform — Test log_session_start includes platform in metadata
    • test_log_session_end_in_fallback_mode — Test log_session_end returns True in fallback mode
    • test_log_session_end_includes_duration — Test log_session_end includes duration in metadata

tests/services/test_stream_normalizer.py

20 tests

  • TestStreamNormalizer (20 tests)
    • test_initialization
    • test_openai_format
    • test_gemini_format
    • test_anthropic_format
    • test_reasoning_extraction
    • test_reasoning_details_array_extraction — Test Grok 4.1 reasoning_details array format
    • test_reasoning_details_string_array_extraction — Test reasoning_details as array of strings
    • test_reasoning_details_text_field — Test reasoning_details with 'text' field instead of 'content'
    • test_anthropic_thinking_delta — Test Anthropic extended thinking delta events (Claude Sonnet 4, etc.)
    • test_anthropic_signature_delta — Test Anthropic signature delta events for extended thinking verification
    • test_anthropic_content_block_start_thinking — Test Anthropic content_block_start event with thinking type
    • test_anthropic_content_block_start_text — Test Anthropic content_block_start event with text type
    • test_anthropic_message_stop — Test Anthropic message_stop event
    • test_anthropic_tool_input_delta — Test Anthropic input_json_delta for tool use streaming
    • test_anthropic_ping_ignored — Test that Anthropic ping events are ignored
    • test_anthropic_message_start_ignored — Test that Anthropic message_start events are ignored
    • test_anthropic_content_block_stop_ignored — Test that Anthropic content_block_stop events are ignored
    • test_anthropic_full_extended_thinking_flow — Test a complete extended thinking flow with multiple events
    • test_finish_reason_normalization
    • test_object_chunk

tests/services/test_subscription_management.py

25 tests

  • TestGetCurrentSubscription (3 tests)
    • test_get_subscription_with_active_subscription — Test getting subscription for user with active subscription
    • test_get_subscription_without_subscription — Test getting subscription for user without subscription
    • test_get_subscription_user_not_found — Test getting subscription for non-existent user
  • TestUpgradeSubscription (7 tests)
    • test_upgrade_invalid_product_id_returns_basic — Test upgrade fails when product ID resolves to basic tier
    • test_upgrade_invalid_product_id_returns_none — Test upgrade fails when product ID resolves to None
    • test_upgrade_allowance_reset_failure — Test upgrade fails when allowance reset fails
    • test_upgrade_logs_credit_transaction — Test that upgrade logs audit trail via credit transaction
    • test_upgrade_pro_to_max — Test upgrading from Pro to Max tier
    • test_upgrade_without_subscription — Test upgrading when user has no subscription
    • test_upgrade_inactive_subscription — Test upgrading when subscription is not active
  • TestDowngradeSubscription (6 tests)
    • test_downgrade_invalid_product_id_returns_basic — Test downgrade fails when product ID resolves to basic tier
    • test_downgrade_invalid_product_id_returns_none — Test downgrade fails when product ID resolves to None
    • test_downgrade_allowance_reset_failure — Test downgrade fails when allowance reset fails
    • test_downgrade_logs_credit_transaction — Test that downgrade logs audit trail via credit transaction
    • test_downgrade_max_to_pro — Test downgrading from Max to Pro tier
    • test_downgrade_without_subscription — Test downgrading when user has no subscription
  • TestCancelSubscription (4 tests)
    • test_cancel_at_period_end — Test canceling subscription at end of billing period
    • test_cancel_immediately — Test canceling subscription immediately
    • test_cancel_without_subscription — Test canceling when user has no subscription
    • test_cancel_already_canceled_subscription — Test canceling already canceled subscription
  • TestTransactionTypes (2 tests)
    • test_subscription_upgrade_transaction_type_exists — Test that SUBSCRIPTION_UPGRADE transaction type exists
    • test_subscription_downgrade_transaction_type_exists — Test that SUBSCRIPTION_DOWNGRADE transaction type exists
  • TestSubscriptionManagementSchemas (3 tests)
    • test_upgrade_request_validation — Test UpgradeSubscriptionRequest validation
    • test_downgrade_request_validation — Test DowngradeSubscriptionRequest validation
    • test_cancel_request_defaults — Test CancelSubscriptionRequest defaults

tests/services/test_subscription_usage_credits.py

19 tests

  • TestSubscriptionUsageCreditsAllocation (8 tests)
    • test_pro_subscription_receives_20_dollar_credits — Test that PRO subscription gets $20 credits per month
    • test_max_subscription_receives_150_dollar_credits — Test that MAX subscription gets $150 credits per month
    • test_pro_subscription_initial_payment — Test PRO subscription initial payment adds credits immediately
    • test_max_subscription_initial_payment — Test MAX subscription initial payment adds credits immediately
    • test_pro_subscription_monthly_renewal — Test PRO subscription renewal adds $20 credits each month
    • test_max_subscription_monthly_renewal — Test MAX subscription renewal adds $150 credits each month
    • test_non_subscription_invoice_skipped — Test that non-subscription invoices don't add credits
    • test_unknown_tier_no_credits — Test that unknown tiers don't receive credits
  • TestSubscriptionCreditsIntegration (3 tests)
    • test_pro_subscription_created_then_first_invoice_paid — Test complete flow: subscription created -> first invoice paid -> credits added
    • test_max_subscription_created_then_first_invoice_paid — Test complete flow: MAX subscription created -> first invoice paid -> credits added
    • test_multiple_users_different_tiers — Test multiple users with different subscription tiers receive correct credits
  • TestSubscriptionCreditsEdgeCases (5 tests)
    • test_missing_user_id_in_subscription_metadata — Test handling when user_id is missing from subscription metadata
    • test_tier_defaults_to_pro_when_missing — Test that tier defaults to 'pro' when not specified in metadata
    • test_case_sensitivity_of_tier — Test that tier matching is case-sensitive (lowercase expected)
    • test_subscription_retrieve_fails — Test handling when Stripe API fails to retrieve subscription
    • test_add_credits_fails_gracefully — Test that errors in add_credits_to_user are propagated
  • TestTrialStatusClearedOnSubscription (3 tests)
    • test_subscription_created_clears_trial_status — Test that creating a subscription clears is_trial flag on API keys
    • test_subscription_updated_to_active_clears_trial_status — Test that updating subscription to active clears is_trial flag
    • test_subscription_updated_to_canceled_does_not_clear_trial — Test that canceling subscription does not clear trial status

tests/services/test_subscription_webhook_handlers.py

27 tests

  • TestSubscriptionCreatedWebhook (3 tests)
    • test_subscription_created_creates_user_plan_entry — Test that subscription.created webhook creates user_plans entry
    • test_subscription_created_deactivates_old_plans — Test that subscription.created deactivates existing user plans
    • test_subscription_created_handles_missing_plan — Test that subscription.created handles tier with no matching plan gracefully
  • TestSubscriptionUpdatedWebhook (3 tests)
    • test_subscription_updated_active_creates_plan_entry — Test that subscription.updated with status=active creates/updates user_plans
    • test_subscription_updated_past_due_does_not_create_plan — Test that subscription.updated with status=past_due doesn't create plan and downgrades tier
    • test_subscription_updated_tier_change_pro_to_max — Test upgrading from Pro to Max tier
  • TestTierResolutionFromSubscriptionItems (6 tests)
    • test_resolve_tier_from_subscription_metadata_present — Test that tier from metadata is used when present
    • test_resolve_tier_from_subscription_metadata_missing_defaults_to_pro_for_active — Test that active subscriptions default to 'pro' when tier can't be determined
    • test_resolve_tier_from_subscription_metadata_basic_defaults_to_pro_for_active — Test that 'basic' metadata is upgraded to 'pro' for active subscriptions
    • test_resolve_tier_from_subscription_items_with_product_lookup — Test tier resolution from subscription items when product_id is mapped
    • test_subscription_created_with_missing_tier_defaults_to_pro — Test that subscription.created with missing tier metadata defaults to 'pro'
    • test_subscription_created_with_basic_tier_defaults_to_pro — Test that subscription.created with tier='basic' upgrades to 'pro'
  • TestCheckoutCompletedTrialStatusClearing (3 tests)
    • test_checkout_completed_clears_trial_status — Test that checkout.session.completed clears trial status for users
    • test_checkout_completed_clears_multiple_api_keys — Test that checkout.session.completed clears trial status for all user's API keys
    • test_checkout_completed_does_not_fail_if_no_api_keys — Test that checkout completed doesn't fail if user has no API keys
  • TestCacheInvalidationOnSubscriptionUpdates (5 tests)
    • test_subscription_created_invalidates_cache — Test that subscription.created webhook invalidates user cache
    • test_subscription_updated_invalidates_cache — Test that subscription.updated webhook invalidates user cache
    • test_subscription_deleted_invalidates_cache — Test that subscription.deleted webhook invalidates user cache
    • test_invoice_payment_failed_invalidates_cache — Test that invoice.payment_failed webhook invalidates user cache
    • test_checkout_completed_invalidates_cache_after_all_updates — Test that checkout.session.completed invalidates cache AFTER all user updates
  • TestUserIdExtractionFallback (7 tests)
    • test_extract_user_id_from_metadata — Test that user_id is extracted from metadata when present
    • test_extract_user_id_fallback_to_customer_lookup — Test that user_id is looked up by stripe_customer_id when metadata is missing
    • test_extract_user_id_returns_none_when_not_found — Test that None is returned when user cannot be identified
    • test_subscription_created_uses_fallback_lookup — Test that subscription.created uses customer_id fallback when metadata missing
    • test_subscription_created_raises_when_user_not_found — Test that subscription.created raises ValueError when user cannot be identified
    • test_subscription_updated_uses_fallback_lookup — Test that subscription.updated uses customer_id fallback when metadata missing
    • test_lookup_user_by_stripe_customer — Test the _lookup_user_by_stripe_customer helper method

tests/services/test_sybil_client.py

14 tests

  • TestSybilClient (14 tests)
    • test_get_sybil_client_raises_without_api_key — Test that get_sybil_client raises ValueError when API key is not configured
    • test_get_sybil_client_returns_pooled_client — Test that get_sybil_client returns a pooled client when API key is configured
    • test_fetch_models_from_sybil_success — Test successful model fetching from Sybil API
    • test_fetch_models_from_sybil_handles_embedding_models — Test that embedding models are properly handled
    • test_fetch_models_from_sybil_http_error — Test that HTTP errors are handled gracefully
    • test_fetch_models_from_sybil_network_error — Test that network errors are handled gracefully
    • test_fetch_models_from_sybil_skips_models_without_id — Test that models without an ID are skipped
    • test_fetch_models_from_sybil_handles_missing_pricing — Test that missing pricing information is handled with defaults
    • test_make_sybil_request_openai — Test making a request to Sybil using OpenAI client
    • test_make_sybil_request_openai_stream — Test making a streaming request to Sybil
    • test_process_sybil_response — Test processing a Sybil response
    • test_is_sybil_model_returns_true_for_valid_model — Test that is_sybil_model returns True for a valid model
    • test_is_sybil_model_returns_false_for_invalid_model — Test that is_sybil_model returns False for an invalid model
    • test_is_sybil_model_handles_errors — Test that is_sybil_model handles errors gracefully

tests/services/test_tempo_otlp.py

2 tests

  • TestTempoOtlp (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/services/test_together_client.py

5 tests

  • TestTogetherClient (5 tests)
    • test_get_together_client — Test getting Together client
    • test_get_together_client_no_key — Test getting Together client without API key
    • test_make_together_request_openai — Test making request to Together
    • test_make_together_request_openai_stream — Test making streaming request to Together
    • test_process_together_response — Test processing Together response

tests/services/test_tools.py

56 tests

  • TestToolResult (3 tests)
    • test_success_result — Test creating a successful result.
    • test_error_result — Test creating an error result.
    • test_to_dict — Test converting result to dictionary.
  • TestBaseTool (4 tests)
    • test_cannot_instantiate_directly — Test that BaseTool cannot be instantiated directly.
    • test_concrete_implementation — Test implementing a concrete tool.
    • test_success_helper — Test the _success helper method.
    • test_error_helper — Test the _error helper method.
  • TestToolRegistry (5 tests)
    • test_get_tool_definitions — Test getting all tool definitions.
    • test_get_tool_by_name_exists — Test getting an existing tool by name.
    • test_get_tool_by_name_not_exists — Test getting a non-existent tool.
    • test_execute_tool_success — Test executing a tool by name with parameter validation.
    • test_execute_tool_not_found — Test executing a non-existent tool raises ValueError.
  • TestChatterboxModels (9 tests)
    • test_turbo_model_english_only — Test that turbo model supports only English.
    • test_multilingual_supports_many_languages — Test that multilingual model supports 22+ languages.
    • test_get_chatterbox_models — Test getting formatted model list.
    • test_validate_chatterbox_model_valid — Test validating valid model IDs.
    • test_validate_chatterbox_model_invalid — Test validating invalid model IDs.
    • test_validate_language_valid — Test validating valid language codes.
    • test_validate_language_invalid_language — Test validating invalid language codes.
    • test_validate_language_wrong_model — Test validating language for wrong model.
    • test_validate_language_invalid_model — Test validating language for non-existent model.
  • TestLanguageNames (1 tests)
    • test_common_languages_present — Test that common languages are present.
  • TestSSRFProtection (6 tests)
    • test_safe_public_urls — Test that public URLs are allowed.
    • test_blocks_localhost — Test that localhost URLs are blocked.
    • test_blocks_private_ips — Test that private IP ranges are blocked.
    • test_blocks_cloud_metadata — Test that cloud metadata URLs are blocked.
    • test_blocks_non_http_schemes — Test that non-HTTP schemes are blocked.
    • test_blocks_empty_or_invalid — Test that empty or invalid URLs are blocked.
  • TestTextToSpeechTool (4 tests)
    • test_execute_success — Test successful TTS execution.
    • test_execute_validation_error — Test TTS execution with validation error.
    • test_execute_runtime_error — Test TTS execution with runtime error.
    • test_execute_with_all_options — Test TTS execution with all options and parameter validation.
  • TestGenerateSpeech (7 tests)
    • test_empty_text_raises_error — Test that empty text raises ValueError.
    • test_whitespace_text_raises_error — Test that whitespace-only text raises ValueError.
    • test_text_too_long_raises_error — Test that text over 5000 chars raises ValueError.
    • test_invalid_model_raises_error — Test that invalid model raises ValueError.
    • test_invalid_language_for_model_raises_error — Test that invalid language for model raises ValueError.
    • test_ssrf_url_raises_error — Test that SSRF URLs are rejected.
    • test_localhost_url_raises_error — Test that localhost URLs are rejected.
  • TestToolsRoute (7 tests)
    • test_list_tools — Test listing all tools.
    • test_get_definitions — Test getting tool definitions.
    • test_get_tool_info_exists — Test getting info for existing tool.
    • test_get_tool_info_not_exists — Test getting info for non-existent tool.
    • test_execute_tool_success — Test executing a tool successfully with auth.
    • test_execute_tool_not_found — Test executing a non-existent tool.
    • test_execute_tool_validation_error — Test executing a tool with validation error.
  • TestToolIntegration (1 tests)
    • test_all_tools_have_valid_definitions — Test that all registered tools have valid definitions.
  • TestSearchAugmentRoute (9 tests)
    • test_search_augment_success — Test successful search augmentation.
    • test_search_augment_with_parameters — Test search augmentation passes correct parameters.
    • test_search_augment_no_results — Test search augmentation with no results.
    • test_search_augment_tool_failure — Test search augmentation when tool execution fails.
    • test_search_augment_validation_empty_query — Test search augmentation with empty query.
    • test_search_augment_validation_invalid_max_results — Test search augmentation with invalid max_results.
    • test_search_augment_truncates_long_content — Test that search augmentation truncates long content.
    • test_search_augment_multiple_results — Test search augmentation formats multiple results correctly.
    • test_search_augment_exception_handling — Test search augmentation handles exceptions gracefully.

tests/services/test_trial_service.py

34 tests

  • TestTrialServiceInit (3 tests)
    • test_init_success — Test successful initialization
    • test_init_missing_url — Test initialization fails without SUPABASE_URL
    • test_init_missing_key — Test initialization fails without SUPABASE_KEY
  • TestStartTrial (5 tests)
    • test_start_trial_success — Test successful trial start
    • test_start_trial_api_key_not_found — Test trial start with invalid API key
    • test_start_trial_already_started — Test trial start when trial already active
    • test_start_trial_database_error — Test trial start with database error
    • test_start_trial_exception — Test trial start with exception
  • TestGetTrialStatus (4 tests)
    • test_get_trial_status_success — Test successful trial status retrieval
    • test_get_trial_status_api_key_not_found — Test trial status with invalid API key
    • test_get_trial_status_database_error — Test trial status with database error
    • test_get_trial_status_exception — Test trial status with exception in _get_api_key_id
  • TestConvertTrial (4 tests)
    • test_convert_trial_success — Test successful trial conversion
    • test_convert_trial_api_key_not_found — Test conversion with invalid API key
    • test_convert_trial_database_error — Test conversion with database error
    • test_convert_trial_exception — Test conversion with exception in _get_api_key_id
  • TestTrackTrialUsage (3 tests)
    • test_track_usage_success — Test successful usage tracking
    • test_track_usage_api_key_not_found — Test usage tracking with invalid API key
    • test_track_usage_exception — Test usage tracking with exception
  • TestGetSubscriptionPlans (3 tests)
    • test_get_plans_success — Test successful plans retrieval
    • test_get_plans_empty — Test plans retrieval with no plans
    • test_get_plans_exception — Test plans retrieval with exception
  • TestValidateTrialAccess (8 tests)
    • test_validate_access_success — Test successful access validation
    • test_validate_access_status_check_failed — Test validation when status check fails
    • test_validate_access_not_trial — Test validation for non-trial account
    • test_validate_access_expired — Test validation for expired trial
    • test_validate_access_insufficient_tokens — Test validation with insufficient tokens
    • test_validate_access_insufficient_requests — Test validation with insufficient requests
    • test_validate_access_insufficient_credits — Test validation with insufficient credits
    • test_validate_access_exception — Test validation with exception
  • TestHelperMethods (3 tests)
    • test_get_api_key_id_success — Test successful API key ID retrieval
    • test_get_api_key_id_not_found — Test API key ID not found
    • test_get_api_key_id_exception — Test API key ID with exception
  • TestGlobalServiceInstance (1 tests)
    • test_get_trial_service_singleton — Test that get_trial_service returns singleton

tests/services/test_trial_validation.py

25 tests

  • test_validate_missing_key_returns_not_found
  • test_validate_non_trial_key
  • test_validate_expired_iso_z_marked_expired
  • test_validate_tokens_cap_exceeded
  • test_validate_requests_cap_exceeded
  • test_validate_credits_cap_exceeded
  • test_validate_valid_trial
  • test_validate_handles_exception
  • test_track_usage_success_updates
  • test_track_usage_key_not_found
  • test_track_usage_handles_exception
  • test_track_usage_with_model_specific_pricing — Test that track_trial_usage uses model-specific pricing when model info is provided
  • test_track_usage_fallback_without_model_info — Test that track_trial_usage falls back to flat rate when model info is not provided
  • test_track_usage_unknown_model_uses_flat_rate — Test that unknown models (not in catalog) use flat rate instead of near-zero pricing
  • test_trial_cache_hit_skips_database — Second call with same API key should use cache (not hit database)
  • test_trial_cache_different_keys_separate_entries — Different API keys should have separate cache entries
  • test_clear_trial_cache_all — clear_trial_cache() with no arguments should clear entire cache
  • test_invalidate_trial_cache_specific_key — invalidate_trial_cache should clear cache for specific key
  • test_get_trial_cache_stats — get_trial_cache_stats should return cache statistics
  • test_track_usage_invalidates_cache — track_trial_usage should invalidate cache after successful update
  • test_validate_retries_on_http2_connection_error — HTTP/2 connection errors should trigger retry with client refresh
  • test_validate_retries_on_connection_terminated_error — ConnectionTerminated errors should trigger retry with client refresh
  • test_validate_retries_on_send_headers_state_error — SEND_HEADERS state errors should trigger retry with client refresh
  • test_validate_max_retries_exceeded — Should fail after max retries are exhausted
  • test_validate_no_retry_on_non_connection_error — Non-connection errors should not trigger retry logic

tests/services/test_unified_catalog_cache.py

23 tests

  • TestModelCatalogCacheEnhancements (10 tests)
    • test_cache_gateway_catalog — Test caching gateway-specific catalog
    • test_get_gateway_catalog_hit — Test cache hit for gateway catalog
    • test_get_gateway_catalog_miss — Test cache miss for gateway catalog
    • test_invalidate_gateway_catalog — Test gateway catalog invalidation
    • test_cache_unique_models — Test caching unique models
    • test_get_unique_models — Test retrieving cached unique models
    • test_cache_catalog_stats — Test caching catalog statistics
    • test_get_catalog_stats — Test retrieving cached catalog statistics
    • test_redis_unavailable_graceful_degradation — Test graceful degradation when Redis is unavailable
    • test_cache_stats_tracking — Test that cache statistics are tracked correctly
  • TestConvenienceFunctions (4 tests)
    • test_cache_gateway_catalog_convenience — Test convenience function for caching gateway catalog
    • test_invalidate_gateway_catalog_convenience — Test convenience function for invalidating gateway catalog
    • test_cache_unique_models_convenience — Test convenience function for caching unique models
    • test_cache_catalog_stats_convenience — Test convenience function for caching catalog stats
  • TestBatchInvalidation (6 tests)
    • test_invalidate_providers_batch_success — Test successful batch invalidation of multiple providers
    • test_invalidate_providers_batch_with_cascade — Test batch invalidation with cascade to full catalog
    • test_invalidate_providers_batch_empty_list — Test batch invalidation with empty provider list
    • test_invalidate_providers_batch_redis_unavailable — Test batch invalidation when Redis is unavailable
    • test_invalidate_providers_batch_pipeline_error — Test batch invalidation when pipeline raises exception
    • test_invalidate_providers_batch_performance — Test that batch invalidation uses single pipeline operation
  • TestCacheIntegration (3 tests)
    • test_cache_key_consistency — Test that cache keys follow consistent naming convention
    • test_ttl_configuration — Test that different cache types have appropriate TTLs
    • test_cache_invalidation_cascade — Test that invalidating a gateway catalog also invalidates full catalog

tests/services/test_user_lookup_cache.py

10 tests

  • TestUserLookupCache (10 tests)
    • test_cache_get_user_first_call_hits_database — First call to get_user should hit the database
    • test_cache_get_user_second_call_uses_cache — Second call with same API key should use cache (not hit database)
    • test_cache_get_user_different_keys_separate_cache_entries — Different API keys should have separate cache entries
    • test_cache_returns_none_when_user_not_found — Cache should handle None returns from database (None is NOT cached)
    • test_clear_cache_all — clear_cache() with no arguments should clear entire cache
    • test_clear_cache_specific_key — clear_cache(api_key) should clear only that key
    • test_invalidate_user — invalidate_user should clear cache for specific user
    • test_get_cache_stats — get_cache_stats should return cache statistics
    • test_set_cache_ttl — set_cache_ttl should update cache TTL
    • test_cache_statistics_empty — get_cache_stats should work on empty cache

tests/services/test_vercel_ai_gateway_client.py

2 tests

  • TestVercelAiGatewayClient (2 tests)
    • test_module_imports — Test that module imports successfully
    • test_module_has_expected_attributes — Test module exports

tests/services/test_web_search_tool.py

18 tests

  • TestWebSearchToolDefinition (4 tests)
    • test_tool_registered — Test that web_search tool is registered.
    • test_get_definition — Test tool definition format.
    • test_search_depth_enum — Test that search_depth has correct enum values.
    • test_max_results_bounds — Test that max_results has correct bounds.
  • TestWebSearchToolExecution (11 tests)
    • test_execute_success — Test successful search execution.
    • test_execute_without_query — Test execution without query parameter.
    • test_execute_with_empty_query — Test execution with empty query.
    • test_execute_without_api_key — Test execution without API key configured.
    • test_execute_with_invalid_api_key — Test execution with invalid API key.
    • test_execute_rate_limit — Test execution with rate limit error.
    • test_execute_timeout — Test execution timeout handling.
    • test_execute_connection_error — Test execution connection error handling.
    • test_execute_with_all_options — Test execution with all optional parameters.
    • test_execute_max_results_clamped — Test that max_results is clamped to valid range.
    • test_execute_no_results — Test execution with no search results.
  • TestWebSearchToolIntegration (3 tests)
    • test_get_tool_by_name — Test getting WebSearchTool by name.
    • test_execute_via_registry — Test executing web_search via execute_tool function.
    • test_tool_definition_in_all_definitions — Test that web_search is included in get_tool_definitions.

tests/services/test_webhook_deduplication.py

7 tests

  • TestWebhookDeduplication (7 tests)
    • test_event_not_processed_initially — Test that new event is not marked as processed
    • test_event_already_processed — Test that processed event is detected
    • test_record_processed_event — Test recording a processed event
    • test_webhook_handler_skips_duplicate_events — Test that webhook handler skips duplicate events
    • test_record_event_with_metadata — Test recording event with metadata
    • test_get_processed_event — Test retrieving processed event details
    • test_event_processing_error_does_not_block — Test that database errors don't block event processing

tests/services/test_xai_client.py

13 tests

  • TestXaiReasoningDetection (4 tests)
    • test_is_reasoning_model_grok_4 — Test that Grok 4 models are detected as reasoning models
    • test_is_reasoning_model_grok_3_mini — Test that Grok 3 mini models are detected as reasoning models
    • test_non_reasoning_models — Test that non-reasoning models are correctly identified
    • test_case_insensitivity — Test that model detection is case-insensitive
  • TestXaiReasoningParams (5 tests)
    • test_reasoning_params_for_reasoning_model — Test reasoning params are generated for reasoning models
    • test_reasoning_params_for_non_reasoning_model — Test no reasoning params for non-reasoning models
    • test_explicit_enable_reasoning — Test explicitly enabling reasoning
    • test_explicit_disable_reasoning — Test explicitly disabling reasoning
    • test_explicit_enable_on_non_reasoning_model — Test that explicit enable on non-reasoning model returns empty
  • TestXaiRequestWithReasoning (4 tests)
    • test_make_request_adds_reasoning_params — Test that reasoning params are added to requests for reasoning models
    • test_make_request_no_reasoning_for_old_models — Test that reasoning params are NOT added for non-reasoning models
    • test_make_stream_request_adds_reasoning_params — Test that streaming requests also get reasoning params
    • test_explicit_reasoning_override — Test that explicit enable_reasoning parameter works

tests/services/test_zai_client.py

12 tests

  • TestZaiClient (12 tests)
    • test_get_zai_client — Test getting Z.AI client
    • test_get_zai_client_no_key — Test getting Z.AI client without API key
    • test_make_zai_request_openai — Test making request to Z.AI
    • test_make_zai_request_openai_with_kwargs — Test making request to Z.AI with additional parameters
    • test_make_zai_request_openai_error — Test handling errors from Z.AI
    • test_make_zai_request_openai_stream — Test making streaming request to Z.AI
    • test_make_zai_request_openai_stream_with_kwargs — Test making streaming request to Z.AI with additional parameters
    • test_make_zai_request_openai_stream_error — Test handling streaming errors from Z.AI
    • test_process_zai_response — Test processing Z.AI response
    • test_process_zai_response_no_usage — Test processing Z.AI response without usage data
    • test_process_zai_response_multiple_choices — Test processing Z.AI response with multiple choices
    • test_process_zai_response_with_tool_calls — Test processing Z.AI response with tool calls

tests/services/test_zero_model_fallback.py

19 tests

  • TestZeroModelDetection (4 tests)
    • test_empty_list_detected_as_zero_models — Test that empty list response is detected as zero models
    • test_none_data_detected_as_zero_models — Test that None data is detected as zero models
    • test_below_minimum_threshold_detected — Test that model count below minimum threshold is detected
    • test_api_response_zero_models_detection — Test that API returning zero models is properly detected
  • TestDatabaseBackedFallback (3 tests)
    • test_fallback_returns_db_models_when_api_fails — Test that fallback returns database models when API returns none
    • test_fallback_preserves_model_metadata — Test that fallback models preserve essential metadata
    • test_fallback_empty_when_no_db_models — Test graceful handling when no database fallback models exist
  • TestRetryMechanisms (2 tests)
    • test_cache_clear_triggers_refetch — Test that clearing cache triggers a refetch attempt
    • test_auto_fix_attempts_recovery — Test that auto-fix attempts to recover from zero-model state
  • TestProviderFailoverOnZeroModels (2 tests)
    • test_failover_chain_skips_zero_model_providers — Test that failover chain skips providers with zero models
    • test_failover_prioritizes_healthy_providers — Test that failover prioritizes healthy providers with models
  • TestZeroModelAlerting (2 tests)
    • test_zero_model_event_can_be_logged — Test that zero-model events can be properly logged
    • test_gateway_health_includes_model_count — Test that gateway health status includes model count information
  • TestGatewayRecovery (2 tests)
    • test_cache_repopulates_after_recovery — Test that cache repopulates after API recovers
    • test_health_status_updates_after_recovery — Test that health status properly updates after recovery
  • TestFailoverServiceIntegration (2 tests)
    • test_explain_failover_shows_recommendation_for_missing_model — Test that explain_failover correctly shows recommendation when model has limited providers
    • test_explain_failover_structure — Test that explain_failover returns expected structure
  • TestMinimumModelThresholds (2 tests)
    • test_all_gateways_have_minimum_thresholds — Test that all gateways have configured minimum model thresholds
    • test_major_gateways_have_reasonable_thresholds — Test that major gateways have reasonable minimum thresholds

tests/smoke/

1 files, 20 tests

tests/smoke/test_deployment.py

20 tests

  • TestApplicationHealth (3 tests)
    • test_app_is_running — Test that the application responds
    • test_health_endpoint_structure — Test health endpoint returns proper structure
    • test_root_endpoint_responds — Test root endpoint responds
  • TestCriticalEndpointsExist (5 tests)
    • test_chat_completions_endpoint_exists — Test /v1/chat/completions endpoint exists
    • test_messages_endpoint_exists — Test /v1/messages endpoint exists (Anthropic/Claude API)
    • test_images_endpoint_exists — Test /v1/images/generations endpoint exists
    • test_catalog_models_endpoint_exists — Test /catalog/models endpoint exists
    • test_catalog_providers_endpoint_exists — Test /catalog/providers endpoint exists
  • TestAuthenticationSystem (2 tests)
    • test_endpoints_require_authentication — Test that protected endpoints reject unauthenticated requests
    • test_invalid_api_key_rejected — Test that invalid API keys are rejected
  • TestAuthenticatedRequests (2 tests)
    • test_can_check_user_balance — Test that authenticated user can check balance
    • test_can_make_chat_completion_request — Test that authenticated user can make chat completion request
  • TestDatabaseConnectivity (1 tests)
    • test_database_connection_via_health — Test database connection through health endpoint
  • TestExternalServices (1 tests)
    • test_can_reach_openrouter — Test that OpenRouter is reachable (if configured)
  • TestResponseTimes (2 tests)
    • test_health_check_responds_quickly — Test health check responds within acceptable time
    • test_catalog_endpoints_respond_quickly — Test catalog endpoints respond within acceptable time
  • TestErrorHandling (3 tests)
    • test_invalid_endpoint_returns_404 — Test that invalid endpoints return 404
    • test_malformed_json_returns_400 — Test that malformed JSON returns 400
    • test_missing_required_fields_returns_422 — Test that missing required fields returns validation error
  • TestCORSConfiguration (1 tests)
    • test_cors_headers_present — Test that CORS headers are present in responses

tests/test_admin_stats_growth_fix.py/

1 files, 9 tests

tests/test_admin_stats_growth_fix.py

9 tests

  • TestAdminUsersStats (3 tests)
    • test_stats_returns_accurate_totals_not_limited — Test that stats endpoint returns accurate totals beyond 1,000 users
    • test_stats_with_filters — Test stats endpoint with email filter applied
    • test_stats_unauthorized — Test that stats endpoint requires admin authentication
  • TestAdminUsersGrowth (4 tests)
    • test_growth_returns_correct_data_points — Test that growth endpoint returns correct number of data points
    • test_growth_different_day_ranges — Test growth endpoint with different day ranges
    • test_growth_unauthorized — Test that growth endpoint requires admin authenticati
Add a custom footer

Home

Reading Path (start here, in order)

  1. Conceptual Model
  2. Stability Definition
  3. Conceptual Model Features
  4. Features
  5. Delta Report
  6. Features-Acceptance-Criteria

Testing

Security & Access

Billing

Monitoring

Features

Providers

Operations

Data References

Clone this wiki locally