Skip to content

[FEATURE] Support Vertex AI multi-region endpoint URL pattern (.rep.googleapis.com) #780

@fwuensche

Description

@fwuensche

Scope check

  • This is core LLM communication (not application logic)
  • This benefits most users (not just my use case)
  • This can't be solved in application code with current RubyLLM (only via monkey-patching internals)
  • I read the Contributing Guide

Due diligence

  • I searched existing issues
  • I checked the documentation

What problem does this solve?

Google Vertex AI exposes generative models through three URL families:

Location kind Host
Per-region (e.g. europe-west9) https://<region>-aiplatform.googleapis.com
Multi-region (eu, us) https://aiplatform.<eu|us>.rep.googleapis.com
Global https://aiplatform.googleapis.com

RubyLLM::Providers::VertexAI#api_base handles per-region and global but not multi-region. Setting vertexai_location = "eu" currently builds https://eu-aiplatform.googleapis.com/..., which does not resolve, producing an HTML 404 page from Google.

This matters today because Gemini 3.5 Flash (GA on May 19, 2026) is only published to multi-region and global endpoints — no individual region serves it (verified empirically across europe-west1/3/4/8/9, europe-central2, europe-north1, europe-southwest1, us-central1 — all return Publisher Model not found). Any RubyLLM user who needs EU data residency for 3.5 Flash (GDPR / HDS / French health hosting) currently has no path forward without patching the gem.

Proposed solution

Either of these would solve it; option A is more transparent, option B is more general:

Option A — Recognize eu/us as multi-region in the URL builder. Roughly:

def api_base
  case @config.vertexai_location.to_s
  when 'global'  then 'https://aiplatform.googleapis.com/v1beta1'
  when 'eu', 'us' then "https://aiplatform.#{@config.vertexai_location}.rep.googleapis.com/v1beta1"
  else "https://#{@config.vertexai_location}-aiplatform.googleapis.com/v1beta1"
  end
end

Option B — Add a vertexai_api_base config option mirroring openai_api_base, gemini_api_base, perplexity_api_base, etc. The Vertex provider is currently the only Google provider without an api_base override (see also #774 for Perplexity).

For reference, our current monkey-patch (an Ai:: module we prepend on RubyLLM::Providers::VertexAI):

module VertexAiMultiregion
  MULTI_REGION_LOCATIONS = %w[eu us].freeze

  def api_base
    location = @config.vertexai_location.to_s
    if MULTI_REGION_LOCATIONS.include?(location)
      "https://aiplatform.#{location}.rep.googleapis.com/v1beta1"
    else
      super
    end
  end
end

RubyLLM::Providers::VertexAI.prepend(VertexAiMultiregion)

Why this belongs in RubyLLM

URL construction for a provider's published endpoints is core to LLM communication. Multi-region endpoints are documented public Vertex AI infrastructure (not a niche or experimental feature), and they are the only way to access certain Gemini models from EU-residency-sensitive workloads. The fix is provider-internal — no application-level workaround is possible without monkey-patching api_base. Other providers in RubyLLM already expose <provider>_api_base overrides; aligning Vertex with that pattern (or auto-handling the standard multi-region locations) would remove the divergence.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions