feat(ai-proxy): support GCP Application Default Credentials / Workload Identity (metadata server) for Vertex AI auth

[namannagar89]

Description

The ai-proxy / ai-proxy-multi GCP (Vertex AI) auth currently supports only a static service-account JSON key. When APISIX runs on GKE with Workload Identity (or any environment where Application Default Credentials are available), there is no way to authenticate to Vertex AI without mounting a long-lived SA key.

This forces users to create, store, and rotate a static SA key purely for APISIX — a downgrade from the keyless posture GKE Workload Identity otherwise provides across the cluster.

Current behavior

fetch_gcp_access_token in apisix/plugins/ai-transport/auth.lua reads the key from gcp.service_account_json or the GCP_SERVICE_ACCOUNT env var, then hands it to apisix/utils/google-cloud-oauth.lua, which mints a token via the JWT-bearer grant:

-- apisix/utils/google-cloud-oauth.lua
function _M.refresh_access_token(self)
    -- POST self.token_uri with
    --   grant_type = "urn:ietf:params:oauth:grant-type:jwt-bearer",
    --   assertion  = self:generate_jwt_token()   -- jwt:sign(self.private_key, {iss=self.client_email})

If no key is provided, auth_conf is {}, client_email/private_key are nil, and token generation fails. There is no metadata-server / ADC fallback.

Proposed enhancement

When no service_account_json / GCP_SERVICE_ACCOUNT is configured (empty auth conf), fall back to fetching an access token from the GCE/GKE metadata server, which is how Application Default Credentials / Workload Identity work:

GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token
    Header: Metadata-Flavor: Google
-> { "access_token": "...", "expires_in": 3599, "token_type": "Bearer" }

This is a small, self-contained addition to google-cloud-oauth.lua (a code path that, given empty config, queries the metadata endpoint instead of signing a JWT). The existing token-caching logic (max_ttl, expire_early_secs) can be reused unchanged, since the metadata response already carries expires_in.

Suggested config (opt-in, or auto when key absent):

ai-proxy:
  provider: vertex-ai
  auth:
    gcp:
      use_metadata_server: true   # or: infer when service_account_json/env unset
  provider_conf:
    project_id: my-project
    region: us-east5
  options:
    model: gemini-2.5-flash

Use case

• APISIX on GKE with Workload Identity: bind the gateway's KSA to a GCP SA (roles/aiplatform.user) and let ai-proxy obtain tokens keylessly — no static SA key to mount, store, or rotate.
• Aligns APISIX with how other Google-SDK-based gateways already authenticate to Vertex AI.

Alternatives considered

• Static SA key (current) — works, but reintroduces a long-lived credential.
• External sidecar that fronts Vertex with ADC — extra moving part; defeats using ai-proxy directly.

Environment

• APISIX gateway 3.17.x (ai-proxy plugins present in-image).
• Deployment: GKE with Workload Identity enabled.