ETL: multi-tenant token management for high-volume data pipelines

[francescobianco]

Problem

In multi-tenant SaaS platforms, each tenant has its own Openapi credentials. Running ETL jobs for hundreds of tenants simultaneously requires generating, caching, and refreshing tokens per-tenant — something the current SDK does not model.

Current workaround

foreach ($tenants as $tenant) {
    $oauth = new OauthClient($tenant['username'], $tenant['apikey']);
    $token = json_decode($oauth->createToken($scopes), true)['token'];

    // Problem: no TTL tracking, no reuse across jobs, no central cache
    $client = new Client($token);
    runEtlJob($tenant, $client);
}

What would help

A TokenPool or TokenStore abstraction that:

• Caches tokens per-tenant until near expiry
• Refreshes automatically before a job starts
• Plugs into the existing CacheInterface

$pool = new TokenPool($oauthClient, cache: new RedisCache($redis));
$client = new Client(tokenPool: $pool->forTenant($tenantId));

Open questions

• Is TokenPool within scope for the core SDK or a separate package?
• Should it integrate with the existing Cache\CacheInterface or define its own contract?
• How should token expiry be tracked — store the TTL alongside the token?