feat(rest): introduce AuthManager/AuthSession and migrate OAuth2

[plusplusjiajia]

Modeled on Java's AuthManager API (the init/catalog session lifecycle, the Noop/OAuth2 manager set, and the SigV4-wraps-a-delegate composition coming in the follow-up), adapted to Rust idioms.

What it does

• AuthManager/AuthSession traits in a new auth/ module: init_session() serves the GET /v1/config handshake, catalog_session(merged_props) serves everything after, so a manager can rebuild its session from server-merged properties.
• Noop/OAuth2 managers, selected via a new rest.auth.type property (oauth2 is the default and behaves as no auth when neither token nor credential is set), injectable through RestCatalogBuilder::with_auth_manager.
• OAuth2 token handling moves out of HttpClient into OAuth2Manager, with the cached token surviving the config handshake; OAuth2Manager is publicly constructible (new() + with_*).
• Review items from feat(catalog-rest): introduce AuthManager/AuthSession, rework SigV4 as a wrapping auth manager #2815 folded in: the config field is auth_manager, and the test-only fake-request token shim is gone — tests observe the session's cached bearer (#[cfg(test)] bearer_token()) and assert the header the mock server receives.

No new dependencies; no public API removed (additions only, public-api.txt regenerated).

Java reference: org.apache.iceberg.rest.auth.

Deviations from Java

• No tableSession/contextualSession yet — in Java they are default methods falling back to the catalog/parent session, and the Rust REST catalog has no call sites for them (contextualSession also needs a SessionCatalog concept that doesn't exist here yet). Adding defaulted trait methods later is non-breaking.
• No close() — Rust relies on Drop, and this OAuth2 implementation has no background refresh executor to shut down.
• AuthSession gains invalidate()/refresh() (not in Java) to back the existing RestCatalog::invalidate_token/regenerate_token APIs.
• authenticate mutates the request in place instead of returning a new one.

[plusplusjiajia]

Split from #2815 per your review, @CTTY — first of two: the AuthManager/AuthSession traits + OAuth2 migration (pure refactor, existing OAuth2 tests pass unmodified). SigV4 follows by reworking #2660 on top.
Would appreciate a review when you have a chance.

[DerGut]

Carrying my feedback from the other PR over.

I think right now having the AuthManager and AuthSession as fields of the HttpClient is perfectly fine. However, we're going to need to add additional methods to the AuthManager trait which complicate this:

fn table_session(_: TableIdent, parent: Arc<dyn AuthSession>) -> Arc<dyn AuthSession>;

fn contextual_session(_: SessionContext, parent: Arc<dyn AuthSession>) -> Arc<dyn AuthSession;

table_session() and contextual_session() will help us to enable credentials vending at table-level and query-level authentication respectively.
Both take arguments that a low-level HttpClient should have no business in dealing with IMO. E.g. TableIdent is Iceberg-specific and not HTTP-specific. Same goes for the SessionContext. In that sense, I feel like the RestCatalog may be a better place to host these two fields.

[DerGut]

I wonder whether we should call out in the comments that these methods are only exposed for backwards-compatibility and that they aren't the intended main interface to work with going forward.

To give implementers of custom AuthManagers some guidance.

[DerGut]

IIUC this method is only (indirectly) used in three tests.

I'd argue that its somewhat redundant with the fn authenticate() and test helpers that simplify the ergonomics should probably live closer to the tests rather than extending the trait (which is public API).

Test helpers could rebuild this functionality in a test module. A shortened version:

async fn bearer_token_from_session(session: &dyn AuthSession) -> Result<Option<String>> {
    let header = authorization_header_from_session(session).await?;
    let bearer_token = header
        .map(|header| header.strip_prefix("Bearer "))
    Ok(bearer_token)
}

async fn authorization_header_from_session(session: &dyn AuthSession) -> Result<Option<String>> {
    let req = Request::new(Method::GET, Url::parse("http://fake.com")?);
    let mut req = AuthRequest::new(req);
    session.authenticate(&mut req).await?;

    Ok(req.headers().get(AUTHORIZATION))
}

[DerGut]

I wonder whether we can instantiate the init_session in the scope of its use (the first /v1/config request) so that we don't have to deal with explicit drops.

This could be another signal that the AuthManager should rather live in the catalog because the HttpClient is not aware of which request is being made, and so it can't tell which session is the appropriate one to use (or to build).
In that sense, it's implicitly temporally coupled to what the session field has been set to, and has to assume that the first request being made is a /v1/config request.

[DerGut]

Just flagging that this diverges from Java's default to none.

Even though the OAuth2Manager behaves similarly without a token, it doesn't match the NoopAuthManager's behavior exactly. For example:

• a configured NoopAuthManager on client initialization will always noop - an OAuth2Manager can start authenticating if the /v1/config endpoint returns a token in the properties (this is arguably the better default behavior)
• a call to NoopSession::refresh() will always succeed but a call on OAuth2Session::refresh() will fail if no token is backing it