feat(catalog-rest): parse server-advertised endpoints from GET /v1/config

[huan233usc]

Which issue does this PR close?

Related to #1690 (server-side scan planning), which relies on endpoint
negotiation. This PR lands the negotiation piece on its own so it can be
reviewed independently.

What changes are included in this PR?

The REST spec lets a server advertise the routes it supports in the endpoints
field of its GET /v1/config response, so clients can negotiate optional
capabilities instead of assuming every server implements every operation. The
Rust client currently ignores this field.

This mirrors the capability negotiation the Java client gained in
apache/iceberg#10929 (the endpoints field in the config response, the
Endpoint type, and the default-endpoint fallback when a server omits the list).

This PR:

• Adds an Endpoint type (HTTP method + path template) parsed from the
  "<method> <path>" wire form via FromStr — which validates the single-space
  shape and the HTTP method — with serde delegating to it. The method is stored
  as a typed http::Method (kept out of the public API; method() returns
  &str).
• Parses the config response's endpoints into Option<Vec<Endpoint>> so an
  absent field and an explicit empty list are modelled distinctly, and stores
  the negotiated set on the catalog's runtime context.
• Resolves the set following the spec's optional-field semantics: an absent
  field → a standard base set of namespace/table operations is assumed (so an
  older server still resolves its core operations as supported); a present list
  — even an empty one — is used verbatim. Optional endpoints outside the base
  set stay unsupported unless explicitly advertised.
• Exposes RestCatalog::supports_endpoint(&Endpoint) to query the negotiated
  set.

No existing behaviour changes for callers that don't consult supports_endpoint.

Are these changes tested?

Yes:

• Unit tests for Endpoint: FromStr/serde round-trip, rejection of malformed
  input (no / extra / leading / trailing space, empty), HTTP-method
  normalization, and the default endpoint set.
• mockito catalog tests: a server that advertises endpoints (asserting
  supports_endpoint is true/false for listed/unlisted routes), a server that
  omits the field (base operations resolve as supported), and a server that
  sends an explicit empty list (nothing is supported).

[amogh-jahagirdar]

Overall looks great @huan233usc just had some questions on some comments and I think we probably want to handle the explicit empty case a bit differently by just falling back to the default.

[huan233usc]

Thank you @amogh-jahagirdar and @kevinjqliu for the review! Tried addressing the comments from both! PTAL, thanks

[kevinjqliu]

is there already an integration test setup using the iceberg-rest-fixture docker image? i think theres is, but not 100% sure.

Would be great to add some integration tests agains that IRC endpoint

[huan233usc]

is there already an integration test setup using the iceberg-rest-fixture docker image? i think theres is, but not 100% sure.

Would be great to add some integration tests agains that IRC endpoint

Yeah — crates/integration_tests runs against the iceberg-rest-fixture image. Added endpoint_negotiation_test.rs exercising supports_endpoint against the fixture's real GET /v1/config. Thanks!

[huan233usc]

Hi @amogh-jahagirdar and @kevinjqliu, I tried addressing the comments, could you help taking a look when you have a chance? Thanks

[amogh-jahagirdar]

Thanks for your patience @huan233usc , the change looks fundamentally right to me I'd just tighten up the comments in the integration tests so they're a bit less verbose. Let's give it another day or so for @kevinjqliu @CTTY or any others to review before merging

[kevinjqliu]

is it better to store endpoints inside of RestCatalogConfig? The comment leads me to believe it might be a good place

    /// Runtime config is fetched from rest server and stored here.
    ///
    /// It's could be different from the user config.

[kevinjqliu]

and it seems like a lot of endpoint related functions are in there

iceberg-rust/crates/catalog/rest/src/catalog.rs

Lines 168 to 341 in 22e80a6

	impl RestCatalogConfig {
	fn url_prefixed(&self, parts: &[&str]) -> String {
	[&self.uri, PATH_V1]
	.into_iter()
	.chain(self.props.get("prefix").map(|s| &**s))
	.chain(parts.iter().cloned())
	.join("/")
	}
	fn config_endpoint(&self) -> String {
	[&self.uri, PATH_V1, "config"].join("/")
	}
	pub(crate) fn get_token_endpoint(&self) -> String {
	if let Some(oauth2_uri) = self.props.get("oauth2-server-uri") {
	oauth2_uri.to_string()
	} else {
	[&self.uri, PATH_V1, "oauth", "tokens"].join("/")
	}
	}
	fn namespaces_endpoint(&self) -> String {
	self.url_prefixed(&["namespaces"])
	}
	fn namespace_endpoint(&self, ns: &NamespaceIdent) -> String {
	self.url_prefixed(&["namespaces", &ns.to_url_string()])
	}
	fn tables_endpoint(&self, ns: &NamespaceIdent) -> String {
	self.url_prefixed(&["namespaces", &ns.to_url_string(), "tables"])
	}
	fn rename_table_endpoint(&self) -> String {
	self.url_prefixed(&["tables", "rename"])
	}
	fn register_table_endpoint(&self, ns: &NamespaceIdent) -> String {
	self.url_prefixed(&["namespaces", &ns.to_url_string(), "register"])
	}
	fn table_endpoint(&self, table: &TableIdent) -> String {
	self.url_prefixed(&[
	"namespaces",
	&table.namespace.to_url_string(),
	"tables",
	&table.name,
	])
	}
	/// Get the client from the config.
	pub(crate) fn client(&self) -> Option<Client> {
	self.client.clone()
	}
	/// Get the token from the config.
	///
	/// The client can use this token to send requests.
	pub(crate) fn token(&self) -> Option<String> {
	self.props.get("token").cloned()
	}
	/// Get the credentials from the config. The client can use these credentials to fetch a new
	/// token.
	///
	/// ## Output
	///
	/// - `None`: No credential is set.
	/// - `Some(None, client_secret)`: No client_id is set, use client_secret directly.
	/// - `Some(Some(client_id), client_secret)`: Both client_id and client_secret are set.
	pub(crate) fn credential(&self) -> Option<(Option<String>, String)> {
	let cred = self.props.get("credential")?;
	match cred.split_once(':') {
	Some((client_id, client_secret)) => {
	Some((Some(client_id.to_string()), client_secret.to_string()))
	}
	None => Some((None, cred.to_string())),
	}
	}
	/// Get the extra headers from config, which includes:
	///
	/// - `content-type`
	/// - `x-client-version`
	/// - `user-agent`
	/// - All headers specified by `header.xxx` in props.
	pub(crate) fn extra_headers(&self) -> Result<HeaderMap> {
	let mut headers = HeaderMap::from_iter([
	(
	header::CONTENT_TYPE,
	HeaderValue::from_static("application/json"),
	),
	(
	HeaderName::from_static("x-client-version"),
	HeaderValue::from_static(ICEBERG_REST_SPEC_VERSION),
	),
	(
	header::USER_AGENT,
	HeaderValue::from_str(&format!("iceberg-rs/{CARGO_PKG_VERSION}")).unwrap(),
	),
	]);
	for (key, value) in self
	.props
	.iter()
	.filter_map(|(k, v)| k.strip_prefix("header.").map(|k| (k, v)))
	{
	headers.insert(
	HeaderName::from_str(key).map_err(|e| {
	Error::new(
	ErrorKind::DataInvalid,
	format!("Invalid header name: {key}"),
	)
	.with_source(e)
	})?,
	HeaderValue::from_str(value).map_err(|e| {
	Error::new(
	ErrorKind::DataInvalid,
	format!("Invalid header value: {value}"),
	)
	.with_source(e)
	})?,
	);
	}
	Ok(headers)
	}
	/// Get the optional OAuth headers from the config.
	pub(crate) fn extra_oauth_params(&self) -> HashMap<String, String> {
	let mut params = HashMap::new();
	if let Some(scope) = self.props.get("scope") {
	params.insert("scope".to_string(), scope.to_string());
	} else {
	params.insert("scope".to_string(), "catalog".to_string());
	}
	let optional_params = ["audience", "resource"];
	for param_name in optional_params {
	if let Some(value) = self.props.get(param_name) {
	params.insert(param_name.to_string(), value.to_string());
	}
	}
	params
	}
	/// Check if header redaction is disabled in error logs.
	///
	/// Returns true if the `disable-header-redaction` property is set to "true".
	/// Defaults to false for security (headers are redacted by default).
	pub(crate) fn disable_header_redaction(&self) -> bool {
	self.props
	.get(REST_CATALOG_PROP_DISABLE_HEADER_REDACTION)
	.map(|v| v.eq_ignore_ascii_case("true"))
	.unwrap_or(false)
	}
	/// Merge the `RestCatalogConfig` with the a [`CatalogConfig`] (fetched from the REST server).
	pub(crate) fn merge_with_config(mut self, mut config: CatalogConfig) -> Self {
	if let Some(uri) = config.overrides.remove("uri") {
	self.uri = uri;
	}
	let mut props = config.defaults;
	props.extend(self.props);
	props.extend(config.overrides);
	self.props = props;
	self
	}
	}

[huan233usc]

Good catch — that comment was misleading. config here is actually the user config merged with the server's config overrides (not raw server state), so it's not quite the right home for the negotiated endpoints. I clarified both field comments to make the distinction clear. Happy to move it if you'd still prefer, though!

[CTTY]

Generally looks good! Left some questions

[CTTY]

Some comments would make this more readable like

(Method::GET, "/v1/{prefix}/namespaces"), // v1 list namespaces

We can refer to java's naming

[huan233usc]

Done — annotated each entry with its operation name. Thanks!

[CTTY]

Why do we need this api public?

[CTTY]

Also not sure if there is any blockers that are preventing us from enforcing the supported endpoint? (Checking supported endpoints for every catalog operations) Just storing the endpoints while not using them seems weird.

If we do plan to enforce them, when should we do that?

[huan233usc]

Kept it pub since it's now used by the exists checks (below) and is what scan planning will build on.

[huan233usc]

Great point. table_exists and namespace_exists now use it — they issue the HEAD only when the server advertises it, and fall back to a GET load otherwise (mirrors Java's RESTSessionCatalog). Added tests for both paths. Thanks!

[CTTY]

1. I saw we only gated table_exists and namespace_exists, but why not all other operations like list_namespaces, list_tables, etc.?

2. I'm still not sure why this has to be public. exists check doesn't require the api to be public. Maybe we can set this to pub once the scan planning feature is in place and absolutely need this. wdyt?

[kevinjqliu]

maybe we can keep this PR lean and just introduce changes related to the Endpoint, we can follow up with the gates. might make reviews easier. WDYT?

[CTTY]

1. I saw we only gated table_exists and namespace_exists, but why not all other operations like list_namespaces, list_tables, etc.?

2. I'm still not sure why this has to be public. exists check doesn't require the api to be public. Maybe we can set this to pub once the scan planning feature is in place and absolutely need this. wdyt?

[CTTY]

Should we add static endpoint for each of them, similar to Java?

pub(crate) static V1_LIST_NAMESPACES: LazyLock<Endpoint> =
    LazyLock::new(|| Endpoint::new(Method::GET, "/v1/{prefix}/namespaces"));
pub(crate) static V1_LOAD_TABLE: LazyLock<Endpoint> =
    LazyLock::new(|| Endpoint::new(Method::GET, "/v1/{prefix}/namespaces/{namespace}/tables/{table}"));
// ... one per route ...

pub(crate) static DEFAULT_ENDPOINTS: LazyLock<HashSet<Endpoint>> = LazyLock::new(|| {
    [
        &*V1_LIST_NAMESPACES,
        &*V1_LOAD_TABLE,
        // ... the rest of the base-set routes ...
    ]
    .into_iter()
    .cloned()
    .collect()
});

This would make each endpoint referable.