RANGER-5655: Shared partition plan library, DB patch 078, plugin SPIFFE auth

[ramackri]

Summary

Foundation for RANGER-5655 — Admin-managed Kafka audit partition plan and SPIFFE workload identity. This PR combines the first three merge slices (PR1 + PR2 + PR5 from the feature split); Admin REST (PR3) and Ingestor routing/inbound SPIFFE (PR4) follow in separate PRs.

Parent: RANGER-5655 — Ranger Admin-Managed Kafka Audit Partition Plan & SPIFFE Auth

Sub-tasks covered in this PR:

• RANGER-5719 — Shared partition-plan library and SPIFFE header auth utilities
• RANGER-5720 — DB patch 078 for audit partition plan global state
• RANGER-5723 — Plugin SPIFFE outbound auth for audit-server destination

Sub-tasks deferred to follow-on PRs:

• RANGER-5721 — Admin-managed audit partition plan REST and lifecycle hooks
• RANGER-5722 — Ingestor dynamic routing, SPIFFE inbound auth, and Admin plan poller

SPIFFE outbound auth — aligned with RANGER-5700 (#1096)

• Outbound auth is handled in Java only (RangerAuditServerDestination + PluginHeaderAuthConfig + SpiffeIdentityResolver + RangerRESTClient.setTrustedAuthHeaders()), using the audit destination prefix (JWT-style), not ranger.<service>.authn.* security props.
• This mirrors #1096: configure auth on the component that owns the HTTP flow, not in every plugin security config.

Audit XML properties for outbound SPIFFE (plugin → audit ingestor)

All SPIFFE outbound settings live under the audit-server destination prefix in ranger-<service>-audit.xml:

xasecure.audit.destination.auditserver

Property	Required?	Default	Purpose
xasecure.audit.destination.auditserver.authn.header.enabled	Yes (to enable SPIFFE)	false	Master switch; SPIFFE headers are not sent unless set to true
xasecure.audit.destination.auditserver.authn.header.spiffe	No	X-Spiffe-Id	HTTP header name for the workload SPIFFE ID
xasecure.audit.destination.auditserver.authn.spiffe.value	No	(unset)	Explicit SPIFFE ID (highest priority)
xasecure.audit.destination.auditserver.authn.spiffe.file	No	/var/run/secrets/spiffe.io/identity/spiffe	SPIRE/agent identity file (first non-blank line)

SPIFFE ID resolution order (when authn.header.enabled=true):

1. authn.spiffe.value in audit XML
2. authn.spiffe.file in audit XML (or default SPIRE path above)
3. SPIFFE_ID environment variable

Minimum enable (typical Kubernetes / SPIRE deployment — identity from file or env):

<property>
  <name>xasecure.audit.destination.auditserver.authn.header.enabled</name>
  <value>true</value>
</property>

Explicit SPIFFE ID (e.g. docker/dev without SPIRE file):

<property>
  <name>xasecure.audit.destination.auditserver.authn.header.enabled</name>
  <value>true</value>
</property>
<property>
  <name>xasecure.audit.destination.auditserver.authn.spiffe.value</name>
  <value>spiffe://example.com/ns/ranger/sa/hive</value>
</property>

Optional custom header name:

<property>
  <name>xasecure.audit.destination.auditserver.authn.header.spiffe</name>
  <value>X-Spiffe-Id</value>
</property>

Also required for audit delivery (unchanged by this PR; not SPIFFE-specific):

• xasecure.audit.destination.auditserver = true
• xasecure.audit.destination.auditserver.url = ingestor base URL (e.g. http://ranger-audit-ingestor:7081)

Defaults / safety: authn.header.enabled defaults to false in code (PluginHeaderAuthConfig.isHeaderAuthEnabled). No install-template or enable-script changes are required — operators opt in via audit XML only. Safe to merge before PR4 (ingestor inbound SPIFFE) lands.

• Per-service audit allow-list on the ingestor comes from policy.download.auth.users in the partition plan (PR3/PR4), not static ingestor XML
• AuditPluginIdResolver intentionally omitted here; deferred to PR3 with AuditPartitionPlanMgr

Test plan

• mvn -pl agents-common -am test — partition plan + SPIFFE unit tests pass
• mvn -pl agents-audit/dest-auditserver -am test -Dtest=RangerAuditServerDestinationTest — pass
• CI: build-17, plugins-docker-build, services-docker-build, plugins-download-archives
• DB patch 078 SQL included for all supported DB backends (validated via compile/upgrade path in CI)
• End-to-end plugin → ingestor SPIFFE POST requires follow-on PR4 (AuditHeaderAuthFilter)

[ramackri]

Closing in favor of three focused PRs split from this branch:

• RANGER-5719: Shared partition-plan library and SPIFFE header auth utilities #1137 — RANGER-5719: Shared partition-plan library and SPIFFE header auth utilities
• RANGER-5720: DB patch 078 for audit partition plan global state #1138 — RANGER-5720: DB patch 078 for audit partition plan global state
• RANGER-5723: Plugin SPIFFE outbound auth for audit-server destination #1139 — RANGER-5723: Plugin SPIFFE outbound auth for audit-server destination (RANGER-5719: Shared partition-plan library and SPIFFE header auth utilities #1137 is a prerequisite)

Merge order: #1137 and #1138 can land in parallel; #1139 after #1137 (rebase onto master).