Skip to content

RDMA Operations

Ben McClelland edited this page Sep 18, 2026 · 2 revisions

This page covers observability and administration that exist only in the vgwrdma binary. For gateway-wide metrics and admin APIs, see the general Metrics and Admin APIs pages. The client side of the v2 interactions is implemented by ROCm hipObject.

Access logging, metrics, and events

hipobj-rc-v2 transfers enabled by --rdma-rc-enable feed the same configured access-log, metrics, and event-notification sinks as the standard S3 path. One terminal record is published for the logical object operation:

  • the access-log action is PutObject or GetObject, with the logical bucket/object path and actual transferred byte count;
  • StatsD/DogStatsD metrics use action:PutObject or action:GetObject, include the logical bucket tag, and increment the corresponding success/failure and bytes-written/bytes-read counters;
  • a PUT that committed the backend object emits an ObjectCreated:Put event, even if a later native finalization step reports an error.

Successful PREPARE, READY, and CANCEL HTTP control calls are not counted as separate S3 operations. Failed requests that do not create a session and sessions that fail, expire, or are canceled are published as failures when an operational sink is configured. A malformed or unauthenticated control request that fails before session creation is therefore still observable.

Publication occurs once per session and is drained before the operational sinks close during gateway shutdown.

Native RC diagnostics

Native RC diagnostics use the gateway logger. Normal mode reports native errors. --debug enables detailed RC diagnostics in addition to the normal gateway debug output.

Use the standard vgwrdma options to configure the operational sinks:

  • --access-log or --log-webhook-url for access records;
  • --metrics-statsd-servers and --metrics-dogstatsd-servers for metrics;
  • the event-notification options for object-created events.

Live-session admin endpoint

With both --rdma-rc-enable and --admin-port configured, vgwrdma adds a diagnostic route to the admin listener:

  • Method: GET
  • Endpoint: /rc-sessions
  • Authentication: SigV4 for service s3; root or admin role required
  • Successful content type: JSON

The route is not registered when RC v2 is disabled, and no admin server exists unless --admin-port is set.

{
  "sessions": [
    {
      "session_id": "example-session-id",
      "op": "GET",
      "target": "/bucket/object",
      "state": "transferring",
      "reap_pending": false,
      "age_ms": 125,
      "staging_bytes": 1048576
    }
  ]
}

The fields are:

Field Meaning
session_id RC v2 session identifier
op Logical GET or PUT operation
target Logical /bucket/object target
state Current native session state
reap_pending Whether teardown/reaping is pending
age_ms Session age in milliseconds
staging_bytes Bytes currently reserved for staging

Possible states are prepared, publishing, transferring, completing, reaping, and unknown.

The response is a point-in-time snapshot for diagnosis and capacity planning, not a historical transfer log. Completed sessions disappear and should be tracked through access logs and metrics instead. Error responses use the normal XML S3 error format.

Resource-limit and timeout configuration is documented in the RDMA User Guide.

Clone this wiki locally