Skip to content

feat(dgw): zip multi-clip recording downloads - #1923

Merged
irvingouj@Devolutions (irvingoujAtDevolution) merged 4 commits into
masterfrom
agents/zip-multiple-recordings-on-download
Aug 10, 2026
Merged

feat(dgw): zip multi-clip recording downloads#1923
irvingouj@Devolutions (irvingoujAtDevolution) merged 4 commits into
masterfrom
agents/zip-multiple-recordings-on-download

Conversation

@irvingoujAtDevolution

Copy link
Copy Markdown
Contributor

Add GET /jet/jrec/pull/{id} that streams a ZIP of recording.json and every clip listed in the session manifest. Callers no longer need to guess a single clip filename when a reconnect produced multiple files. The existing per-file pull route is unchanged for player and granular access.

Add GET /jet/jrec/pull/{id} that streams a ZIP of recording.json and every
clip listed in the session manifest. Callers no longer need to guess a
single clip filename when a reconnect produced multiple files. The
existing per-file pull route is unchanged for player and granular access.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

Let maintainers know that an action is required on their side

  • Add the label release-required Please cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module) when you request a maintainer to cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module)

  • Add the label release-blocker Follow-up is required before cutting a new release if a follow-up is required before cutting a new release

  • Add the label publish-required Please publish libraries (`Devolutions.Gateway.Utils`, OpenAPI clients, etc) when you request a maintainer to publish libraries (Devolutions.Gateway.Utils, OpenAPI clients, etc.)

  • Add the label publish-blocker Follow-up is required before publishing libraries if a follow-up is required before publishing libraries

@irvingoujAtDevolution

Copy link
Copy Markdown
Contributor Author

Implementation notes:

  • New route: GET /jet/jrec/pull/{id} beside the existing GET /jet/jrec/pull/{id}/{filename}.
  • Auth is the same JREC pull token / jet_aid check as per-file pull.
  • ZIP membership is manifest-driven: always recording.json, then each safe files[].fileName that exists on disk.
  • Always ZIP (including single-clip sessions) so clients have one download contract.
  • Streamed with async_zip over a duplex pipe (AsyncReadBody); entries use STORED compression because WebM/etc. are already compressed.
  • Active sessions are allowed; the last clip may still be growing.
  • OpenAPI + generated TS/dotnet clients updated (PullRecordingSession).
  • Unit tests cover path safety, manifest filtering, and ZIP contents.

Note

LLM-assisted content (no human feedback).

Map missing or corrupt manifests to 404, require pull operation tokens,
fail the HTTP body on mid-stream packaging errors, and stop ZIP work on
gateway shutdown so clients do not treat truncated archives as success.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@irvingoujAtDevolution

Copy link
Copy Markdown
Contributor Author

Addressed code-review findings in 50a3b652:

  • Missing/corrupt recording.json404 (not 500); removed dead empty-entries branch
  • Mid-stream ZIP failures now yield a body stream error (no clean EOF on truncated archive)
  • ZIP task honors shutdown_signal
  • Both pull routes require jet_rop == pull
  • Small cleanups: ZIP_CHUNK_SIZE, is_dir/is_file only, concrete DuplexStream writer
  • Extra unit tests for not-found manifest, corrupt manifest, and mid-stream open failure

Note

LLM-assisted content (no human feedback).

@irvingoujAtDevolution
irvingouj@Devolutions (irvingoujAtDevolution) marked this pull request as ready for review August 10, 2026 19:14
Copilot AI balanced review requested due to automatic review settings August 10, 2026 19:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds authenticated ZIP downloads for complete JREC sessions, including the manifest and all available clips.

Changes:

  • Adds the session ZIP streaming endpoint.
  • Adds ZIP generation and validation tests.
  • Regenerates OpenAPI documentation and clients.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
devolutions-gateway/src/openapi.rs Registers the endpoint in OpenAPI.
devolutions-gateway/src/api/jrec.rs Implements ZIP creation and streaming.
devolutions-gateway/openapi/ts-angular-client/api/jrec.service.ts Adds the Angular client method.
devolutions-gateway/openapi/gateway-api.yaml Documents the endpoint contract.
devolutions-gateway/openapi/dotnet-client/src/Devolutions.Gateway.Client/Api/JrecApi.cs Adds .NET client methods.
devolutions-gateway/openapi/dotnet-client/README.md Lists the new .NET API.
devolutions-gateway/openapi/dotnet-client/docs/JrecApi.md Documents .NET usage.
devolutions-gateway/openapi/doc/index.adoc Adds generated API documentation.
devolutions-gateway/Cargo.toml Adds async_zip.
Cargo.lock Locks the new dependency graph.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread devolutions-gateway/src/api/jrec.rs Outdated
return Err(HttpError::not_found().msg("requested recording does not exist"));
}

let entries = match list_recording_zip_entries(&recording_dir).await {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 2013b996.

Download membership is now a one-shot snapshot (RecordingZipPlan): we read recording.json once, derive the clip list from those bytes, and archive those exact bytes as the ZIP’s recording.json (never re-open the live file). A reconnect that rewrites the on-disk manifest after the snapshot cannot widen or desync the package. Clips are still opened once at package time from that frozen name list.

Added regression test zip_keeps_snapshotted_manifest_when_disk_manifest_changes.

Note

LLM-assisted content (no human feedback).

Comment thread devolutions-gateway/src/api/jrec.rs Outdated
Comment on lines +804 to +806
let builder = ZipEntryBuilder::new(file_name.clone().into(), Compression::Stored);
let mut entry_writer = zip_writer
.write_entry_stream(builder)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 2013b996.

Replaced async_zip streaming entries with the standard zip crate writing STORED entries to a temp file (known sizes/CRC in local headers), then streaming that finished archive. This avoids data-descriptor / null-CRC local headers that some OS unzippers reject.

Interop coverage: streams_interoperable_zip_with_all_listed_clips round-trips through zip::ZipArchive (not the writer’s own reader).

Note

LLM-assisted content (no human feedback).

Refuse session package pulls that exceed 128 files or 2 GiB uncompressed
with HTTP 413 so pathological manifests cannot pin the gateway on huge
bulk transfers while normal multi-clip sessions stay unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Address review feedback on session package downloads: freeze the
recording.json bytes that define membership so reconnects cannot widen
the archive mid-download, and build STORED entries with the zip crate so
local-file headers carry real sizes/CRCs accepted by common unzip tools.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@irvingoujAtDevolution
irvingouj@Devolutions (irvingoujAtDevolution) merged commit e34e0a4 into master Aug 10, 2026
42 checks passed
@irvingoujAtDevolution
irvingouj@Devolutions (irvingoujAtDevolution) deleted the agents/zip-multiple-recordings-on-download branch August 10, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants