Skip to content

Stream log exports row-by-row to avoid unbounded memory buffering - #13

Open
DeveloperDurp wants to merge 1 commit into
mainfrom
codex/propose-fix-for-log-export-vulnerability
Open

Stream log exports row-by-row to avoid unbounded memory buffering#13
DeveloperDurp wants to merge 1 commit into
mainfrom
codex/propose-fix-for-log-export-vulnerability

Conversation

@DeveloperDurp

Copy link
Copy Markdown
Owner

Motivation

  • The plaintext export endpoint previously loaded all deployment log rows and concatenated them into a single in-memory buffer, which can exhaust server memory for large logs and enable a DoS-style availability issue.
  • The goal is to produce the same textual export but without materializing the full result set in process memory.

Description

  • Added Repository.ForEachDeploymentLogByDeploymentAsc which streams rows via QueryContext and invokes a callback per db.DeploymentLog to avoid building a large slice. (internal/repository/repository.go).
  • Reworked the web handler ExportLogs to write the file header and then stream each formatted log line directly to the http.ResponseWriter using the new ForEachDeploymentLogByDeploymentAsc callback instead of ListDeploymentLogsByDeployment + strings.Builder (removes unbounded buffering). (internal/handler/logs.go).
  • Applied the same streaming change to the API handler ExportLogs so both web and API export paths now stream rows row-by-row. (internal/handler/api/logs.go).

Testing

  • Ran formatting/linters: gofmt -l and git diff --check on the modified files (passed).
  • Attempted to run the TestExportLogs unit tests via go test -run 'TestExportLogs' ./internal/handler ./internal/handler/api, but running the tests is blocked in this environment because template-generated packages (templ outputs) are missing and templ generate cannot be executed here (templ not installed and module proxy access restricted); therefore the export tests could not be executed end-to-end in this session.
  • Verified the change compiles locally for the edited files and committed the patch (Stream log exports without buffering).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant