Skip to content

feat: add archive test source for ZIP and tar.gz files#179

Merged
skylenet merged 10 commits into
masterfrom
archive-source
Apr 2, 2026
Merged

feat: add archive test source for ZIP and tar.gz files#179
skylenet merged 10 commits into
masterfrom
archive-source

Conversation

@skylenet
Copy link
Copy Markdown
Member

@skylenet skylenet commented Apr 2, 2026

Summary

  • Add new archive source type under tests.source that downloads and extracts ZIP or tar.gz archives from URLs or local paths, then discovers tests via glob patterns within extracted contents
  • Automatically detect and convert GitHub Actions artifact browser URLs to API calls with bearer token auth
  • Support parallel range-request downloads (8 workers, 25 MiB chunks) with automatic fallback to sequential when the server doesn't support range requests
  • Cache downloaded archives locally so repeated runs with the same URL skip the download
  • Auto-extract inner tarballs found inside ZIP files (e.g. GitHub Actions artifacts)
  • Pass BENCHMARKOOR_RUNNER_GITHUB_TOKEN into the Docker container in the GitHub Action
  • Fix null tests array crash in UI suite cell tooltip

Example config

tests:
  source:
    archive:
      file: https://github.com/NethermindEth/gas-benchmarks/actions/runs/123/artifacts/456
      pre_run_steps:
        - "perf-devnet-3/gas-bump.txt"
      steps:
        setup:
          - "perf-devnet-3/tests/setup/*.txt"
        test:
          - "perf-devnet-3/tests/test/*.txt"
        cleanup:
          - "perf-devnet-3/tests/cleanup/*.txt"

Test plan

  • Config validation tests (valid archive, missing file, multiple sources)
  • IsConfigured() test for archive source
  • Local ZIP and tar.gz extraction tests
  • Inner tarball auto-extraction test
  • GitHub artifact URL detection and conversion test
  • Download caching test (second run skips download)
  • Parallel range-request download test with httptest server
  • Sequential fallback test
  • Bearer token forwarding test
  • Format detection by extension and magic bytes
  • Race detector clean on parallel download

skylenet added 10 commits April 2, 2026 11:10
Add a new `archive` source type under `tests.source` that supports
downloading and extracting ZIP or tar.gz archives from URLs or local
paths, then discovering tests via glob patterns within the extracted
contents. Inner tarballs (e.g. GitHub Actions artifacts) are
automatically extracted.
GitHub Actions artifact browser URLs return 404 for plain HTTP GET.
Detect the URL pattern and convert to the GitHub API download endpoint
with bearer token authentication.
Log total file size at download start, periodic progress every 10 MiB
with percentage (when Content-Length is known), and final size on
completion. Also warn when GitHub artifact URL is detected without a
configured token.
Probe the server with a HEAD request for Accept-Ranges support. When
supported and the file is >= 10 MiB, download in parallel using 4
workers with 25 MiB chunks. Falls back to sequential download when
the server does not support range requests or the file is small.
Progress logging works across concurrent workers using atomic counters.
Hash the configured URL to produce a stable cache key in the cache
directory. On subsequent runs with the same URL, the cached file is
reused without making any HTTP requests. Downloads use atomic rename
to prevent partial cache files.
@skylenet skylenet merged commit e2b38c0 into master Apr 2, 2026
7 checks passed
@skylenet skylenet deleted the archive-source branch April 2, 2026 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant