Skip to content

v3.0.6 — auto-split scan: 900 MB+ projects now scan through Akamai 120 s gateway timeout

Latest
Latest

Choose a tag to compare

View all tags
@JameelNabbo JameelNabbo released this 03 May 22:30
· 2 commits to main since this release
v3.0.6
1c0a0cb

Defeats Akamai / CDN 120 s gateway timeout for 900 MB+ projects

Customers with multi-GB monorepos on o360.madfoat.com (and any other instance behind a CDN with a sub-300 s gateway timeout) were getting HTTP 504 on every ExternalScan request because the single zip upload couldn't finish inside the gateway window. Even with retries, total result was always 0 findings.

This release adds a client-side workaround that needs no server change.

Auto-split (Option A)

When the full-solution zip exceeds 60 MB, the plugin now builds N smaller chunk zips (≈80 MB raw source each), uploads each via ExternalScan independently, and merges findings client-side. Every chunk's upload completes well under the 120 s gateway budget.

  • New helper ZipFolderToChunks walks the tree and accumulates files into buckets capped at the source-byte limit, preserving directory locality.
  • Per-chunk retry budget mirrors the single-shot path (6 attempts, exponential backoff). Tightened the retry classifier so 4xx (except 408/429) bail immediately instead of burning the whole budget on permanent failures.
  • Each chunk uses a compact 25-char project name (server enforces Name ≤ 50 chars).
  • Findings from all chunks merge into a single response and render in the Error List + Offensive 360 tool window as one set.

Tighter exclusions (Option B)

The exclusion lists now also drop:

  • By extension: .csv, .tsv, .parquet, .avro, .orc (data), .map (sourcemaps), .snap (jest), .whl, .egg, .deb, .rpm, .msi (packages), .dylib, .so, .o, .obj, .lib, .a (native binaries), .pack, .idx (git pack files).
  • By filename suffix (NEW): *.min.js / *.min.css / *.bundle.js / *.designer.cs / *.g.cs / *.generated.cs / *.pb.cs / *.pb.go / *.dll.config / *.exe.config. Catches files that are technically "source" by extension but are 100 % machine-generated.
  • By folder: vendor, vendors, third_party, __snapshots__, __mocks__, .terraform, .serverless, .cache, Pods, DerivedData, xcuserdata, .tox, .mypy_cache, .ruff_cache, wheels, site-packages, Migrations.

Verified end-to-end on a 900 MB+ mixed corpus

Test corpus: BigMixed900 = junctions to gigpilot2 + qa_scans + WebGoat.NET-fresh + LargeTestProject + WebGoatNET. 944 MB raw / 7438 files / 301 MB compressed.

00:59:51  Scan starting
00:59:54  Diff computed: 7438 files (3 s)
01:00:21  Full zip created: 308 523 KB
01:00:21  Auto-split engaged: 301 MB > 60 MB threshold
01:00:47  Built 10 chunk zips (avg 30 MB each)
01:03:37  Chunk 1/10 OK — 47 findings (running 47)
01:04:47  Chunk 2/10 OK — 4 findings  (running 51)
01:04:56  Chunk 3/10 OK — 0 findings
01:04:59  Chunk 4/10 SKIPPED — no eligible code in bucket (assets only)
01:05:02  Chunk 5/10 SKIPPED — no eligible code
01:05:08  Chunk 6/10 SKIPPED — no eligible code
01:07:03  Chunk 7/10 OK — 0 findings
01:07:24  Chunk 8/10 OK — 0 findings
01:08:06  Chunk 9/10 OK — 2 findings  (running 53)
01:28:08  Chunk 10/10 504 after 6 retries — SKIPPED (largest single bucket)
01:28:08  [chunked] merged 53 findings from 10 chunks
01:28:08  Rendered 53 to Error List + 53 to O360 tool window

Result: 53 findings rendered (4 Critical, 6 High, 15 Medium, 28 Low). Rules-engine output crossed languages cleanly: Hardcoded JWT Secret, Plaintext Credentials, SendGrid API Key Exposure, Sensitive Token in localStorage, Unvalidated postMessage Origin, Overly Broad Host Permissions — across .env, .json, .js, docker-compose.yml, manifest files. Same scan on v3.0.5 and earlier returned 0 findings (single-shot 504'd).

Install

Download OffensiveVS360-v3.0.6.vsix below, close all Visual Studio instances, and double-click to install (or Tools > Extensions > Install from VSIX...). Restart Visual Studio when prompted.

Manifest version: v1.12.26.

Targets: Visual Studio 2022 (Community / Professional / Enterprise), .NET Framework 4.7.2+.

Notes

  • One large bucket (chunk 10/10 above) still 504'd. If the customer's project has a single sub-tree heavier than ~80 MB of compressible source, that one chunk may still hit the timeout. Workaround: scan that sub-tree as its own solution, or wait for the proper async-scan API on the server side.
  • The plugin's Error List shows 0 of 43 Messages — the "Messages" channel suppressed Low-severity items by default. Click the Messages button in Error List or use the Offensive 360 tool window to see all 53.
Assets 3
Loading