[Community Benchmark] [v3.7.4] help us build a real-world protocol comparison page

[axpnet]

Why we are doing this

When AeroFTP says "protocol X is faster than protocol Y on provider Z", that claim should be defensible. Real-world transfer speed depends on your ISP, your distance from the provider region, time of day, your TLS stack, even your MTU. No single test machine can cover that space.

So we are turning the protocol comparison page into a community effort. This issue is the entry point.

What is shipping

A new CLI subcommand, aeroftp-cli benchmark, available since AeroFTP v3.7.3 and stabilized in v3.7.4. It runs a standardized matrix of upload, download, list, stat and delete operations against any saved profile you choose (your own account, your own credentials, your own bandwidth) and emits a strictly anonymized JSON report you can paste into a benchmark report Issue.

We then aggregate the reports (median, p95, stddev, never the misleading arithmetic mean), filter outliers, and publish a public protocol comparison page based on what real users measure, not what one developer measures.

Full contributor guide: docs/COMMUNITY-BENCHMARK.md. It explains every level, every field of the report, what is collected, what is not, and how to submit.

What we collect, what we do not

The CLI runs a sanitization pass before writing the report. If the pass cannot anonymize a field, it refuses to produce the report rather than risk a leak.

Collected:

• Time-of-day bucket (morning, afternoon, evening, night)
• Coarse region and connection type that you pick from a short dropdown in the issue form
• OS family + architecture + coarse CPU class (e.g. linux x86_64 x64-modern)
• Protocol type and a coarse provider hint (e.g. amazon-s3-eu-west-1)
• Throughput, latency, TLS handshake time as statistical aggregates (p50, p95, stddev, min, max)
• AeroFTP CLI version + Rust target triple

Never collected, ever:

• Bucket / container / drive name
• Hostname, IP address, MAC address
• Path, directory, filename
• Username, email, account ID
• Access keys, refresh tokens, OAuth secrets
• Exact timestamps (rounded to the hour)
• CPU model, exact OS version, kernel build
• Local IP or network topology

If you want to contribute even more privately, the --anonymize-extra flag will hash the provider hint as well, so even "this person uses S3" is not visible in the published comparison page.

A first reference dataset is already published

Before asking other people to run the matrix, the maintainer ran it on every profile saved on the dev host. The result is a 32-report dataset you can read alongside this issue:

• 27 profiles with the full matrix (1 MiB + 10 MiB + 100 MiB + 1 GiB, 3 runs each, 5 operations).
• 5 profiles partial (capped at the provider's free-tier max).
• 3 profiles fixed in v3.7.4 thanks to bugs surfaced by the sweep itself (Filen native chunked upload, Yandex transient upload-target race, Drime list() mutating current_path).

The dataset bundle, the rankings, the per-profile JSON reports and the maintainer-side REPORT.md live in docs/dev/benchmarks/2026-05-07_community-benchmark-v3.7.3/ (see the repo). The raw JSON is intentionally gitignored: it is attached to this issue thread instead, alongside community submissions, so we can curate them together.

The dataset is explicitly a single-host, single-uplink, single-timezone sample. It is not a population-level benchmark. Selection bias and the rationale are spelled out in §9 of the maintainer's report.

How to submit

Three steps, all inside GitHub, no third-party form.

1. Run the benchmark locally

aeroftp-cli --profile "Your Profile" benchmark standard \
  --consent-publish --report benchmark.json

--consent-publish wraps the JSON in BEGIN AEROFTP BENCHMARK REPORT / END AEROFTP BENCHMARK REPORT markers and runs the sanitization sweep. Levels: quick (~30 s), standard (~5 min), deep (~30 min, more useful but consumes more quota).

2. Open the benchmark report issue template

Click here to open a pre-filled benchmark report issue.

The template asks for two pieces of coarse metadata the CLI cannot infer:

• Coarse region picked from a short list (Western Europe, Eastern Europe, North America (East/West), South-East Asia, ...). No country, no city, no postcode.
• Connection type picked from a short list (fiber, DSL, mobile, corporate, data center, unknown). No ISP name, no plan tier.

Then there is one big text area: paste the contents of benchmark.json (everything between the BEGIN / END markers). Submit.

3. That is it

The issue lands in this repo with the benchmark-report label. A maintainer reads it, runs the sanitization grep again as a second-pass safety check, and merges the report into the aggregation set that feeds the future public comparison page. You can comment further or close the issue right after, your call. Your GitHub handle is the only thing tying you to the report; if you want even that off the record, file the issue from a throwaway account.

Honest disclaimers

• Selection bias is real: people who self-select to run benchmarks tend to have above-average connections. The published page will say so. We over-represent fiber and tech-forward users. If you are on a slow link, mobile hotspot, corporate proxy, or geographically far from your provider's region, we want your data even more than the fiber crowd's.
• Provider rate limits: the CLI rate-limits itself, but if you run deep against a free-tier account, please use a non-critical bucket. We are not responsible if a provider rate-limits you, although in practice this is unlikely at the limits we set.
• No realtime dashboard yet: this is Phase 1. We collect via issue submissions, manually curate, and update the docs page. If Phase 1 produces good data, Phase 2 (automated submission, public dashboard) follows.
• One-developer project: if we get fewer than 5 submissions in 2 months, this initiative is closed as wontfix and the page reverts to a manual qualitative comparison. We would rather be honest than maintain dead infrastructure.

What is already known to work and what is not

Validated by the maintainer's reference round (full results in dataset/rankings/rankings-custom1g.md):

• Full matrix at 1 GiB: AWS, Cloudflare R2, Alibaba OSS, S3 Storj, Tencent COS, Google S3, Mega S3, S3Drive, Drime, Internxt (within quota), Jotta, Koofr (native + WebDAV), FileLu, MEGA, Google Drive, Dropbox, OneDrive, pCloud, Azure Blob, FeliCloud, S5 FileLu, aeroftp.app FTPS.
• Capped by the provider, not by us: jianguoyun (refuses 1 G), DriveHQ free WebDAV, Box (250 MB cap on free), Internxt (10 GB total free), Zoho WorkDrive (free tier 1 G refused, 100 M OK).
• Out of scope this round: kDrive and SeaFile WebDAV both refuse operations on the root path and need a sub-path matrix; idrive S3 and InfiniCloud JP need a longer per-profile timeout flag than the current sweep allows. These are tracked separately and will join the next round.

Want to sign up before you run it?

Optional, just helps us know who is interested before v3.7.4 packages reach every distribution channel. Comment with this template (every field is optional):

Coarse region:    e.g. "Western Europe" / "South-East Asia" / "North America (East)"
Connection type:  e.g. fiber / DSL / mobile / corporate / data center
Providers I can test:    e.g. AWS S3, Google Drive, Dropbox, self-hosted Nextcloud
Levels I am comfortable running:    quick / standard / deep
Anything else you want us to know:

That signup is not a binding commitment. It just helps us ping you when v3.7.4 packages are live everywhere.

Timeline

• 2026-05-06: APPENDIX, JSON schema v1, contributor guide, issue template prepared for v3.7.3.
• 2026-05-07 / 08: maintainer reference round (32 sanitized reports, 5 bug fixes shipped in v3.7.3 patch + v3.7.4).
• After v3.7.4 packages are available everywhere: first community benchmark report issues open, real data collection begins.
• 2026-07-08 target check: aggregation, public docs page, decision on Phase 2.

We will update this issue as the timeline firms up.

Questions?

Drop them in the comments. The most common ones (privacy, time commitment, what gets shared, what does not) we will fold into a FAQ section right under this section.

Thanks for considering this. AeroFTP is built in the open, and a community-validated benchmark page is rare in file-transfer tooling. Your data, even one report, makes it possible.

[axpnet]

Maintainer reference baseline (2026-05-07 / 08)

Before opening this issue I ran the same matrix on every saved profile on the development host. Posting the result here as a starting point so the community can compare and contrast their own runs.

Attachments

• REPORT.pdf: 7-page A4 maintainer-side report with rankings, fix list, methodology and bias disclaimers.
• aeroftp-bench-2026-05-07-baseline.zip: raw sanitized dataset (35 JSON reports, 4 CSV summaries, 1 markdown rankings, 3 manifest files). Every report passed sanitization: no hostnames, paths, credentials, bucket names, or account ids.

Coverage

• 27 profiles with the full matrix (1 MiB + 10 MiB + 100 MiB + 1 GiB, 3 runs each, 5 operations).
• 5 profiles partial, capped at the provider's free-tier max (e.g. Box 250 MB, MyZoho 100 MB).
• 3 profiles fixed in v3.7.4 thanks to bugs surfaced by the sweep itself:
  • Filen native: chunked AES-GCM upload (22a4bd8f) + per-chunk retry on egest body decode (d16a63cc).
  • Yandex Disk: transient upload-target retry with fresh signed URL (post-v3.7.4 commit cb1e80b6).
  • Drime: list() no longer mutates current_path (253f2cc2).
  • Verify-round JSON for those three is in the bundle as *_v374-verify.json.

Rough top-3 from this baseline (1 GiB payload, p50)

Direction	🥇 #1	🥈 #2	🥉 #3
Upload	pCloud (281.8 Mbps)	Jotta (271.2)	Dropbox (240.5)
Download	Google S3 (585.1 Mbps)	Mega S3 (584.9)	Tencent COS (569.3)

Full per-payload tables in dataset/rankings/rankings-custom1g.md (inside the zip) and in section 5 of the PDF.

Honest disclosure

This baseline is a single-host, single-uplink, single-timezone sample (Western Europe fiber, Linux x86_64). It is published as a starting point, not as a population-level benchmark. The future protocol comparison page on docs.aeroftp.app will only draw protocol-level conclusions after aggregating community submissions across at least three regions and two connection types. Any conclusion drawn from this baseline alone is provisional.

How to add yours

aeroftp-cli --profile "<name>" benchmark standard --consent-publish --report bench.json

Then comment below with the JSON pasted between the BEGIN / END markers, or open the issue template linked in the body above. Even one report from a different region or connection type is genuinely useful.

[axpnet]

The baseline is now also browsable on the docs site, with all 188 throughput data points rendered as inline bar charts (no PDF download needed):

docs.aeroftp.app/test-reports/community-benchmark/2026-05-07

Same content as the PDF and zip already attached, just laid out for the browser. The page lives under a new "Performance and benchmarks" section in the test-reports index, separate from the existing capability matrices.

Future rounds will land under /test-reports/community-benchmark/<date>/ as community submissions accumulate.

[axpnet]

Follow-up: cross-validation against rclone (and a bug we found in ourselves)

After publishing the baseline I ran a head-to-head between aeroftp-cli benchmark and rclone copyto on the same vault profiles (Cloudflare R2, Alibaba OSS, S3 Storj, Azure Blob), 100 MiB payload, 1 warmup + 3 timed runs, single thread, single connection, sequential. The point was specifically to check whether the numbers in the baseline reflect reality, or a quirk of our own harness.

The cross-check surfaced a real bug on our side, which I want to flag honestly before pointing at any chart:

aeroftp's S3 upload defaulted to multipart above 5 MiB. On the Storj S3 gateway specifically, the resulting 20-part segment layout was measurably slower to read back than rclone's single-PUT default (rclone uses a 200 MiB cutoff).

On a 100 MiB GET this showed up as a -56% download gap against rclone on Storj. Cross-tool A/B confirmed it was the upload side, not the download side:

Pattern	Time	Throughput
rclone uploaded → aeroftp downloaded	2.4 s	340 Mbps
aeroftp uploaded → aeroftp downloaded	5.0 s	170 Mbps
rclone uploaded → rclone downloaded	1.9 s	421 Mbps
aeroftp uploaded → rclone downloaded	4.7 s	170 Mbps

Same download code, different file: the slowness lived in who uploaded, not who downloads. The fix is MULTIPART_THRESHOLD: 5 MiB → 200 MiB plus a default part size of 16 MiB so larger files still split into a manageable number of parts. It is on main as commit 5160ad1b and ships in the v3.7.5 release we are cutting now.

Post-fix numbers (100 MiB, 3 timed runs, p50)

Provider	aeroftp upload	rclone upload	aeroftp download	rclone download
Cloudflare R2	241.5 Mbps	170.5 Mbps	313.4 Mbps*	407.2 Mbps*
Alibaba OSS	290.5 Mbps	247.5 Mbps	562.1 Mbps	443.8 Mbps
S3 Storj	270.4 Mbps	239.7 Mbps	452.5 Mbps	381.3 Mbps
Azure Blob	282.4 Mbps	210.3 Mbps	46.3 Mbps	50.4 Mbps

*R2 download: Cloudflare R2 sits behind an edge cache that is server-side and applies to any HTTP client. Verified symmetric across both tools — aeroftp on a warm path: 0.6 s (~1.4 Gbps); rclone on a fresh aeroftp-uploaded path: 2.1 s (~390 Mbps). The numbers in the table reflect the path-warmth profile of each harness, not a client-side limitation.

Methodology

• Same 100 MiB random binary used by both tools.
• aeroftp: aeroftp-cli benchmark custom --pre-delete --runs 3 (each upload is a fresh PUT, never a same-size HEAD short-circuit).
• rclone: rclone copyto --ignore-times --s3-no-check-bucket --transfers=1 --checkers=1 --no-traverse plus an explicit rclone deletefile between runs (mirror of our --pre-delete).
• Sequential. We re-ran Storj in inverse order (rclone first, aeroftp second) to rule out connection warm-up bias; results held.

Take-away for anyone running their own round

You can absolutely run this comparison yourselves with both clients side by side. With v3.7.5 the numbers should land at parity or in aeroftp's favour on every S3 backend in this round. If you find a backend where they don't, please post the raw JSON and the rclone config, I'd rather hear about a real gap than a ghost one.

If you want to script it, the new aeroftp-cli export rclone (also in v3.7.5) writes a working rclone.conf from your saved profile vault in one command, so you don't have to retype credentials.

[EhudKirsh]

I ran some benchmarks using the GUI Speed Test button in v3.8.5 on Windows 10. It only allows me to compare WebDAV and S3.
I'm curious if it's currently possible to run an ideal benchmark also on API and OAuth cloud drives like you mention with aeroftp-cli in the GUI, either in this Speed Test button/window, or in the Debug Panel. I think it's worth mentioning, and enabling such option.

I ran 100MB with SHA-256 and 2 parallel runs twice. For some reason, only Filen WebDAV but not S3, S3Drive and MEGAcmd withstood this in both runs without failing. Not sure what to make of this, other than the fact that we already know that OpenDrive fails on 100MB files and bigger for free-tier accounts: https://www.opendrive.com/pricing. If some of these failed due to the file size, it'd be interesting to add this to the list of max file sizes in the docs on Chunking: #272 (comment). The 2nd run felt stuck like it was going to take forever so I stopped it. Here are the results of the first:

aeroftp-speedtest-1780055306040.json
aeroftp-speedtest-1780055291142.md
aeroftp-speedtest-1780055290030.csv

I then changed to 10MB and more cloud drives managed to not fail, though Koofr and YandexDisk still failed. OpenDrive passed barely.
Before the second run of 10MB, I connected to each of these manually to ensure that I don't have connection issues, but something else.
I'm getting this error in the Activity Log 📜:

INFO - Used-storage scan failed: Used-storage scan failed: Server error: List recursive failed (400 Bad Request): <?xml version="1.0" encoding="utf-8"?>

Here are the results:

aeroftp-speedtest-1780058076787.json
aeroftp-speedtest-1780058073986.md
aeroftp-speedtest-1780058075531.csv

I then re-ran the 10MB test and got almost the same results:

aeroftp-speedtest-1780059475678.md
aeroftp-speedtest-1780059472718.json
aeroftp-speedtest-1780059474222.csv

Then 2 runs of 1MB tests:

aeroftp-speedtest-1780059689844.json
aeroftp-speedtest-1780059691292.csv
aeroftp-speedtest-1780059692557.md

aeroftp-speedtest-1780060459789.md
aeroftp-speedtest-1780060457397.json
aeroftp-speedtest-1780060458605.csv

I conducted these benchmarks with pretty good internet non-fiber cable connection from western Europe.
It's beautiful to see some of these results, as most of these are connections that I suggested to pre-configure in AeroFTP.

I decided to reply here so there'll be some discussion here, and also to ask if you think it'll be beneficial for me to repeat this benchmark elsewhere, namely the CLI, and I can then open a dedicated Community Benchmark Report issue. Not sure if you want to move those to the Discussions tab, though I do expect some issues at first like Koofr WebDAV not working.

By the way, I still need to zoom out/decrease the font a lot to be able to reach the bottom of the Speed Test window to be able to reach the 'Export' and 'Run again' buttons. Recall #133 (comment). The vertical scrollbar helps, but doesn't solve the issue.

[axpnet]

Thank you, this is exactly the kind of cross-platform, real-line data the page needs, and a Windows + cable + Western Europe sample is a different point in the space from the dev host. I pulled your three runs into two charts first, both carrying the same line-limited caveat your data deserves (you are on cable, not fiber, so these are not provider rankings).

Your results as a pass / fail map across the three payload sizes, with the down / up speed inside every cell that completed:

And the throughput on your line at 10 MiB, for the providers that completed it:

Now the point by point.

On benchmarking API/OAuth drives from the GUI. You read it right: the GUI Speed Test deliberately runs at the raw-protocol layer and only accepts ftp / ftps / sftp / s3 / webdav. That is why it can compare your WebDAV and S3 profiles but not the API/OAuth-native drives (native MEGA, Google Drive, Dropbox, OneDrive, native pCloud/Filen, etc.). The full provider path, including every API/OAuth backend, is what aeroftp-cli benchmark exercises, because it goes through the same provider DAG the app uses for real transfers. So yes: please do repeat this on the CLI, it will cover the providers the GUI cannot. Surfacing API/OAuth coverage in the GUI (Speed Test or Debug Panel) is a fair ask and I will track it as a separate enhancement.

Filen S3 cross-confirmation. Your 100 MB Filen S3 failure (local.s3.filen.io:1800 unreachable) reproduced on my side in the 2026-06-07 CLI round (127.0.0.1:1800 not reachable). Same root cause on both OSes: the local Filen S3 bridge was not up. I then started Filen Desktop with Network Drive > S3 enabled and re-ran the full 1M/10M/100M matrix with --pre-delete: it passes clean, 15/15 runs, zero errors, including the 100 MB upload and download. So this one is confirmed not a transfer regression: it only needs Filen Desktop running. Worth noting in the docs that the Filen S3 preset is useless without Filen Desktop live, which is easy to forget.

Koofr WebDAV. Consistent 404 Not Found on upload across all three sizes in your run. That looks like a real path/target bug on the Koofr WebDAV writer, not a size limit. Tracking it for v4.0.5; if you can share the Koofr profile shape (root path vs sub-path) it will speed up the repro.

Yandex Disk WebDAV. Two distinct symptoms in your data: connection failures to webdav.yandex.ru at 10/100 MB, and an upload throughput that collapses to ~0.13 Mbps at 1 MB while download stays normal. That points at the WebDAV upload path specifically. Also tracking for v4.0.5.

OpenDrive. The 100 MB failure matches the documented free-tier limit, so that one is provider-side, not ours. The flat ~1.6 Mbps at 10/1 MB looks like OpenDrive throttling rather than a client cap. Good call on documenting per-provider max sizes: I will add a max-file-size column to the Chunking notes referenced in #272.

Used-storage scan failed: List recursive failed (400 Bad Request). Thanks for isolating this in the Activity Log. That is the quota/used-storage scan, separate from the transfer path, and 400 on a recursive LIST points at a provider that rejects the recursive form we send. Tracking it for v4.0.5.

Speed Test window cannot reach Export / Run again. Confirmed, this is the same layout problem as #133: the dialog overflows the viewport and the footer buttons fall off the bottom. The scrollbar is a workaround, not a fix. I will treat the footer reachability as a blocker for the Speed Test dialog.

On repeating elsewhere and opening a dedicated report. Yes please, a CLI round from your line would be very welcome, and a dedicated Community Benchmark Report is the right home for the raw JSON. We can keep the discussion here and link it. Expect the WebDAV-on-Koofr/Yandex rough edges above until the v4.0.5 fixes land; your data is what lets us tell a real gap from a ghost one.

[EhudKirsh]

I'm nearly certain that I had Filen Desktop up and running the local S3 server.
I think it's more likely that there was some problem with 100MB because it was too much for it, than me forgetting.

For Koofr, I use(d) a / Remote Path and an empty Local Path.

[EhudKirsh]

The baseline is now also browsable on the docs site, with all 188 throughput data points rendered as inline bar charts (no PDF download needed):

Your benchmark chart seems interesting. I wonder if benchmarks like this can actually be used by cloud providers in the future to decide whether to develop their own custom API or use a standard protocol like S3.
AeroFTP can indeed be a research platform as you mentioned in #266 (comment).

[axpnet]

Follow-up round (2026-06-07): DAG smoke + Filen S3 re-test, CLI 4.0.4

A second maintainer round, this time focused on the DAG transfer engine and on functional coverage rather than on throughput rankings. The honesty caveat from the baseline applies again, and it is worth restating because it matches @EhudKirsh's setup almost exactly: this ran on a domestic, non-fiber, asymmetric line (upload bandwidth far below download). So the raw speed numbers are not a provider ranking. The useful signal is functional and regression coverage: connect, scratch dir, upload, download, list, stat, delete, cleanup, and DAG routing under current code.

• CLI: aeroftp-cli 4.0.4, Linux x86_64, afternoon run
• benchmark custom --sizes 1M,10M,100M --runs 1
• 1 GiB payloads excluded by design (kept selective)

Reading speed on a slow line: normalize it away

On a capped line the absolute bars are misleading to the eye: every download looks dramatically faster than every upload, but that is the connection, not the providers. So instead of plotting Mbps, the charts below show each provider's percentage deviation from the round average, upload and download separately. That removes the line and leaves only what actually differs between providers.

Cloud S3 / Azure backends (Filen and iDrive shown separately):

Upload differences are tiny: every S3 backend lands within ~7 points of the average, because on a slow uplink they all hit the same line ceiling, so the provider barely matters. Download spread is wider and driven almost entirely by Azure Blob at -13%; the rest sit within about +/-3% of each other.

Overall, with Filen S3 folded in:

The direct cloud backends still cluster tightly. Filen S3 sits well below the pack (-51% upload, -31% download), which is expected and not a defect: it is a local end-to-end-encrypted bridge, so the numbers carry on-device crypto plus the localhost hop to Filen Desktop, costs a plain S3 PUT never pays. iDrive S3 is excluded from both charts because it ran under a tighter per-profile timeout and is not comparable.

Raw absolute numbers (100 MiB, line-limited, sanity only)

Profile	Up Mbps	Down Mbps
AWS	18.08	87.16
Alibaba OSS	19.34	90.53
Azure Blob	19.79	76.08
Backblaze Dev	19.73	89.89
Cloudflare R2	19.67	86.38
Google	19.20	88.02
Mega Dev S4	19.77	89.88
Oracle	19.76	90.00
S3 Storj	19.78	88.55
Tencent	18.82	89.10
Filen S3	8.96	58.43
iDrive S3	8.28	15.16

The ~18-20 Mbps upload ceiling vs ~75-90 Mbps download is the line, not the providers. Same shape Ehud reported from his cable line.

S3/Azure matrix: passed

AWS, S3 Backblaze, iDrive S3, Cloudflare R2, Alibaba OSS, Azure Blob, S3 Storj, Tencent, Mega S3, Google, S3Drive, Mega Dev S4, axpbuntu lab MinIO, Oracle, Backblaze Dev.

Skipped by instruction: Wasabi and Quotaless S3 (trial/access expired).

DAG coverage and the multipart caveat

--transfer-engine auto routes single-file network transfers through the DAG router. For S3/Azure/B2, however, 1M/10M/100M all stay below the multipart fan-out threshold:

• 100 MiB validates DAG single-core transfer behavior.
• It does not validate multipart UploadPart fan-out.
• Multipart fan-out needs a separate selective 256 MiB run.

Threshold unit checks passed: shaped_upload_below_multipart_threshold_stays_single_put and shaped_upload_at_or_above_multipart_threshold_fans_out.

Cross-confirmation with @EhudKirsh: Filen S3 local bridge

In the first pass Filen S3 failed because the local bridge 127.0.0.1:1800 was not reachable during the CLI run. This is the same failure Ehud saw from the GUI on Windows at 100 MB (https://local.s3.filen.io:1800/... unreachable). Two independent runs, two OSes, same root cause: the local Filen S3 bridge was not up, not a transfer-side regression.

To close the loop I brought Filen Desktop up (Network Drive > S3 enabled, port 1800) and re-ran the full matrix with --pre-delete. It passes clean, 15/15 runs, zero fatal and zero transient errors:

Payload	Upload Mbps	Download Mbps
1 MB	6.12	14.16
10 MB	9.54	53.20
100 MB	8.96	58.43

So Filen S3 is confirmed healthy end to end (upload, download, list, stat, delete) once the bridge is running. The earlier failures, both Ehud's and mine, were purely "bridge not up", and the throughput shape is the usual asymmetric-line skew, not a provider trait. Worth a docs note: the Filen S3 preset does nothing without Filen Desktop live, which is easy to forget.

S5 FileLu: overwrite-on-PUT sensitivity (new finding)

Initial CLI result looked like a DAG failure (502 Bad Gateway on 10 MiB DAG, while legacy passed). After GUI evidence and follow-up CLI tests the interpretation changed:

• GUI completed cleanly on S5 FileLu via S3 (upload 3.7 MB and 48 MB, rename both, download both, delete both).
• S5 FileLu --transfer-engine dag --pre-delete --sizes 10M then passed.

Conclusion: this is not a generic DAG failure. The benchmark was overwriting the same payload-<size>.bin key between warmup and measured run, and S5 FileLu is sensitive to overwrite-on-PUT. It should be benchmarked with --pre-delete or unique per-run keys. Recommended command:

aeroftp-cli --profile "S5 FileLu" benchmark custom \
  --sizes 1M,10M,100M --runs 1 \
  --operations upload,download,list,stat,delete \
  --pre-delete --report s5-filelu.json

Additional S5 FileLu provider behavior worth recording: large-object rename is slow (48 MB rename took ~48 s), and delete/batch-delete can intermittently return 500/502 though retry/fallback may still complete.

Next selective tests

1. Re-run S5 FileLu with --pre-delete across 1M/10M/100M and update status.
2. 256 MiB selective run for S3/Azure/B2/MinIO Dev to exercise multipart DAG fan-out.
3. Keep 1 GiB tests separate and selective.

[EhudKirsh]

Random thought on this topic: I've heard that where S3 shines over WebDAV and even FTP is when it comes to many small files,
because it streams files instead of having to sign a handshake for each file, or something like this. The Speed Test in the GUI only allows for
4 parallel runs. Why not allow benchmarking 100 or even 200 1MB files all in 1 run? It's the same storage as 100MB in 2 runs.

[axpnet]

Two things here, and the first is a clarification worth making:

The "4 parallel runs" isn't a files-per-run limit. That cap is the compare concurrency: how many different servers the Speed Test races at once. Each run still transfers a single file (1/10/100 MB), so "100 files in one run" isn't a throttled knob today, it's a workload we simply don't have yet.

And you've put your finger on exactly the gap. A "many small files" workload is where per-file cost (handshake, signing, metadata round-trips) dominates, and it's precisely where S3 pulls ahead of WebDAV and FTP. Neither the GUI Speed Test nor the CLI benchmark measures it right now. They should.

Plan: add it CLI-first (no translation surface to slow it down), as a new dimension of benchmark custom: --file-count N --file-size S with dedicated ops (upload-all / download-all / list-a-dir-of-N / stat-all / delete-all). The report would surface files/sec and mean per-file time alongside raw throughput, since those are the numbers that actually show the small-files advantage. The community page would keep many-files and single-large-file as separate axes; averaging them together would be misleading.

Before I wire the GUI side, I'll come back here with the proposed report shape so you can sanity-check it. Good catch: this is the scenario that makes the protocol comparison page honest.

[axpnet]

Coming back as promised with the proposed report shape, now that it runs.

It landed CLI-first (no translation surface to slow it down), as a separate axis of benchmark — it never gets averaged into the single-file throughput numbers, because mixing "100 small files" with "one big file" would hide exactly the effect you're pointing at.

aeroftp-cli benchmark --profile "<server>" --file-count 1000 --file-size 64K

--file-count N adds five dedicated ops — upload-all, list-dir (one listing of the N-file directory), stat-all, download-all, delete-all — and each one reports files/sec plus a per-file latency distribution (p50 is the mean per-file time), alongside the raw Mbps for the transfer ops. Each file gets unique random content so content-dedup backends can't short-circuit the upload and fake a high files/sec.

Here is a real run from saved profiles on the dev host (30 files x 8 KB, same workload on each), with the honest caveat that has applied to every round in this thread: this is a domestic, asymmetric, non-fiber line, so the upload column is RTT-bound and is not a provider ranking. The signal you predicted shows up cleanly in the pure-metadata ops (stat / list / delete):

files/sec (higher = better)        upload-all   list-dir   stat-all   download-all   delete-all
-------------------------------------------------------------------------------------------------
S3   - Backblaze B2                       1.6        354       20.7           11.4         18.7
S3   - Storj                              3.6        469       22.4            6.0         17.0
WebDAV - Koofr                            1.5        204        9.7            2.3          6.0
SFTP - NAS (LAN, 20 files x 16 KB ref)  147.3        730      500.6           34.1       1256.3

On stat-all the two S3 backends sit at ~21-22 files/s versus ~10 for WebDAV, and list-dir / delete-all tell the same story — that's the per-file-overhead gap you described, isolated from bandwidth. (The SFTP/NAS row is on a LAN and with slightly different params, so it's a local reference point, not a peer of the three cloud rows.)

The JSON report stays schema v1: the two new fields (file_count, files_per_second) are additive and simply absent on the single-file rows, so existing consumers keep working. Docs: COMMUNITY-BENCHMARK.md and the CLI guide both have a "many small files" section now.

Before I wire the GUI side, does this report shape look right to you — files/sec + per-file p50 as the headline, the five ops, and the two axes kept strictly separate? If the shape is good I'll mirror it into the Speed Test window next.

[EhudKirsh]

It looks good, thanks. Just for the table, I think it's best to write "higher is better" instead of "=".
Also, please only have 1 space between "S3" and "-". I'm not sure if this table is just your manually-written summary or if it's going to be part of the improvements to the CLI benchmarks, so I gave feedback on it.