Improve performance of directory listings

[palant]

Description

The benchmarks in #355 indicated that SWS directory listings are slow, so I looked into how they can be improved. I managed to improve performance of the directory listing benchmark by around 10% which is ok but not really sufficient to catch up with the competition.

The performance improvements are largely due to the way the listing is constructed. There is a large improvement by not percent-decoding the file name but using the name we already have. And constructing table_rows variable from an iterator also improves things.

Remarkably, the changes to sort_file_entries() function are barely noticeable in the benchmark. Presumably, Rust optimizer already figured out that no sorting needs to be performed for order_code == 6 and that variables like name don’t need to be overwritten with the same value repeatedly.

How Has This Been Tested?

I did functional testing on Linux.

[semanticdiff-com]

Review changes with SemanticDiff.

Analyzed 1 of 1 files.

Overall, the semantic diff is 33% smaller than the GitHub diff.

	Filename	Status
✔️	src/directory_listing.rs	32.82% smaller

[palant]

For reference, I did try using write!() into a string to compose the entire page and not merely the table. This made the code easier to follow in some respects but it didn’t have any noteworthy impact on the performance. So it seems that there are no significant gains left in the string composition any more. Yes, I should probably try an actual profiler.

[joseluisq]

Looking at the benchmark results, it seems like there is a slightly small improvement when listing large directory files but for small directories is even slightly slower. I did compare the current master results with the PR.

[joseluisq]

I left some minor comments to check first.
But overall for me looks ok.

[palant]

Looking at the benchmark results, it seems like there is a slightly small improvement when listing large directory files but for small directories is even slightly slower.

The benchmark results aren’t stable enough to measure small time differences reliably. You have to consider the other web servers as the baseline – their time also went up, so it’s something related to the action runner, not the code. If I check the proportion for SWS time and other servers on the 50th percentile (median), for an empty directory it went down consistently. Not a huge change but a few percent. In fact, the results are actually less conclusive for large directories.

[palant]

Looking at the benchmark results, it seems like there is a slightly small improvement when listing large directory files but for small directories is even slightly slower.

The benchmark results aren’t stable enough to measure small time differences reliably. You have to consider the other web servers as the baseline – their time also went up, so it’s something related to the action runner, not the code. If I check the proportion for SWS time and other servers on the 50th percentile (median), for an empty directory it went down consistently. Not a huge change but a few percent. In fact, the results are actually less conclusive for large directories.

Looking at the new test run – yes, the performance changes here are negligible. Somehow that looked differently in my previous benchmark runs.

[palant]

Looking at the benchmark results, it seems like there is a slightly small improvement when listing large directory files but for small directories is even slightly slower.

The benchmark results aren’t stable enough to measure small time differences reliably. You have to consider the other web servers as the baseline – their time also went up, so it’s something related to the action runner, not the code. If I check the proportion for SWS time and other servers on the 50th percentile (median), for an empty directory it went down consistently. Not a huge change but a few percent. In fact, the results are actually less conclusive for large directories.

I’ve done some benchmarking on my computer, the numbers are more stable then. I could confirm – the performance for a large directory does improve but it is only around 3%. Not overwhelming but measurable.

[joseluisq]

Nice! Dir listing is now way faster than before.
https://github.com/static-web-server/static-web-server/actions/runs/8823589340

Awesome!