Too many open files when pushing large bindles

[dicej]

Per spinframework/spin#180 (comment), we discovered that, on POSIX systems with a default file descriptor limit of 1024, pushing a bindle containing many assets using StandaloneRead::push results in a "Too many open files" error. This is due to the unlimited concurrency of future::join_all.

Steps to reproduce:

1. Using MacOS or Linux, set the file descriptor limit for your current shell: ulimit -n 1024 (this is the default on most systems).
2. Install the latest Spin release.
3. Clone and build the spin-kitchensink project.
4. From within the spin-kitchensink project directory, use spin to push the bindle to your favorite bindle-server and observe the output:

 $ BINDLE_URL=http://127.0.0.1:8000/v1 spin bindle push --file spin.toml
Error: Failed to push bindle to server

Caused by:
    0: Failed to push bindle from '/tmp/.tmplIh9Uw' to server at 'http://127.0.0.1:8000/v1'
    1: Error creating request
    2: error sending request for url (http://127.0.0.1:8000/v1/_i/spin-kitchensink/1.0.0@6394e8e650d04c26ed20b6058d5e93119f544ef1bd00aefb942297d2e2f6c7e1): error trying to connect: tcp open error: Too many open files (os error 24)
    3: error trying to connect: tcp open error: Too many open files (os error 24)
    4: tcp open error: Too many open files (os error 24)
    5: Too many open files (os error 24)

Setting the file limit to a sufficiently large number will fix the problem, but that may not be possible if the user does not have sufficient privileges.

I have a fix for this and will open a PR shortly.

[dicej]

@thomastaylor312 Thanks for merging this. I'd like to pull this fix into Spin. Is there a release planned soon-ish? If not, I'd be happy to help make one happen.

[thomastaylor312]

Yeah I have #334 that I need to tie up and merge. Then we can cut a new release. This one will be a pretty big breaking release so we'll need to document it well