Sia doubles storage contracts prematurely

[mtlynch]

BUG REPORT

How to reproduce it (as minimally and precisely as possible)

1. Deposit 500 SC into Sia wallet
2. Set allowance of 500 SC for 12w
3. Wait for 50 contracts to form
4. Upload 300 GiB of data to Sia, 5 files at a time

Instructions for repro with the tools I used are as follows:

DATA_DIR=~/repro-data

# Generate data to upload
BYTES_PER_FILE=10737418240 # 10 GiB
TOTAL_SIZE=322122547200 # 300 GiB

git clone https://github.com/mtlynch/dummy_file_generator.git
pushd dummy_file_generator
python dummy_file_generator/main.py \
  --size_per_file $BYTES_PER_FILE \
  --total_size $TOTAL_SIZE \
  --output_prefix "${DATA_DIR}/dummy-file-
popd

# Upload to Sia
OUTPUT_DIR=/tmp/sia-test-output

git clone https://github.com/mtlynch/sia_load_tester.git
pushd sia_load_tester
pip install -r requirements.txt
python sia_load_tester\main.py \
  --dataset_root $DATA_DIR \
  --output_dir OUTPUT_DIR

Expected Behavior

Sia does not invest more into contracts until its existing 50 contracts are exhausted.

Actual Behavior

Sia maintains its initial set of 50 contracts, but increases spending in most of them, wasting renter funds.

More data: https://docs.google.com/spreadsheets/d/1g6bMB7CrH_Nn7orUfguD7RDd_9rmHKlXLJrrqv06sd4/edit#gid=1191012075

Logs

https://gist.github.com/mtlynch/ccc29acfbd7cf4c9716180eac2af39a5

Environment

• Sia version: 1.3.1
• OS: Windows 10 x64

[tbenz9]

It might be useful to track the number of contracts as well. It's possible that this issue is related to #2938

[mtlynch]

Contract count is in the spreadsheet data. After the initial 50 contracts, contract increases to 51 at 5h7m, then 52 contracts at 15h56m. Test ended with a contract count of 52.

[DavidVorick]

I'm still not convinced this one in particular is a bug. Sia renews contracts as they fill up, and there will be a tendency (especially during your first renew cycle) for the faster and better hosts to fill up sooner. Future cycles (after your first 12 weeks) should see less turbulence.

When you create an allowance of X, only X/3 gets spent in the first round, because we don't know which hosts are going to be the good ones. If we knew, we'd spend the full X right away, but historically when Sia did that (a few versions ago), you'd end up with 0 money in the contracts with the best hosts, and the worst hosts would bottleneck your upload speeds substantially.

[mtlynch]

Closing as it sounds like this behavior is WAI.