Error "volume is not on local disk" using volume.tier.upload

[remram44]

Describe the bug
Error volume is not on local disk during normal operations.

If the volume is removed from the local volume servers by volume.tier.upload, the next call will try to upload again and run into this error. It looks like the error stops the processing of the other volumes which is a serious issue.

System Setup

• SeaweedFS 3.00 on Ubuntu 20.04
• version 30GB 3.00 b1dac20c704cf5322c6c516e5f064eb00701aec9 linux amd64

Expected behavior
volume.tier.upload should work every time.

Screenshots

$ printf 'lock; volume.tier.upload -dest=s3.aws -fullPercent=10 -quietFor=10s; unlock\n' | weed shell
master: localhost:9333 filers: [192.168.1.210:8111]
collect volumes quiet for: 10 seconds
tier upload volumes: [2 4 3 7 8 9 6 1 5]
markVolumeReadonly 2 on 192.168.1.210:8080 ...
error: copy dat file for volume 2 on 192.168.1.210:8080 to s3.aws: rpc error: code = Unknown desc = volume 2 is not on local disk

$ printf 'lock; volume.tier.upload -dest=s3.aws -fullPercent=10 -quietFor=10s; unlock\n' | weed shell
master: localhost:9333 filers: [192.168.1.210:8111]
collect volumes quiet for: 10 seconds
tier upload volumes: [1 3 5 8 6 4 7 9 2]
markVolumeReadonly 1 on 192.168.1.210:8081 ...
error: copy dat file for volume 1 on 192.168.1.210:8081 to s3.aws: rpc error: code = Unknown desc = volume 1 is not on local disk

$ printf 'lock; volume.tier.upload -dest=s3.aws -fullPercent=10 -quietFor=10s; unlock\n' | weed shell
master: localhost:9333 filers: [192.168.1.210:8111]
collect volumes quiet for: 10 seconds
tier upload volumes: [9 3 8 2 4 6 1 5 7]
markVolumeReadonly 9 on 192.168.1.210:8081 ...              ## upload finally happened here!
markVolumeReadonly 3 on 192.168.1.210:8081 ...
error: copy dat file for volume 3 on 192.168.1.210:8081 to s3.aws: rpc error: code = Unknown desc = volume 3 is not on local disk

[chrislusf]

added a fix

[remram44]

You're fast! Thanks a lot