Testing with SPs

[xinaxu]

Engage and test with our list of SPs to test the deal making part and make sure it works as part of the complete motion experience.

• Prepare test data of ~100GB
• Make deals with test SPs
• Test retrievals

Tasks

Beta Give feedback

1. [Beta Testing with SPs] Confirm pull-mode storage deal configuration is set up to test Powermeta #188
   beta +1
   
2. [Beta Testing with SPs] Run Filswan Test #187
   beta +1
3. [Beta Testing with SPs] Write Retrieval Script #186
   beta +1
   
4. [Beta Testing with SPs] Write Status Script #185
   beta +1
   
5. [Beta Testing with SPs] Write storage script #184
   beta +1
   
6. [Beta Testing with SPs] Confirm Motion test instance IP address #183
   beta +1
   
7. [Beta Testing with SPs] Confirm pull-mode storage deal configuration is set up to test Filswan #182
   beta +1
   
8. [Beta Testing with SPs] Run Powermeta Test #189
   beta +1
9. [Beta Testing with SPs] Set up Motion Test Instance #190
   beta +1

[rvagg]

Underway. This is the state of play as of today, waiting for sealing as far as I know, so we still have the data locally:

We had a bit of trouble making deals but have sorted out some bugs along the way. Notably data-preservation-programs/singularity#413 for proposing usable deals and data-preservation-programs/singularity#416 for serving pieces; plus a couple of problems on the other side too but we seem to be good for now.

Next steps:

1. Sealing all these pieces
2. Performing and measuring some baseline retrievals
3. Deploying and measuring @gammazero's retrieval fixes

I'll hopefully get some basic work for 2 done today. We have a record (from the https://github.com/filecoin-project/motion-sp-test status.json file) of files we've stored, their UUID mapping, their size and SHA2-256 digest so we can do some retrievals with different size blobs and confirm we get what we expect. We could even toy around with streaming videos out of Motion for demonstration purposes I think.

[rvagg]

Going to call this one done and write up the learnings; it didn't end up being as methodical as hoped but what we learned should help us start a new iteration of testing, with existing and new SPs.