Arweave Integration Milestone 1: Publish using transaction ID

[MantisClone]

Warning
Revert the following commit before (or after) merging this PR. This commit skips the code coverage step in the CI which fails because CC_TEST_REPORTER_ID is not set in PRs by external contributors.

• f26ef55

Warning
Revert the following commit before (or after) merging this PR. This commit skips test_nonocean_tx which fails because REMOTE_TEST_PRIVATE_KEY1 == '' empty string in PRs by external contributors

• 64c5ad2

Towards oceanprotocol/pm#151.

Changes proposed in this PR

• Add ArweaveFile class and test_arweave_file() to test it.
• Add get_arweave_file() helper and arweave_file fixture.
• Make existing tests use the arweave_file fixture. including: test_encrypt, test_fileinfo, test_initialize, test_consume_flow, test_compute_flow_registered_asset_in_arweave, and all of the tests in test_ocean_assets.py
• When downloading a file stored in Arweave, replace filename with random UUID hex so filename doesn't leak transaction ID.

Note
This does not include the ability to upload to Arweave. That is left for a future PR.

Changes that were implemented but then reverted

• Generalize existing URL files structure (change "url" key to "value") (for details see Add arweave file type to DDO spec docs#1019 (comment))
• I hacked pytest.yml to use MantisClone/barge repo which uses dmats/provider:arweave-integration docker image to make the CI tests pass. Reverted now that provider PR Arweave Integration Milestone 1: Publish using transaction ID provider#485 is merged.

[MantisClone]

I believe all the changes related to Arweave files are working as intended.

All the tests are passing except the polygon config test due to error:

E requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://rpc.polygon.oceanprotocol.com/

https://github.com/oceanprotocol/ocean.py/runs/6950317363?check_suite_focus=true#step:7:150

I believe this error is unrelated to the Arweave integration changes.

[AnaLoznianu]

Hi @MantisClone! I've added the reviewers back. I see the tests are failing though, can you please fix them?

[MantisClone]

Thanks for pointing that out @AnaLoznianu. I believe that I've fixed all the issues now - all the tests pass locally. But i'm having trouble getting the CI checks to pass.

I've already spent over 2 hours on it this morning. Each CI run takes over 30 minutes. There are frequent random failures, especially Address file not found #939. But more recently, other failure modes have emerged.

1. Invalid or unresponsive aquarius url http://172.15.0.5:5000/ https://github.com/oceanprotocol/ocean.py/actions/runs/3227265307/jobs/5281846034#step:7:263

2. urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='127.0.0.1', port=8545): Read timed out. (read timeout=10) https://github.com/oceanprotocol/ocean.py/actions/runs/3222597344/jobs/5271829143#step:7:1115
   This one was particularly upsetting because it ran for over 3 hours instead of the typical 30 minutes.

[MantisClone]

Hello @alexcos20 @calina-c @mariacarmina I believe this PR is ready for review.

[MantisClone]

@alexcos20 @calina-c @mariacarmina CC @AnaLoznianu
The tests are broken but they were working 9 days ago. I'd really love to get the changes in this PR merged in. Is this a realistic goal? Or should I stop maintaining this?

[MantisClone]

@alexcos20 CC @AnaLoznianu
Happily, it seems that the tests are passing again. Can you please review this at your soonest convenience?

[trentmc]

This needs a README to show how to use. It can look similar to:

• REST API flow - publish & consume REST API-style URIs, showing Binance ETH price feed
• GraphQL flow - publish & consume GraphQL-style URIs
• On-chain data flow - publish & consume on-chain data

[MantisClone]

Hi @trentmc. Thanks for the comments. This PR has dropped off the end of my priority list and I don't have the bandwidth to continue pushing it forward.

I could hand-off this PR to someone on the core team by retargeting it to a development branch on the upstream repo instead of old-main). If this approach is desirable, please let me know what branch should be the target.

[trentmc]

Closing, this was superseded by PR #1226, and merged to main in d6f6a90