Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ipfs gateway url to fetch dir images from other creator nodes #718

Merged
merged 1 commit into from
Jul 31, 2020

Conversation

dmanjunath
Copy link
Contributor

@dmanjunath dmanjunath commented Jul 31, 2020

What's the problem?
IPFS gateway calls for images inside directories takes the form /ipfs/Qm123/Qmabc instead of /ipfs/Qm123/<filename>.jpg

What's the solution?
Pass in the file name from the files table (original.jpg, 150x150.jpg etc...) into the saveFileForMultihash function so requests to other cnode can be in the form /ipfs/Qm123/<filename>.jpg

How is this tested?

  1. Setup a fresh audius instance locally and created a user and uploaded a track
  2. Set ipfs operation timeouts to 1ms so they'll timeout and it will always to go to fetch from other cnode gateways.
  3. Triggered a sync
  4. Verified that the number of items in both /file_storage volumes match the number of files in the files table in the db. The saveFileForMultihash function also runs ipfs add with the only has option to make sure the CID and the actual contents of the file match.
  5. Clear the secondary's /file_storage volume. Do an ipfs repo gc on all ipfs nodes and make sure you can't cat the files on any node. Run steps 3&4 again
  6. Add new cover art to my user profile. This should trigger a sync. Verify according to step 4 again.

Copy link
Contributor

@hareeshnagaraj hareeshnagaraj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add a QA checklist and run through some local validation before merge

@hareeshnagaraj
Copy link
Contributor

hareeshnagaraj commented Jul 31, 2020

prior to this fix, gateway URLS were formatted as http://cn1_creator-node_1:4000/ipfs/ipfs/QmTcm6ntbHAhz9MdzgEPFQnLzDsfw9Sb8q57gW7LTXkGnQ1 when fetching images in saveFileForMultihash

after the fix, gateway URLS are now formatted as follows
dir:

[2020-07-31T23:23:57.544Z]  INFO: audius_creator_node/746 on e3b5d3ad750c: gatewayUrlsMapped 
[ 'http://cn2_creator-node_1:4001/ipfs/QmcSqpPN5R5UheCUXMRByLHLiFvFtzFWNK4Jsww31e2evF/480x480.jpg' ] 
(requestID=GIW4YI7Fzl, requestMethod=POST, requestHostname=localhost, requestUrl=/sync)

non-dir:

[2020-07-31T23:23:57.452Z]  INFO: audius_creator_node/746 on e3b5d3ad750c: gatewayUrlsMapped 
[ 'http://cn2_creator-node_1:4001/ipfs/QmSoBan8MsFX9xLLZneoMT5243q1V3feFHD5UGz8NCzrTA' ] 
(requestID=GIW4YI7Fzl, requestMethod=POST, requestHostname=localhost, requestUrl=/sync)

Copy link
Contributor

@hareeshnagaraj hareeshnagaraj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dmanjunath dmanjunath merged commit 67033a8 into master Jul 31, 2020
@dmanjunath dmanjunath deleted the dm-saveFileForMultihash-bugfix-07-31 branch July 31, 2020 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants