-
-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[IPFS add] "nocopy" option for "add" method doesn't work in a HTTP API call. #4558
Comments
I am not super familiar with how the HTTP API works but I can look into to. Seams easy enough. |
@v696973 Sorry for the delay. This should probably error out somewhere but the problem is that the full path of the file is not being sent to the server only the name of the file. In order for the filestore to work it needs to path of the file so it can find it on the filesystem. |
Thanks, @kevina! So it turns this is my mistake, although this wasn't mentioned in the docs (it still should pretty obvious, but the lack of explicit error message from the API left me confused). I'm closing this issue for now. |
This should at least return an error rather than silently falling back to storing the file in the datastore. Reopening and will close once that is in place. |
For anyone wondering how to make this work: The request to add needs to send body as Here is how a correct nocopy request should look like:
(The issue here is that we allow adding files with no |
@magik6k setting custom header values inside multi-part form fragments seems very non-standard, I don't see how to do this without crafting a custom post body by hand (curl assumes you would only use Content-Disposition and Content-Type) I guess the issue is that paths are relativized (ie sending |
hmm so this approach is in violation of RFC 7578:
which explains why other tools do not support setting these the restriction on using absolute paths is softer:
it appears that if we are to bend the rules to specify these, using absolute paths in the existing Content-Disposition header is the lesser evil |
This restriction only applies to
This also sounds a bit like we should use |
It makes it hard to tell where object root is, and ties dag paths to files in the filesystem (though the second point would only be an issue for people who could easily work around it) |
hmm that wasn't my reading of 4.8, though perhaps 5.1 in 2046 does override it (I would prefer do you have an example curl invocation or comparable code snippet that creates a valid request with this header? |
@Stebalien's example just worked for me on master: curl 'http://127.0.0.1:5001/api/v0/add?nocopy=true' -F 'file=@SOMEFILE;headers="Abspath: /home/magik6k/full/path/to/SOMEFILE"' |
Note that leaving out this header will generate |
(Let's move this discussion to #5984) |
Version information:
go-ipfs version: 0.4.13-
Repo version: 6
System version: amd64/linux
Golang version: go1.9.2
Type:
Bug
Severity:
Low-Medium
Description:
I've been trying to use an IPFS HTTP API in order to add files using experimental filestore feature, and found out that "nocopy" option for "add" method doesn't actually prevent IPFS from copying files when used in a HTTP API call. The size of the blockstore gets increased by the size of the file I'm trying to add, effectively duplicating the file.
CLI version still works as expected
Notes on the IPFS version: I'm using a pre-built binary from the last go-ipfs release (0.4.13).
Steps to reproduce:
Let's make sure that we indeed enabled the filestore:
Check the size of our test data file:
And the size of our blockstore:
Let's try to add our file using CLI:
Checking the blockstore size...
A minor increase in size. Looks like it works, just like it should.
Now, let's do a cleanup and try to add the same file, only now using HTTP API:
Add the file using HTTP API:
Judging by the size of the blockstore, we've got a copy of our file, despite the "nocopy" option.
So, it looks like it's a bug to me, although I may be missing something important, like any additional parameters that I should pass to the API call. The API is still quite poorly documented, unfortunately.
The text was updated successfully, but these errors were encountered: