Transfer of object.patch.set-data in http API

[ianopolous]

The data in object.patch.set-data is stored in the URL. This is quite limiting if I want to put arbitrary binary data there (esp if I want ~512kb). Would putting the data in the request body or a multi-part make sense?

[whyrusleeping]

the input for set data is passed via stdin (aka, the request body)

[ianopolous]

In 0.3.11 it only works as an extra argument in the URL.

[whyrusleeping]

mmmm, no. You definitely can pass file data:

$ ipfs object patch `ipfs object new` set-data < somefile

[whyrusleeping]

unless something really weird is happening and turning that into a url parameter... have you tried it from the cli and checked what request it makes?

[whyrusleeping]

oh weird:

cmds.StringArg("args", true, true, "extra arguments").EnableStdin(),

So it can take file input, but apparently the file gets read in and turned into a normal 'string' argument...
thats kindof annoying

[ianopolous]

Yep, at the moment this is the only thing stopping us fully using IPFS in Peergos.

[jbenet]

@whyrusleeping can we make this a cmds.FileArg ?

[whyrusleeping]

@jbenet its not quite that simple unfortunately... we need to find a way to support arbitrary position arguments between subcommands: CMD SUB <arg> SUB <arg>

[whyrusleeping]

dammit. we've been able to do this the whole time.

[whyrusleeping]

@ianopolous should be fixed now on dev0.4.0, confirm?

[ianopolous]

Hmm, I get a 400 on dev0.4.0 for add-link:
Server returned HTTP response code: 400 for URL: http://127.0.0.1:5001/api/v0/object/patch?arg=QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n&arg=add-link&arg=somelinkname&arg=QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n

I get a similar 400 for the set-data post as well.

[whyrusleeping]

hrm... let me try a few things...

[whyrusleeping]

@ianopolous are you using the js-ipfs-api?

[ianopolous]

Nope, the Java one, but I haven't pushed the changes to make the set-data use the http body..

[whyrusleeping]

@ianopolous alright, you'll have to make a change to the way your url is constructed. A valid one from the CLI looks like:

/api/v0/object/patch/add-link?api=%2Fip4%2F127.0.0.1%2Ftcp%2F5005&arg=QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn&arg=test&arg=QmQMx3hDUQqgCRi9kXBKLYDcJj3Q213o5jm91ZiKdcDLNj&encoding=json&stream-channels=true

Note that 'add-link' is no longer an argument, but one of the commands.

[ianopolous]

Okay, that's fixed add-link to work again, but set-data still 400s: e.g.
http://127.0.0.1:5001/api/v0/object/patch/set-data?arg=QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n
with a non empty body of some random bytes.

[whyrusleeping]

@ianopolous check the http trailers sent on the request, they should give you error messages (if you don't get anything useful from the body). The data is expected to be multipart to mesh easily with everything else that we use multipart for.

On a related note, are there any tests for the java api bindings that we could run in CI? It would be nice if we could see when changes we make here break code that depends on us.

[ianopolous]

Ah I didn't realise it had to be multipart. That'll do it.

There are indeed tests, run using "make tests" in the root, obviously requiring the jdk to be installed

[whyrusleeping]

@ianopolous cool. We will discuss automated testing of libraries that are dependant on our code during the infrastructure sprint this week. cc @lgierth @RichardLitt

[ianopolous]

I've switched it to multipart, but now it gets a 500, and the Trailer response header says: X-Stream-Error
The response body is empty. The Multipart works fine for other commands like block.put

[ianopolous]

The url the multipart is using is e.g. http://127.0.0.1:5001/api/v0/object/patch/set-data?arg=QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n&stream-channels=true

[whyrusleeping]

@ianopolous is this still an issue?

[ianopolous]

@whyrusleeping Not sure why this is still open. You fixed it during that chat we had ages ago. All is good now. :-)