Document example of sending a Dict to POST API

[s-celles]

Hello,

I'm facing a difficulty (maybe because doc need some improvements) to convert a Julia 0.6 code which was using Request.jl to a Julia 0.7/1.0 code using HTTP.jl

This is my original code

using Requests: post, json
raw_response = post(url_query; data = params)
response = json(raw_response)

I tried this

using HTTP
using JSON
raw_response = HTTP.request("POST", url_query, params)
response = JSON.parse(raw_response)

but API endpoint ( see https://pushover.net/ and https://pushover.net/api for my use case) returned

ERROR: HTTP.ExceptionRequest.StatusError(400, HTTP.Messages.Response:
"""
HTTP/1.1 400 Bad Request
Server: nginx
Date: Sun, 19 Aug 2018 09:30:08 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, OPTIONS
Access-Control-Allow-Headers: X-Requested-With, X-Prototype-Version, Origin, Accept, Content-Type, X-CSRF-Token, X-Pushover-App, Authorization
Access-Control-Max-Age: 1728000
Cache-Control: no-cache
X-Request-Id: fca220b7-291f-4526-845f-99afd114f0f7
X-Runtime: 0.007232

{"message":"cannot be blank","errors":["message cannot be blank"],"status":0,"request":"fca220b7-291f-4526-845f-99afd114f0f7"}""")

My parameters dict looks like

julia> params
Dict{String,String} with 4 entries:
  "priority" => "1"
  "token"    => "NzXXXXXXXXXXXXXnp"
  "message"  => "My first message"
  "user"     => "RAXXXXXXXXXXXXjN"

I also tried

HTTP.request("POST", url_query, [], JSON.json(params))

but it still raise same exception.

Any idea what is going wrong here and how/if doc need to be improved or not (if that's the case I will submit a PR)

Kind regards

[quinnj]

I think you want HTTP.request("POST", url_query, [], params), note the 3rd positional argument is for headers, the 4th is for the request body. You can generally debug these things by calling HTTP.request(...; verbose=2).

[s-celles]

I also tried that but it's hanging

julia> using HTTP

julia> url_query = "https://api.pushover.net/1/messages.json"
"https://api.pushover.net/1/messages.json"

julia> params =  Dict(
         "token"    => "NzXXXXXXXXXXXXXnp",
         "message"  => "My first message",
         "user"     => "RAXXXXXXXXXXXXjN"
       )
Dict{String,String} with 3 entries:
  "token"   => "NzXXXXXXXXXXXXXnp"
  "message" => "My first message"
  "user"    => "RAXXXXXXXXXXXXjN"

julia> HTTP.request("POST", url_query, [], params; verbose=2)
HTTP.Messages.Request:
"""
POST /1/messages.json HTTP/1.1
Host: api.pushover.net
Transfer-Encoding: chunked

"""
HTTP.Messages.Request.body: Dict("token"=>"NzXXXXXXXXXXXXXnp","message"=>"My first message","user"=>"RAXXXXXXXXXXXXjN")

That's not a server side issue... because I installed an old Julia 0.6 and Requests, and it's working.

[samoconnor]

Passing a julia Dict as the request body is probably not what you want.
You probably need to send the body in a format that the pushover server understands and set the Content-Type header to tell it what format you've used.

The Pushover API doc says to set Content-Type to application/x-www-form-urlencoded or application/json.

This will probably work:

HTTP.request("POST", url_query,
             ["Content-Type" => "application/json"],
             JSON.json(params))

or this

HTTP.request("POST", url_query,
             ["Content-Type" => "application/x-www-form-urlencoded"],
             HTTP.URIs.escapeuri(params))

[samoconnor]

maybe because doc need some improvements

👍

The client examples in the README are both GET. It would be useful to add a POST example that sends a Dict as JSON.

[s-celles]

All right

julia> raw_response = HTTP.request("POST", url_query,
                    ["Content-Type" => "application/json"],
                    JSON.json(params))
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Server: nginx
Date: Mon, 20 Aug 2018 09:22:30 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, OPTIONS
Access-Control-Allow-Headers: X-Requested-With, X-Prototype-Version, Origin, Accept, Content-Type, X-CSRF-Token, X-Pushover-App, Authorization
Access-Control-Max-Age: 1728000
X-Limit-App-Limit: 7500
X-Limit-App-Remaining: 7487
X-Limit-App-Reset: 1535778000
ETag: W/"4bde8b01b463bba5bf7b9c7419867349"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 4e029681-e44e-457b-9f9b-be54bd1f7e4d
X-Runtime: 0.052171
X-Frame-Options: DENY
Strict-Transport-Security: max-age=31536000

{"status":1,"request":"4e029681-e44e-457b-9f9b-be54bd1f7e4d"}"""

maybe example could go up to parsing JSON response.

[s-celles]

So

julia> JSON.parse(String(raw_response.body))
Dict{String,Any} with 2 entries:
  "status"  => 1
  "request" => "4e029681-e44e-457b-9f9b-be54bd1f7e4d"

Isn't there a way to do it "simpler"? Like Requests.jl was doing?

Maybe a more high-level API should be considered?

[samoconnor]

@scls19fr My understanding is that Requests.jl was modelled after python's "Requests: HTTP for Humans". As such it implements heuristics and default behaviours that try to guess the user's intention and make it easier to use on the command line or in simple shell-script style programs.

HTTP.jl's focus is to provide robust deterministic functions with predictable results. It does not attempt to guess how to format the request or what metadata headers might be appropriate. It does not try to guess how to parse the response body.

Huristics like these belong in a higher level module. e.g. AWSCore.jl has heuristics that work for AWS services but would not be appropriate in general. GitHub.jl has its own similar thing where the detail is tuned to work with the GitHub API.

Your use-case of a JSON-based web API is certainly a common one. It might be worth building a seperate JSONREST.jl package that provides a simple request function for the case where the request and response are always formatted as JSON.