feat(webserver): implement content negotiation and deprecated endpoints

[andrewazores]

Fixes #1016

redefine existing mimeType() method as produces() which provides a List of HttpMimeType. Existing implementations simply wrap their old return values into a singleton List. The default implementation is an empty list, which means that the webserver ignores content negotiation for such handlers (ie. v1 handlers). For compliant handlers (all v2+), content negotiation will be used if the client specifies an Accept request header. If no Accept request header is given then the behaviour is unchanged from before. If the Accept header is given then the handler will only be called if an acceptable content type can be negotiated, falling back to the general 'GET /'-style handler if negotiation fails. There are no such examples yet, but any handlers which produce() multiple mime types should use ctx.getAcceptableContentType() to check what format they should respond with.

The basic hook for consumes() on the other end of content negotiation is also included, but there are no implemented examples.

[andrewazores]

All of the actual meaningful change in this PR is contained within WebServer, RequestHandler, and AbstractV2RequestHandler classes/interfaces. The rest are essentially all redefining mimeType() to produces().

[andrewazores]

To test, pick a v2 endpoint like /api/v2.1/discovery and try the following:

$ sh smoketest.sh
$ # in another terminal do:
$ https :8181/api/v2.1/discovery Authorization:"Basic $(echo user:pass | base64)" # should get a JSON response
$ https :8181/api/v2.1/discovery Authorization:"Basic $(echo user:pass | base64)" Accept:text/plain # should get the web-client index.html because content negotiation failed
$ https :8181/api/v2.1/discovery Authorization:"Basic $(echo user:pass | base64)" Accept:application/json # should get a JSON response

The content negotiation behaviour of falling back to the web-client index.html doesn't seem great at the moment, so I'm looking into how to make that into a 406 or 415 response instead.

[andrewazores]

Generally fixed in the last commit.

$ sh smoketest.sh
$ # in another terminal do:
$ https :8181/api/v2.1/discovery Authorization:"Basic $(echo user:pass | base64)" # should get a JSON response, this is implicitly using Accept:*/*
$ https :8181/api/v2.1/discovery Authorization:"Basic $(echo user:pass | base64)" Accept:text/plain # 406 because content negotiation failed (handler only produces application/json)
$ https :8181/api/v2.1/discovery Accept:text/plain #406 because content negotiation failed (handler only produces application/json) - maybe a 401 would make sense here too, but 406 seems fine
$ https :8181/api/v2.1/discovery Authorization:"Basic $(echo user:pass | base64)" Accept:application/json # should get a JSON response
$ https :8181/api/v2.1/discovery Authorization:"Basic $(echo user:pass | base64)" Accept:application/* # should get a JSON response
$ https :8181/api/v2.1/discovery Authorization:"Basic $(echo user:pass | base64)" Accept:*/* # should get a JSON response
$ https :8181/api/v2.1/nosuchhandler # 404
$ https :8181/api/v2.1/nosuchhandler Accept:application/json # 404

The corner case captured by the final two examples seems a little trickier to solve. There doesn't seem to be a nice way to ask the Vert.x Router if there is a matching route for a given request path, it just determines that internally and calls the handler implementation if there is one. Since paths can include parameters it isn't as easy as simply checking the request path (ex. /api/v2/rules/myrule) against the set of handlers' paths (which would find things like /api/v2/rules/:ruleName). This doesn't really seem worth fixing at this point - the client will receive a 4xx either way. If they get an unexpected 406 then perhaps they retry without the Accept header or use */*, in which case the response is a 404 which is correct anyway.

[andrewazores]

Not sure why the itests keep hanging in CI. They run fine on my machine...

[maxcao13]

Everything looks good, thanks for doing this!

I'm wondering about vert.x consumes since you've included it in the PR along with produces - is including consumes info in relevant endpoints something that should be done separately?

[andrewazores]

I can add it on here as well if you'd like. It's only really relevant for POST endpoints, I think, so there aren't too many.

[andrewazores]

As an example, I modified the AuthTokenPostHandler with:

    @Override
    public List<HttpMimeType> consumes() {
        return List.of(HttpMimeType.MULTIPART_FORM, HttpMimeType.URLENCODED_FORM);
    }

Then we have the following behaviours:

Negotiation failure when POSTing JSON:

$ http -v :8181/api/v2.1/auth/token Authorization:"Basic $(echo user:pass | base64)" resource=/api
POST /api/v2.1/auth/token HTTP/1.1
Accept: application/json, */*;q=0.5
Accept-Encoding: gzip, deflate
Authorization: Basic dXNlcjpwYXNzCg==
Connection: keep-alive
Content-Length: 20
Content-Type: application/json
Host: localhost:8181
User-Agent: HTTPie/3.2.1

{
    "resource": "/api"
}


HTTP/1.1 415 Unsupported Media Type
content-length: 0
content-type: text/plain
vary: origin

Success when POSTing urlencoded form:

 $ http -v -f :8181/api/v2.1/auth/token Authorization:"Basic $(echo user:pass | base64)" resource=/api
POST /api/v2.1/auth/token HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Authorization: Basic dXNlcjpwYXNzCg==
Connection: keep-alive
Content-Length: 15
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:8181
User-Agent: HTTPie/3.2.1

resource=%2Fapi

HTTP/1.1 200 OK
content-encoding: gzip
content-length: 498
content-type: application/json
vary: origin

{
    "data": {
        "result": {
            "resourceUrl": "/api?token=eyJjdHkiOiJKV1QiLCJlbmMiOiJBMjU2R0NNIiwiYWxnIjoiZGlyIn0..wXBKbx-uEILvAtmH.gkEYMmDvuT_ok5RjWBSV_gXY3vf2x_R_W3-VMFgSH4bxVy8cjzULlEiUeUjwm3AT1GrymiNymLJxSakT3l1jJAXOL3vWSA_iTkb-N87_UwLcuBicGpSPxnDPblS2ZshTIxfRTzY2fLeZwQs9CRNAFpa_5ZmwAW13BwG_X95rOyqjz9N3-wHBfRTcO5Cf6AdEWI-0PPhqAtvpERbQGqZ83CC6I5vGjMyGH90dY1BGvS6XRbiRJyVhaS6VIXXseOXw9-yJ8cm4RbwgLGpXwQPgBgoYLu9EYuQZVr6qQNX3ou4XTQYg9YKMZeWteNSNwAJi8PGZUGl63HkaJzCoCdJwwTXPA5NsFKqFvSGLWi0gQp7TeruHPydVJetRNlEAlpw.7dGZQJNOI1sGpLrU66FVfg"
        }
    },
    "meta": {
        "status": "OK",
        "type": "application/json"
    }
}

and success when POSTing multipart form:

$ http -v --multipart :8181/api/v2.1/auth/token Authorization:"Basic $(echo user:pass | base64)" resource=/api
POST /api/v2.1/auth/token HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Authorization: Basic dXNlcjpwYXNzCg==
Connection: keep-alive
Content-Length: 131
Content-Type: multipart/form-data; boundary=3093fe0a8c374db5b59bd7fbc1ba38bc
Host: localhost:8181
User-Agent: HTTPie/3.2.1

--3093fe0a8c374db5b59bd7fbc1ba38bc
Content-Disposition: form-data; name="resource"

/api
--3093fe0a8c374db5b59bd7fbc1ba38bc--


HTTP/1.1 200 OK
content-encoding: gzip
content-length: 498
content-type: application/json
vary: origin

{
    "data": {
        "result": {
            "resourceUrl": "/api?token=eyJjdHkiOiJKV1QiLCJlbmMiOiJBMjU2R0NNIiwiYWxnIjoiZGlyIn0..CskNInMd_NXqdkRb.Ap94JzWLlM9xAsp08MdA9S3Av2ik9Cv40wrAjLr6uJaIbh_iWSEUvXbL9SYK2NFUjaV7EVaVvqLG4vqhbeXicmVfEwfLDsZlTx5GoRKxQFAhlTktVeU86MbhnyQK02Xupx5FXSYc4yJPunyQS10HoA1Kt0P62QZcWdxTRKBkvAKiTM38C2hbeaNnSP5Y8Qn3QSJPB7NN4aSteRwIpS_KrtguCaH9ZDiqe5TN0u2LkIZng77lMwhDnv7ccCQazqnyIJvQrXoya71C8fesIL7C3eW9RE94PbsymTepFS1Td8o3eGLbeVs7KxuNL6pexH-Or1Pjaw3ZsEcAZp0YCMSkZy_2RULldW9kZfwHfTH25gHlA4hMJL9dFvDBAt2CEW8.L8nkHxFLYlgR_ErMxU88Yg"
        }
    },
    "meta": {
        "status": "OK",
        "type": "application/json"
    }
}

[maxcao13]

LGTM! I will probably resume to #880 after the next release.