Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve content negociation middlewares #150

Merged
merged 31 commits into from May 13, 2016
Merged

Improve content negociation middlewares #150

merged 31 commits into from May 13, 2016

Conversation

@arteymix
Copy link
Member

@arteymix arteymix commented Jan 12, 2016

In short, it handles fuzzy matching and set the appropriate header in the response.

using Valum.ContentNegotiation;

app.get ("", accept ("text/html", (req, res) => {
    res.body.write_all ("<!DOCTYPE html><html></html>", null);
})).then (accept ("text/xhtml", (req, res) => {
    res.body.write_all ("...");
}, NegociateFlags.FINAL));

The NegociateFlags.FINAL indicate that this is the last option and a 406 Not Acceptable should be raised on failure.

It needs some tests before being merged.

Fuzzy matching is provided by submitting a custom compare function to
'negociate'.

In short,

 - 'accept' handle '*/*' and 'type/*' patterns.
 - 'accept_charset', 'accept_encoding' and 'accept_language' handle '*'

Set the corresponding response header when a negociation succeeds.

If a charset is supplied to 'accept', it is negociated with
'accept_charset'.

Provide 'NegociateFlags.FINAL' to raise a 'ClientError.NOT_ACCEPTABLE'
if the negociation fails.

Provide minimal tests for 'negociate' and 'accept'.
@arteymix arteymix added the feature label Jan 12, 2016
@arteymix arteymix self-assigned this Jan 12, 2016
@arteymix arteymix added this to the 0.3.0 milestone Jan 12, 2016
@arteymix arteymix force-pushed the negociation-middlewares branch from 63c38b2 to 381f491 Jan 12, 2016
@arteymix arteymix force-pushed the master branch from bef03cf to 3b94bb8 Jan 12, 2016
Fix a bad typo as well, I got messed up with my french.
@arteymix arteymix force-pushed the negociation-middlewares branch from 381f491 to 166c72f Jan 12, 2016
@arteymix
Copy link
Member Author

@arteymix arteymix commented Jan 12, 2016

It would be great to have something that allow us to build sequences and connect the last next to the keep routing after the sequence.

Something like:

app.get ("", sequence(
    accept ("text/xml", produce_xml),
    accept ("text/html", produce_html, NegociateFlags.FINAL)
));

That could get rid of then.

@arteymix arteymix force-pushed the master branch 4 times, most recently from f8ebcac to 6817312 Jan 31, 2016
@arteymix
Copy link
Member Author

@arteymix arteymix commented Feb 5, 2016

It would be great to handle quality values in some way.

arteymix added 2 commits Feb 5, 2016
Provide two examples of negotiated compression.
@arteymix
Copy link
Member Author

@arteymix arteymix commented Feb 5, 2016

To handle quality values properly, we could provide multiple choices to negotiate and let it forward the best possible:

accept ({"text/html", "text/html+xml"}, (req, res, next, stack, content_type) {
    // ...
}};
@arteymix
Copy link
Member Author

@arteymix arteymix commented Feb 5, 2016

The tests work, but it produces a SEGFAULT when running the application :(

@arteymix arteymix force-pushed the negociation-middlewares branch 3 times, most recently from a8437c0 to 877c7d2 Feb 5, 2016
@arteymix
Copy link
Member Author

@arteymix arteymix commented Feb 5, 2016

Okay, everything is correct now!

Since we handle multiple expectations, the default behaviour is to raise a 406 Not Acceptable, which can be overwritten by setting the NegotiateFlags.NEXT.

No expectations always raise a 406 Not Acceptable and a missing header forwards the first expectation.

We need some tests for the edge cases and I have some documentation already ;)

arteymix added 6 commits Feb 5, 2016
Provide a list of expecations to 'negotiate', which will forward the one
with the highest quality value.

Update docs and examples.
As we can handle quality values, it's not useful to call 'next' by
default. Instead, a '406 Not Acceptable' is raised if none of the
expectations can be satisfied.

Replace 'NegotiateFlags.FINAL' by 'NegotiateFlags.NEXT' to call 'next'
instead of raising a '406 Not Acceptable' status.

Provide a 'forward' function to serve as a default value. It let
end-user write the following:

    app.use (accept ({"application/json"}));

Update tests to take these changes into consideration.
Systematically raise a '406 Not Acceptable' if the expectations array is
empty.

Fix the SEGFAULT when the application is running, it looks like a Vala
memory bug.

Fix the example by correcting a typo and setting the
'NegotiateFlags.NEXT' flag.
If no charset were explicitly set, it is assumed to be 'utf-8'. If it's
unchanged, don't do anything.
@arteymix arteymix force-pushed the negociation-middlewares branch from 0c63760 to 18f4003 Feb 5, 2016
Use an explicit reference for 'forward'.
@arteymix arteymix force-pushed the master branch from 01882fc to 100261e Feb 7, 2016
return negotiate ("Accept-Charset", charsets, (req, res, next, stack, charset) => {
HashTable<string, string> @params;
var content_type = res.headers.get_content_type (out @params);
var old_charset = @params["charset"] ?? "utf-8";

This comment has been minimized.

@arteymix

arteymix Feb 7, 2016
Author Member

The default charset for HTTP/1.1 is iso-8859-1 (https://www.w3.org/International/O-HTTP-charset), we might want to make that assumption for accept_charset. The only thing is, it's not much practical for end-user as app will typically produce UTF-8 data.

@arteymix arteymix force-pushed the negociation-middlewares branch from 9196a50 to 87b207c Feb 7, 2016
@arteymix
Copy link
Member Author

@arteymix arteymix commented Feb 25, 2016

The merge will have to deal with a couple of breaking changes:

  • return the result of forward
  • stack is a context (not used anyway)
@arteymix
Copy link
Member Author

@arteymix arteymix commented Feb 29, 2016

Just picked the codecov commit, so that we could see what parts are missing. It should end-up here: https://codecov.io/github/valum-framework/valum?pr=150

Assume the default charset to be 'iso-8859-1' as specified in
'HTTP/1.1'.

Only apply conversion on 'text/*' and use a case-insensitive comparison
for the charset. It's not a convenient to transform non-text data,
especially since it could break binary codings.
@arteymix
Copy link
Member Author

@arteymix arteymix commented May 10, 2016

It would be nice to use a quality list for the expectations and select the best match by taking the highest qvalue product. This is what Apache uses.

app.get ("/", accept ("text/json; q=1, text/xml; q=0.1", (req, res, next, ctx, expectation) => {
    // produce content according to 'expectation'
}));
@arteymix arteymix force-pushed the negociation-middlewares branch from c492769 to 04b93c3 May 10, 2016
arteymix added 8 commits May 11, 2016
The typical use case for this middleware is to explicitly pass a
callback, so it's not useful to invoke 'next' as a default.
The expectations list can now contain a 'q' parameter which specify the
quality of the specified item.

It is negotiated such that the product of the quality and preference is
maximized.
Choosing when to apply charset conversion is highly depending on the
content type and it would be really difficult to perform that correctly.

Instead, just let the end-user decide how charset should be applied.
Update the code to the latest change in the 'master' trunk.
The same result can be performed with a status handler.

Remove the 'NegotiateFlags' as well.
Cover how preference and quality are processed.
@arteymix
Copy link
Member Author

@arteymix arteymix commented May 12, 2016

I merged the changes from the 0.3 so it should be fast-forward into the trunk.

  • support vendor prefix for accept_encoding
  • documentation update
  • fix for qvalue extraction
  • remove NegotiateFlags.NEXT in favour of a status-handler based approach

Some tests are missing and it should be ready to merge!

arteymix added 2 commits May 12, 2016
For trailing item, it would slice until '-1', which would strip the last
digit of the qvalue.
@arteymix arteymix force-pushed the negociation-middlewares branch from e57a917 to 4e279fe May 13, 2016
For the 'accept_language' middleware, make global variant (eg. 'en')
acceptable for regional preference (eg. 'en-GB'). This invert the
precedent behavior which was not appropriate.
@codecov-io
Copy link

@codecov-io codecov-io commented May 13, 2016

Current coverage is 63.28%

Merging #150 into master will increase coverage by +2.11%

@@             master       #150   diff @@
==========================================
  Files            28         28          
  Lines          1025       1084    +59   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits            627        686    +59   
  Misses          398        398          
  Partials          0          0          
  1. File ...i/vsgi-response.vala was modified. more
    • Misses -2
    • Partials 0
    • Hits +2

Powered by Codecov. Last updated by b7870a2...42c1ab4

@arteymix arteymix force-pushed the negociation-middlewares branch from 42c1ab4 to 426368e May 13, 2016
@arteymix arteymix merged commit 426368e into master May 13, 2016
3 checks passed
3 checks passed
codecov/project 63.38% (+2.20%) compared to b7870a2
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@arteymix arteymix deleted the negociation-middlewares branch May 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.