Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to make FormHttpMessageConverter add the charset in the Content-Type header [SPR-16613] #21154

Closed
spring-projects-issues opened this issue Mar 19, 2018 · 3 comments
Assignees
Labels
in: web type: enhancement
Milestone

Comments

@spring-projects-issues
Copy link
Collaborator

spring-projects-issues commented Mar 19, 2018

Mauro Molinari opened SPR-16613 and commented

When writing out a form that does not need multipart encoding, FormHttpMessageConverter does this in org.springframework.http.converter.FormHttpMessageConverter.writeForm(MultiValueMap<String, String>, MediaType, HttpOutputMessage): if the request headers do not specify a Content-Type (which is the case most of the times, when you use RestTemplate and the RequestEntity is built automatically from the supplied HTTP method and body object), application/x-www-form-urlencoded is added as the content type and the charset used for encoding parameter values is by default (if not specified otherwise) UTF-8.
So, the default behaviour of FormHttpMessageConverter is to encode parameters with UTF-8 and assume the server does the same (to decode them) when just application/x-www-form-urlencoded is specified as a content type.
However, there are cases in which the server may use another "implicit" encoding for this kind of data: Tomcat, for instance, uses ISO-8859-1 in its default configuration (or at least it did so until version 6 or 7, I've not checked what happens in more recent versions). Today I also encountered another web server, running PHP scripts, that assumes ISO-8859-1 if no encoding is specified in the Content-Type.

The solution to this problem is, either:

  • set a different charset on the FormHttpMessageConverter, like ISO-8859-1 in the above example: this is however cumbersome (it requires to cycle through all the default registered HttpMessageConverters on the RestTemplate to find the correct one and change the encoding) and it also limits the characters that can be correctly encoded and transferred to the server (as an example: ISO-8859-1 is missing the € sign)
  • otherwise, you have to find a way to make FormHttpMessageConverter explicitly set the charset in the Content-Type, like this: Content-Type: application/x-www-form-urlencoded;charset=UTF-8

IMHO, I can't think of a good reason for which FormHttpMessageConverter shouldn't do the latter by default (compatibility reasons with very old servers??). In my real-world scenario I see that I can make the server correctly use UTF-8 to decode my form data when I explicitly set the charset in the content type, overriding the default ISO-8859-1 that is used otherwise.

Extending FormHttpMessageConverter to do this is really a pain, because this class was not designed with extensibility in mind: there's not getter for the private charset field, many methods (like writeForm(...) and isMultipart(...)) are private, no getter for partConverters, etc.


Affects: 5.0.4

Referenced from: commits 2e4963f, 5861e96, 1897d8e

0 votes, 5 watchers

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Mar 28, 2018

Andy Wilkinson commented

The fix for this issue has necessitated a change in Spring REST Docs. It previously checked for form-encoded requests using the following:

MediaType.APPLICATION_FORM_URLENCODED.equals(result.getRequestHeaders().getContentType())

As of this change, the check returns false. I've updated it to the following:

MediaType.APPLICATION_FORM_URLENCODED.isCompatibleWith(result.getRequestHeaders().getContentType())

That's probably what it should have been all along, but I thought it worth noting here in case anyone else has the same problem.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Mar 28, 2018

Mauro Molinari commented

Thank you Rossen for the fix!

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Mar 28, 2018

Rossen Stoyanchev commented

Since this is merely a convenience, I've removed 4.3.1.5 from the fix version, leaving it in 5.0.5 only. The likelihood of breaking existing tests, and potentially large number of them is high.

@spring-projects-issues spring-projects-issues added type: enhancement in: web labels Jan 11, 2019
@spring-projects-issues spring-projects-issues added this to the 5.0.5 milestone Jan 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: web type: enhancement
Projects
None yet
Development

No branches or pull requests

2 participants