"UTF-8" Encoding problem

[youssefguenoun]

Hi,

We encounter a problem that we have observed recently, with “UTF-8” encoding. This problem is obviously related to the integration of HDIV Framework to our “Spring MVC / Spring Security” application.

Concerning the management of “UTF-8” encoding, we proceeded as follows:

1- We have activated the Spring Character encoding filter "org.springframework.web.filter.CharacterEncodingFilter" and we have positioned it as the first filter in our deployment descriptor "web.xml" like this:

<!-- Spring Security Char Encoding Filter-->
<filter>
<filter-name>characterEncodingFilter</filter-name>
<filter-class>
org.springframework.web.filter.CharacterEncodingFilter
</filter-class>
<init-param>
<param-name>encoding</param-name>
<param-value>UTF-8</param-value>
</init-param>
<init-param>
<param-name>forceEncoding</param-name>
<param-value>true</param-value>
</init-param>
</filter>
<!-- Spring Security Filter-->
<filter>
<filter-name>springSecurityFilterChain</filter-name>
<filter-class>
org.springframework.web.filter.DelegatingFilterProxy
</filter-class>
</filter>

<!-- the Filter’s mappings -->
<filter-mapping>
<filter-name>characterEncodingFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

<filter-mapping>
<filter-name>springSecurityFilterChain</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

2- Then we set up Tomcat to handle "UTF-8" encoding, by changing the file "sever.xml" like this:
…
<Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" URIEncoding="UTF-8" useBodyEncodingForURI="true"/>
…..
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443"
URIEncoding="UTF-8" useBodyEncodingForURI="true"/>

Without the integration of HDIV Framework, i.e.: only with Spring MVC and Spring Security (+ tomcat setup / and Spring character encoding filter) we have no problem with "UTF-8" encoding.

With the activation of HDIV Framework (Filter + RequestDataValueProcessor, JSTL-HdIV Taglib, etc.) the problem occurs.

Our Spring configuration files:
• HDIV (hdiv-config.xml : https://gist.github.com/4672835),
• Spring Security(security-config.xml : https://gist.github.com/4672833),
• Spring MVC(presentation-config.xml : https://gist.github.com/4672840),
• And our deployment descriptor (web.xml :https://gist.github.com/4672821).

You will also find screenshots of a use case (domain entity update), that permit to reproduce the encoding problem.

In advance, thanks’ for your help!

Regards,

Youssef

[gillarramendi]

Hi Youssef,

we have detected the source of error: InitListener is executed before CharacterEncodingFilter and we read some parameter from the request. This causes the new encoding (UTF-8) does not apply.

We are working to find a solution.

Thanks for the complete information

[youssefguenoun]

Hi Gotzon,
OK thank's for the diagnostic !

So, we'll wait your solution :-)

Regards,

[youssefguenoun]

I fixed the problem with this dirty patch :

public class InitListenerFixEncoding extends InitListener {
private static final Logger LOGGER= LoggerFactory.getLogger(InitListenerFixEncoding.class);
@OverRide
public void requestInitialized(ServletRequestEvent sre) {
HttpServletRequest request = (HttpServletRequest) sre.getServletRequest();
try {
request.setCharacterEncoding("UTF-8");
} catch (UnsupportedEncodingException e) {
LOGGER.debug("Encoding UTF-8 is not supported !",e);
}
super.requestInitialized(sre);
}
}

But ideally InitListner must be executed after CharacterEncodingFilter !

Regards,
Youssef

[ghost]

Thanks Youssef, your solution worked for me too.

[gillarramendi]

Issue resolved in trunk with this commit: 3cacefb

Now request is not 'touched' in InitListener and CharacterEncodingFilter works correctly.

Thanks for your help!

[youssefguenoun]

Cool Thanks !! we're waiting for the release ;o)

[mlegnani]

Hi all,
I had the same problem, then I applied your patch and initially it seemed to work but now I have a new problem: when a special character is sent trough an hidden field, HDIV detects it to be changed (while it is not).
Here is mi code:
in the jsp:
<form:hidden path="nomeAzienda" />

the value of "nomeAzienda" is è-test and this is hdiv log message

20130318 095738 INFO org.hdiv.logs.Logger INVALID_PARAMETER_VALUE;/myapp/pratiche/savempart.html;nomeAzienda;è-test;0:0:0:0:0:0:0:1;0:0:0:0:0:0:0:1;user

Did you find a solution to this problem?
Thanks a lot
Massimo

[gillarramendi]

Hi Massimo,

resolved on the trunk: 2134eda

Thank you!

If you cant update to the last snapshot of HDIV you can disable content escaping in the hidden field as a temporal fix:

<form:hidden path="username" htmlEscape="false"></form:hidden>

[mlegnani]

Thanks a lot! When do you think the new release will be available?

[gillarramendi]

Hi,

Hdiv 2.1.4 release is public on Maven repositories: http://search.maven.org/#artifactdetails%7Corg.hdiv%7Chdiv%7C2.1.4%7Cpom

Regards