UTF-8 not working in Java Application running in Tomcat server-CodePudding

I am working on a Java application. I need to get UTF-8 encoding in my Java webapp to support Bengali (বাংলা) text. I have done the following:

Tomcat's server.xml

<Connector port="8080"
    protocol="HTTP/1.1"
    connectionTimeout="20000"
    redirectPort="8443"
    URIEncoding="UTF-8" />

<Connector executor="tomcatThreadPool"
    port="8080"
    protocol="HTTP/1.1"
    connectionTimeout="20000"
    redirectPort="8443"
    URIEncoding="UTF-8" />

<Connector protocol="AJP/1.3"
    address="::1"
    port="8009"
    redirectPort="8443"
    URIEncoding="UTF-8" />

JVM defaultCharset in catalina.bat file

set JAVA_OPTS=%JAVA_OPTS% -Dfile.encoding=UTF-8

properties in application.properties

spring.datasource.url=jdbc:mysql://localhost:3306/database_name?useUnicode=true\&characterEncoding=UTF-8
spring.datasource.tomcat.connection-properties=useUnicode=true;characterEncoding=UTF-8

spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true

server.tomcat.uri-encoding=UTF-8
spring.webflux.multipart.headers-charset=UTF-8
spring.thymeleaf.encoding=UTF-8

meta tag in html file

<!doctype html>
<html lang="en" xmlns:th="http://www.thymeleaf.org" xmlns:sec="http://www.thymeleaf.org/extras/spring-security">
    <head>
        <meta charset="utf-8">
    </head>

    <body>
    </body>
</html>

utf-8 support in form tag

<form enctype="multipart/form-data" accept-charset="UTF-8" action="#" th:action="@{/create}" th:object="${object}" th:method="POST">
    <div >
        <label for="name" >Name</label>
        <input type="text"  id="name" name="name" th:field="*{name}" placeholder="Enter Name">
    </div>
<div >
    <label for="photo">Photo</label>
    <input type="file"  id="photo" name="photo"/>
</div>
    <div>
        <button  type="submit">Submit</button>
    </div>
</form>

MySQL configuration (my.ini)

[client]
default-character-set = utf8mb4

[mysql]
default-character-set = utf8mb4

MySQL properties:

Database:
Default collation: utf8mb4_0900_ai_ci
Default charactterset: utf8mb4

Table:
Table collation: utf8mb4_0900_ai_ci

Column:
Type: varchar(255)
Character Set: utf8mb4
Collation: utf8mb4_0900_ai_ci

Configuration:

Java 11.0.2
Tomcat 8.5
MySQL 8.0.16
Spring Boot 2.2.4
Maven 3.8.1
Windows Server 2019 Standard (Production) Windows 10 Home (Development)

When I submit a form with value আনোয়ার, it is saved as à¦†à¦¨à§‹à§Ÿà¦¾à¦°

How can I solve this problem?

When I run the application from eclipse it works fine. But when the war file is deployed in Tomcat server it does not work.

I tried the following code. It prints à¦†à¦¨à§‹à§Ÿà¦¾à¦° in tomcat8-stdout file. So I think problem occurs while transferring data from browser to server, from server to database is fine.

@PostMapping("/create")
public String create(@ModelAttribute("object") Object object, @RequestParam("photo") MultipartFile photo) throws IOException {
    System.out.println(object.getName());
    return "redirect:/index";
}

CodePudding user response：

The encoding of POST parameters is not set at the Connector level, but on the ServletRequest object.

Tomcat provides a filter to set it, as explained in the documentation.

Add this to your web.xml file:

<filter>
  <filter-name>setCharacterEncodingFilter</filter-name>
  <filter-class>org.apache.catalina.filters.SetCharacterEncodingFilter</filter-class>
  <init-param>
    <param-name>encoding</param-name>
    <param-value>UTF-8</param-value>
  </init-param>
</filter>

<filter-mapping>
  <filter-name>setCharacterEncodingFilter</filter-name>
  <url-pattern>/*</url-pattern>
</filter-mapping>

CodePudding user response：

As you are using Spring, you can try using CharacterEncondingFilter to enforce UTF-8 encoding.

You can find multiple examples on how to do that. Consider for instance this or this other.

Basically, you need to register the filter somewhere in your Java configuration. Taking one of the indicated examples:

@Bean
@Order(Ordered.HIGHEST_PRECEDENCE)
public FilterRegistrationBean<CharacterEncodingFilter> characterEncodingFilterRegistration() {
  CharacterEncodingFilter filter = new CharacterEncodingFilter();
  filter.setEncoding("UTF-8"); // use your preferred encoding
  filter.setForceEncoding(true); // force the encoding

  FilterRegistrationBean<CharacterEncodingFilter> registrationBean =
    new FilterRegistrationBean<>(filter); // register the filter
  registrationBean.addUrlPatterns("/*"); // set preferred url
  return registrationBean;
}

In fact, this registration process should be performed automatically by the Spring Boot HttpEncodingAutoConfiguration. Please, note the requirements for that:

@Configuration(proxyBeanMethods=false)
@EnableConfigurationProperties(value=ServerProperties.class)
@ConditionalOnWebApplication(type=SERVLET)
@ConditionalOnClass(value=org.springframework.web.filter.CharacterEncodingFilter.class)
@ConditionalOnProperty(prefix="server.servlet.encoding",
                       value="enabled",
                       matchIfMissing=true)

As you can see, the registration of the filter is related to properties with the server.servlet.encoding prefix.

Therefore, as an alternative, to properly configure the charset filter, another thing you could try is to configure your application using the server.servlet.encoding.* related properties:

server.servlet.encoding.charset=UTF-8
server.servlet.encoding.force=true

instead of the old version of that properties, based on the spring.http.encoding prefix, you are currently using in your application.properties configuration.