Resolving the Issue of Nginx+Tomcat Returning 400 Error with Special Characters in the URL

发表于 分类于 Disqus： 本文字数： 894 阅读时长 ≈ 3 分钟

I encountered an unusual problem where I would receive a 400 error when accessing Tomcat through Nginx if my request URL contained square brackets. The error page did not provide any specific error message, and there were no corresponding error logs in the backend.(English version Translated by GPT-3.5, 返回中文)

Problem Description

Nginx Configuration

1
2
3
4
5
6
7
location /files/ {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://127.0.0.1:8080/apps/files/;
}

Tomcat Servlet Configuration

1
2
3
4
@WebServlet(name = "FileServlet", urlPatterns = "/file/*", loadOnStartup = 1)
public class FileServlet extends BaseServlet {
...
}

Request without square brackets (No Issue)

1
2
3
http://host/files/folder1/folder2
-> 
http://127.0.0.1:8080/apps/files/folder1/folder2

Request with square brackets (Causing the Issue)

1
2
3
http://host/files/folder1/folder2/01-[1]-file.file
-> 
http://127.0.0.1:8080/apps/files/folder1/folder2/01-[1]-file.file

Error Page

error

The error page did not provide any specific error message. However, if I replace the square brackets with curly brackets, the following error is displayed.

error2

Troubleshooting Process

Research

After extensively searching on Google, I found several suggested solutions. Most of them recommended removing the URI from the proxy_pass directive, while others mentioned:

  1. A large request header
  2. Missing proxy_set_header Host $host directive

However, none of these solutions resolved my issue, as my request URL and header were relatively small, and I had already included the proxy_set_header Host $host directive.

Enabling Tomcat Debug Mode

Since there were no error logs and the servlet did not receive the request, I decided to enable Tomcat’s Debug mode myself. Enabling debug mode is simple, just add the following line to /${tomcatRoot}/conf/logging.properties:

1
.level = FINE

I added it at the following location:

1
2
3
4
5
6
7
8
9
10
11
...
############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################
# 增加下面这一行, 其他不用改
.level = FINE
# 增加上面这一行, 其他不用改
1catalina.org.apache.juli.AsyncFileHandler.level = FINE
1catalina.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
...

After starting Tomcat with this configuration, numerous debug logs were printed. Then, when I made the request again, I found the following log entry:

1
2
3
4
5
6
7
8
9
10
11
12
.....
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read Socket: [org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@7de71b73:org.apache.tomcat.util.net.NioChannel@657aef19:java.nio.channels.SocketChannel[connected local=/127.0.0.1:8080 remote=/127.0.0.1:47234]], Read direct from socket: [743]
10-Feb-2020 20:32:26.697 FINE [http-nio-8080-exec-2] org.apache.coyote.http11.Http11InputBuffer.parseRequestLine Received [GET 
# 这一行
http://127.0.0.1:8080/apps/files/folder1/folder2/01-[1]-file.file HTTP/1.0
# 这一行
Host: host
X-Real-IP: xxx.xxx.xxx.xxx
X-Forwarded-Proto: https
X-Forwarded-For: xxx.xxx.xxx.xxx
Connection: close
....

At this point, the URL I passed into the browser was already URL-encoded. I attempted to encode the portion after “01-[1]-file.file” for a second time, resulting in “01-%255B1%255D-file.file” (URL-encoded twice). I then made the request and observed the following URL:

1
2
3
4
5
6
7
8
9
10
11
...
10-Feb-2020 20:36:50.698 FINE [http-nio-8080-exec-1] org.apache.coyote.http11.Http11InputBuffer.parseRequestLine Received [GET 
# 这一行
http://127.0.0.1:8080/apps/files/folder1/folder2/01-%255B1%255D-file.file HTTP/1.0
# 这一行
Host: host
X-Real-IP: xxx.xxx.xxx.xxx
X-Forwarded-Proto: https
X-Forwarded-For: xxx.xxx.xxx.xxx
Connection: close
...

I noticed that this URL was exactly the same as the one entered in the browser. Therefore, I concluded that Nginx did not decode the second URL segment, while the first URL segment was decoded correctly by Nginx.

Identifying the Problem

Upon reviewing the proxy_pass documentation on the official Nginx website, Module ngx_http_proxy_module, I found the following relevant information:

When proxy_pass points to a specified URL, the matching portion of the URL is replaced before forwarding the request to the backend web service.

location /name/ {
proxy_pass http://127.0.0.1/remote/;
}

This means that a request to http://host/name/aaa will be forwarded to http://127.0.0.1/remote/aaa.

If proxy_pass does not include any URI, the request is forwarded to the backend web service in its original form without any modification or matching:

location /some/path/ {
proxy_pass http://127.0.0.1;
}

This means that a request to http://host/some/path/ will be forwarded to http://127.0.0.1/some/path/.

When using variables in the proxy_pass directive:

location /name/ {
proxy_pass http://127.0.0.1$request_uri;
}

This means that a request to http://host/name/aaa/bbb will be forwarded to http://127.0.0.1/name/aaa/bbb.

In cases like these, the URI is passed to the server as is.

Solution

According to the documentation, I have two possible solutions (the majority suggests the first one):

1
2
3
4
5
6
7
location /apps/files/ {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://127.0.0.1:8080;
}

or

1
2
3
4
5
6
7
location /apps {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://127.0.0.1:8080$request_uri;
}

For the first solution, if I want to access /apps/image, I would have to configure it separately. Therefore, the second solution satisfies my requirements. However, I have yet to find a solution for scenarios like the following:

Unresolved Scenario

1
2
3
4
5
http://host/content/folder1/folder2/01[1].file
->
http://127.0.0.1:8080/files/folder1/folder2/01[1].file

因为如国按照第一个配置, 我的tomcat的访问必须是/content, 如果使用第二种就变成了http://127.0.0.1:8080/content/folder1/folder2/01[1].file, 而tomcat中也没有/content