Randomly receiving 400 bad request

[rodrigobdz]

I'm using nghttpx between a client and an apache server. When the client makes a request to the server through nghttpx, the client sometimes gets 400 Bad Request and it sometimes doesn't. This happens also when making the same request. When making the request directly to the server I don't get 400 Bad Request.

Client is Chrome Version 54.0.2840.59 (64-bit)
nghttpx version is v1.15.0
The nghttpx call is:
nghttpx --frontend-frame-debug --frontend=0.0.0.0,8080 --backend=127.0.0.1,443 <SSL_KEY> <SSL_CERT>

Attached are screenshots from Google Chrome's response headers.

When it fails these headers are shown

Otherwise in case of a successful request, these ones are shown

[tatsuhiro-t]

Is the connection between chrome and nghttpx http/2?
If so, nghttpx does not generate 400 response. If that is HTTP/1, it may do in some cases.
Could you provide nghttpx INFO level log for the each case: success and failure?

[tatsuhiro-t]

nghttpx may return 400 even with HTTP/2 frontend connection if backend connection is also HTTP/2, just in case.

[rodrigobdz]

Between chrome and nghttpx and between nghttpx and apache I'm using HTTP/2.

[rodrigobdz]

This is the log when receiving 400 Bad Request

[tatsuhiro-t]

Unfortunately, this is not the INFO level log I mentioned. Try -LINFO option.

[rodrigobdz]

In the attachment you can find the INFO level log. Please let me know if you need anything else.
nghttpx_info_log.txt

[tatsuhiro-t]

For some reason, it does not include INFO log. Could you remove --frontend-frame-debug option?
By default, log is written to stderr.

[rodrigobdz]

I just removed the --frontend-frame-debug option but now nothing is being logged to the console.

[tatsuhiro-t]

It is very strange. Do you have /etc/nghttpx/nghttpx.conf?

[rodrigobdz]

I do, however, I haven't changed its contents.
Here the uncommented content in /etc/nghttpx/nghttpx.conf:

frontend=127.0.0.1,3000;no-tls
backend=127.0.0.1,80
errorlog-syslog=yes
workers=1

[tatsuhiro-t]

Did you uncomment those lines? If it is not intended, please comment out them again.
errorlog-syslog=yes forwards log to syslog.

[rodrigobdz]

What I meant was that I was just pasting the uncommented lines in the file. I did not uncomment/comment out anything in the file. It is in its default state

[tatsuhiro-t]

Well, if you don't use syslog for nghttpx, could you comment out errorlog-syslog=yes?

[rodrigobdz]

I commented out that line and the info log is now in the attachment. The failed request is at the end. From what I have observed, the proxy accepts always the first request and on the second one it delivers code 400. The problem is not only with the second request.
nghttpx_info_log.txt

[tatsuhiro-t]

It looks like the backend connection is HTTP/1.1.
From the log, I see that nghttpx sends request to backend server, and that is the backend server returned 400 response code. Could you check the backend server why it responded with that status code?

[rodrigobdz]

I will check that. However, I can not understand why it works the first time and the second it doesn't.

[rodrigobdz]

By changing the server port through which the proxy was communicating no request fails with 400 error. I opted to use port 80. There seems to be a protocol negotiation problem between the server and nghttpx when I use port 443 for backend.

This is how I start the nghttpx now:
nghttpx --frontend-frame-debug --frontend=0.0.0.0,8080 --backend=127.0.0.1,80 <SSL_KEY> <SSL_CERT>

[tatsuhiro-t]

If backend requires TLS, you need to specify tls parameter to --backend option like this: --backend="127.0.0.1,443;;tls". By default, backend connection is cleartext TCP.
As far as I know, Apache may return 400 if it receives request in cleartext TCP on the port where Apache is configured to use TLS. Still not sure why first request succeeded.

[rodrigobdz]

For some reason, nghttpx is not parsing correctly the cli arguments. It is not recognizing the backend option. I also tried to disable TLS in frontend but that is not being recognized either.
See attachments.
I'm using nghttpx v1.16.0 compiled from git.

[mushiwords]

I suggest you to use the configure file like nghttpx.conf as the configuration , The configure axample you can find in the project path nghttp2/nghttpx.conf.sample . Then youshould start the nghttpx with nghttpx --conf=path/to/your/nghttpx.conf
Good luck !

[tatsuhiro-t]

; has special meaning in shell. You have to quote it with ' or "; e.g., --backend="127.0.0.1,443;;tls"

[rodrigobdz]

Thanks, I ended up using a configuration file as suggested by @yaung