Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nginx aarch64-musl cannot serve over HTTPS #30945

Closed
cyckl opened this issue May 17, 2021 · 5 comments
Closed

nginx aarch64-musl cannot serve over HTTPS #30945

cyckl opened this issue May 17, 2021 · 5 comments
Labels
bug Something isn't working

Comments

@cyckl
Copy link

cyckl commented May 17, 2021 •
edited
Loading

System

It's a Raspberry Pi 4B

  • xuname:
    • Void 5.4.83_1 aarch64-musl Unknown uptodate rF
  • package:
    • nginx-1.18.0_4

Custom image built with rpi4-kernel and rpi4-base

Expected behavior

After serving a site on SSL, nginx should send correct response to request for HTTPS content

The following example was taken from curl output with the following conditions:

  • Server is running on Void x86-64
    • Void 5.11.18_1 x86_64 GenuineIntel uptodate rFFF
  • Same config
  • Same certificate
  • Same nginx version
    • nginx-1.18.0_4
  • Domain name has been censored.
~ # curl -vi https://example.com
*   Trying 127.0.0.1:443...
* Connected to example.com (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=example.com
*  start date: May 16 21:47:47 2021 GMT
*  expire date: Aug 14 21:47:47 2021 GMT
*  subjectAltName: host "example.com" matched cert's "example.com"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.76.1
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
< Server: nginx/1.18.0
Server: nginx/1.18.0
< Date: Mon, 17 May 2021 08:04:26 GMT
Date: Mon, 17 May 2021 08:04:26 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 153
Content-Length: 153
< Connection: keep-alive
Connection: keep-alive
< Strict-Transport-Security: max-age=31536000; includeSubDomains
Strict-Transport-Security: max-age=31536000; includeSubDomains

< 
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.18.0</center>
</body>
</html>
* Connection #0 to host example.com left intact

Actual behavior

nginx will send an empty response on SSL requests, at least on my aarch64-musl install of the package

The following example was taken from curl output with the following conditions:

  • Server is running on Void aarch64-musl
    • Void 5.4.83_1 aarch64-musl Unknown uptodate rF
  • Same config
  • Same certificate
  • Same nginx version
    • nginx-1.18.0_4
  • Domain name has been censored.
* Connected to example.com (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/cacert.pem
*  CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [19 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4060 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=example.com
*  start date: May 16 21:47:47 2021 GMT
*  expire date: Aug 14 21:47:47 2021 GMT
*  subjectAltName: host "example.com" matched cert's "example.com"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x15b00d400)
} [5 bytes data]
> GET / HTTP/2
> Host: example.com
> user-agent: curl/7.76.0
> accept: */*
> 
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [57 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [57 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
} [5 bytes data]
* Empty reply from server
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Closing connection 0
} [5 bytes data]
* TLSv1.3 (OUT), TLS alert, close notify (256):
} [2 bytes data]
curl: (52) Empty reply from server

Steps to reproduce the behavior

  • Configure nginx on any aarch64 / aarch64-musl (not sure yet) Void install
  • Enable SSL on a server block
  • Access from a browser / curl from website

Extra information

  • nginx does not log the error in the error.log located in /var/log/nginx/error.log
  • Configuration is known-good on Raspbian
  • Attempted to use stock config shipped with package on Void and enabling SSL to no avail, same issue
  • Works on x86-64 with the exact same configuration, certificates, domain, package version

Ideas

As I've tested the exact same package on different architectures, the only difference between the two is that there is a patch for the nginx package applied for ARM systems. It appears that the patch is in order to adjust certain configuration values on compilation to fit the ARM architecture, which also leads me to believe that it could be related. A good way to test would be with a few different architectures to see if it's only limited to ARM or maybe something platform specific.

It could also be musl specific, as that is another difference between the two systems and that difference has been known to break packages historically.
@cyckl cyckl changed the title nginx aarch64 cannot serve SSL content nginx aarch64-musl cannot serve SSL content May 17, 2021
@cyckl cyckl changed the title nginx aarch64-musl cannot serve SSL content nginx aarch64-musl cannot serve over HTTPS May 17, 2021
@Johnnynator Johnnynator added the bug Something isn't working label May 17, 2021
@Johnnynator
Copy link
Member

Can confirm, this also happens on aarch64 glibc

@paper42
Copy link
Member

paper42 commented May 17, 2021

3 weeks ago I had HTTPS working on my RPI3B (aarch64-musl), I can not test it now. The only difference was that I was using a local CA certificate.

@Johnnynator
Copy link
Member

Johnnynator commented May 17, 2021 •
edited
Loading

This only affects http2, and seems to be a cross compilation issue, when compiled natively it seems to work on aarch64 (glibc).

@cyckl can you check if your config works without http2?

@Johnnynator
Copy link
Member

https://github.com/void-linux/void-packages/blob/master/srcpkgs/nginx/template#L88

this is most likely the issue, we use a ton of wrong values for aarch64 in there.

@cyckl
Copy link
Author

cyckl commented May 17, 2021

This worked perfectly. Thank you so much! For future reference I remember turning off http2 without any luck...

atweiden added a commit to atweiden/voidpkgs that referenced this issue May 17, 2021
@atweiden
nginx: add auto_config.h file for aarch64
854b146 
closes void-linux/void-packages#30945

void-linux/void-packages@e3f3543
hazayan pushed a commit to hazayan/void-packages that referenced this issue May 21, 2021
@Johnnynator
nginx: add auto_config.h file for aarch64
9ff9783 
closes void-linux#30945
@Duncaen Duncaen mentioned this issue Aug 11, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Johnnynator @paper42 @cyckl