net/haproxy: site redirects fails with ERR_HTTP2_SERVER_REFUSED_STREAM with LibreSSL #2013

Taomyn · 2020-09-03T05:40:11Z

Describe the bug
Since upgrading OPNsense to v20.7.2 many of my HAProxy site redirects fails with "ERR_HTTP2_SERVER_REFUSED_STREAM" when I try to access it - they all seem to be SSL backend servers.

To Reproduce
Steps to reproduce the behavior:

Visit site
Browser reports "ERR_HTTP2_SERVER_REFUSED_STREAM"

Expected behavior
Normal access to site

Relevant log files
Cannot find anything logged on the issue

Additional context
The site in question is my PRTG instance, running on HTTPS port 8081 - if I directly connect to this it works fine, as does the PRTG desktop app that connects directly on my PC. However the PRTG app on my phone which uses the HAProxy redirect has the same connection problem.

The settings does include enabling HTTP2 but I tried disabling this and the same thing occurs.

Environment
OPNsense 20.7.2-amd64
FreeBSD 12.1-RELEASE-p8-HBSD
LibreSSL 3.1.4
os-haproxy (installed) 2.24

Taomyn · 2020-09-04T05:02:39Z

Found another one of my sites not working, all the log tells me is:

2020-09-04T07:00:29 | haproxy[6175] | 192.168.1.19:60997 [04/Sep/2020:07:00:29.781] Web_Server_SSL~ Unifi/Unifi 0/0/0/-1/5 -1 0 - - SD-- 20/20/0/0/0 0/0 "GET / HTTP/2.0"

I just tried disabling HTTP/2 on HAProxy and I get further but then constant errors from the sites and messages like ERR_EMPTY_RESPONSE

mimugmail · 2020-09-04T07:48:36Z

Can you try to revert and check if the error still persists?

opnsense-revert -r 20.7.1 os-haproxy

Taomyn · 2020-09-04T08:02:24Z

Can you try to revert and check if the error still persists?

opnsense-revert -r 20.7.1 os-haproxy

Tried it, but it was the same - I restarted the HAProxy service to be sure but no difference. Do I need to reboot the firewall?

root@bart:~ # opnsense-revert -r 20.7.1 os-haproxy
Fetching os-haproxy.txz: ... done
Verifying signature with trusted certificate pkg.opnsense.org.20200313... done
os-haproxy-2.24: already unlocked
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        os-haproxy: 2.23

Number of packages to be installed: 1
[1/1] Installing os-haproxy-2.23...
Extracting os-haproxy-2.23: 100%
Stopping configd...done
Starting configd.
Keep version OPNsense\HAProxy\HAProxy (2.10.0)
Reloading plugin configuration
Configuring system logging...done.
Reloading template OPNsense/HAProxy: OK
Reloading template OPNsense/Syslog: OK

os-haproxy (installed)	2.23

mimugmail · 2020-09-04T08:16:13Z

It was bumped from 2.0.14 to 20.0.17, but also renamed to haproxy20, @fichtner opnsense-revert will not help here correct?

fichtner · 2020-09-04T08:21:17Z

I have no idea what the issue is, please wait for @fraenki to take a look

sorano · 2020-09-04T10:06:08Z

A longshot but could it be related to:

"honor sort order of all rules, remove special handling of use_[backend|server] options"

from here:
#1933

Take a look at the backend if you are using rules.

Taomyn · 2020-09-04T10:22:05Z

A longshot but could it be related to:

"honor sort order of all rules, remove special handling of use_[backend|server] options"

from here:
#1933

Take a look at the backend if you are using rules.

Should I re-update to the latest os-haproxy to check this? I can't see anything regarding sort order. Here's what is set for one of the failing sites which is what was set before upgrading to 20.7.2:

I also spotted this at the bottom of each window:

sorano · 2020-09-04T10:51:39Z

Looking at your pictures it looks like the PRTG rule isn't used there. So I guess the rules are used in your frontend?

Do you still see that error at the bottom if you go "re-update" to latest haproxy plugin?

Taomyn · 2020-09-04T11:02:52Z

Looking at your pictures it looks like the PRTG rule isn't used there. So I guess the rules are used in your frontend?

Yes

Do you still see that error at the bottom if you go "re-update" to latest haproxy plugin?

It goes when I upgrade again, and unfortunately does not fix the issue.

sorano · 2020-09-04T11:09:32Z

It goes when I upgrade again, and unfortunately does not fix the issue.

Alright, so in that case I would stay at the updated version and then I would try to move the PRTG rule to the start before the maltrail rule. Do you get any warnings when you "Test syntax"?

If it's still not working when prtg is first then at least you know it's not related to sort order.

Btw, if all you do in rules are matching hostname to backend I really recommend using map files.

Taomyn · 2020-09-04T11:21:54Z

Alright, so in that case I would stay at the updated version and then I would try to move the PRTG rule to the start before the maltrail rule. Do you get any warnings when you "Test syntax"?

Moved it, no change - I always run a syntax check, and no errors before I saved ie.

If it's still not working when prtg is first then at least you know it's not related to sort order.

Btw, if all you do in rules are matching hostname to backend I really recommend using map files.

Map Files?

Taomyn · 2020-09-04T11:31:50Z

I also notice, that even though I have detailed logging switched on for the front-send, I see nothing for these non-working sites - but I see plenty for the ones that do work.

sorano · 2020-09-04T11:36:26Z

I also notice, that even though I have detailed logging switched on for the front-send, I see nothing for these non-working sites - but I see plenty for the ones that do work.

Hmmm. I guess you should take a look at the server logs for the backends that are not working in that case. (look at webserverlogs from the server hosting PRTG. Not HAProxy).

I guess you could also try creating a new frontend with basic settings on another port, then put the PRTG backend behind that and test.

Regarding map files, they make it easier to match many hostnames and backends in a single rule, also map files can be edited on the fly. HAProxy configs can be overwhelming as is, so map files really makes it less cluttered. With map files you only need one rule like:

https://www.haproxy.com/blog/introduction-to-haproxy-maps/

Taomyn · 2020-09-04T12:45:32Z

Yeah I've looked at the backend logs for PRTG and there's no sign of anything there, but if I'm not seeing even an attempt logged by HAProxy why would there be? Nothing on the firewall is showing anything. I do however see logging of direct connections to PRTG by my local desktop client, so the web service itself is working.

Thanks, I'll consider the maps once this issue is fixed.

Taomyn · 2020-09-04T13:23:38Z

More info. Was using the PRTG test tool to perform some raw HTTP calls when I noticed that following a restart of the HAProxy service, one connection to the server works, but then all subsequent attempts fail. l used the same command as my browser was trying to send which is the only thing logged by HAProxy:

fraenki · 2020-09-04T13:34:23Z

@Taomyn Please paste your full haproxy.conf (as text, not as screenshot). When cloaking sensitive information, please don't modify the syntax so that we still have a valid test config.

Taomyn · 2020-09-04T13:49:06Z

@fraenki as requested https://pastebin.com/j44jVMw7

Taomyn · 2020-09-07T05:37:11Z

Seems a couple more of my site redirects are experiencing the same issue. I'm hoping this is simply some poor configuration settings I've made over the years,

Taomyn · 2020-09-08T07:34:54Z

I just noticed while trying to narrow down a few parameters that if the HAProxy service is restarted after a change and I quickly attempt a connection, it will give me a partial page mostly badly formatted, but then a refresh and I'm back with the refusal error. The HAProxy log does show some extra connections:

2020-09-08T09:27:15	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:15.504] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /index.htm HTTP/2.0"
2020-09-08T09:27:15	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:15.489] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /index.htm HTTP/2.0"
2020-09-08T09:27:15	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:15.476] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /index.htm HTTP/2.0"
2020-09-08T09:27:15	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:15.350] Web_Server_SSL~ PRTG/PRTG 0/0/1/-1/1 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /index.htm HTTP/2.0"
2020-09-08T09:27:15	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:15.339] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /index.htm HTTP/2.0"
2020-09-08T09:27:15	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:15.328] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /index.htm HTTP/2.0"
2020-09-08T09:27:13	haproxy[64809]	193.91.41.74:53509 [08/Sep/2020:09:27:13.300] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /public/manifest.json.htm HTTP/2.0"
2020-09-08T09:27:13	haproxy[64809]	193.91.41.74:53509 [08/Sep/2020:09:27:13.290] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/0 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /public/manifest.json.htm HTTP/2.0"
2020-09-08T09:27:13	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:13.277] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /favicon.ico HTTP/2.0"
2020-09-08T09:27:13	haproxy[64809]	193.91.41.74:53509 [08/Sep/2020:09:27:13.278] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/0 -1 0 - - SD-- 7/7/1/1/0 0/0 "GET /public/manifest.json.htm HTTP/2.0"
2020-09-08T09:27:13	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:13.256] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/8 -1 0 - - SD-- 7/7/0/0/0 0/0 "GET /favicon.ico HTTP/2.0"
2020-09-08T09:27:13	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:13.242] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 6/6/0/0/0 0/0 "GET /favicon.ico HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.822] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 6/6/0/0/0 0/0 "GET /images/paessler.png HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.808] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 6/6/0/0/0 0/0 "GET /images/paessler.png HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.796] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 6/6/0/0/0 0/0 "GET /images/paessler.png HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.770] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/0 -1 0 - - SD-- 6/6/0/0/0 0/0 "GET /images/prtg_logo_gray.png HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.769] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 6/6/1/1/0 0/0 "GET /css/prtgmini.css?prtgversion=__ HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.754] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 6/6/0/0/0 0/0 "GET /images/prtg_logo_gray.png HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.754] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 6/6/1/1/0 0/0 "GET /css/prtgmini.css?prtgversion=__ HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.743] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 6/6/0/0/0 0/0 "GET /images/prtg_logo_gray.png HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.742] Web_Server_SSL~ PRTG/PRTG 0/0/0/-1/1 -1 0 - - SD-- 6/6/1/1/0 0/0 "GET /css/prtgmini.css?prtgversion=__ HTTP/2.0"
2020-09-08T09:27:12	haproxy[64809]	193.91.41.74:53506 [08/Sep/2020:09:27:12.617] Web_Server_SSL~ PRTG/PRTG 0/0/16/37/53 200 20000 - - ---- 6/6/0/0/0 0/0 "GET /index.htm HTTP/2.0"
2020-09-08T09:27:01	haproxy[56921]	Proxy PRTG started.

marinbernard-pep06 · 2020-09-09T06:57:10Z

You may already know it, but we discovered that the problem disappears after switching the box to the OpenSSL flavour. It might be LibreSSL-related.

Taomyn · 2020-09-09T07:01:04Z

You may already know it, but we discovered that the problem disappears after switching the box to the OpenSSL flavour. It might be LibreSSL-related.

No, I had no idea but wow, it's almost like LibreSSL is trying to force me to switch to OpenSSL - I was considering it a week or so ago in order to get support for server TLS1.3

I'll see when I get home if I have time to try this out, but any idea if it's something that could get fixed for LibreSSL soon?

actual-nsnow · 2020-09-14T17:56:59Z

I'm having the same issue, I just finished switching over to openssl and I'm going to test. In my case, most of my services were working, it was opnsense I couldn't reach. My default webgui port is 8443 and I use haproxy to serve opnsense based on hostname rules. I was getting ERR_HTTP2_SERVER_REFUSED_STREAM whenever trying to access opnsense over the proxy.

Update/reboot is finished, everything seems to be working as expected so far.

Taomyn · 2020-09-15T08:11:41Z

Thanks @actual-nsnow - I still have not had the time to switch my installation back to OpenSSL, hopefully this week, but I'm now wondering if it's also the cause of a mysterious issue I also have with the Let's Encrypt plug-in that also started when I upgraded to 20.7.x

Taomyn · 2020-09-24T18:59:52Z

I finally got around to switching from LibreSSL to OpenSSL then rebooted for good measure, and can report that from internal all the sites that were failing to work are now all fine again.

I did first try upgrading to 20.7.3 but that itself made no difference.

browne-net · 2020-09-29T10:47:20Z

I am having the same issue.

System: OPNsense 20.7-amd64FreeBSD 12.1-RELEASE-p7-HBSDLibreSSL 3.0.2
HAProxy: 2.23

Once I upgrade to 20.7.3 my HAProxy redirects stop working.
The browsers give an error about some SSL issues.

EDIT:

@Taomyn

I just noticed while trying to narrow down a few parameters that if the HAProxy service is restarted after a change and I quickly attempt a connection, it will give me a partial page mostly badly formatted, but then a refresh and I'm back with the refusal error.

I also noticed the same behaviour. After restarting HAProxy I can connect to my services for about 5 seconds, but then the connections are refused again.

fraenki · 2020-09-29T10:51:35Z

So I recommend switching to the OpenSSL flavour if you hit this issue. There's nothing else I can do about it.

browne-net · 2020-09-29T11:07:08Z

I will just stick with 20.7 until the issue is resolved.

fraenki · 2020-09-30T15:20:36Z

HAProxy 2.0.18 was released today and contains minor SSL fixes, not sure if this will improve the situation with LibreSSL:
https://www.mail-archive.com/haproxy@formilux.org/msg38547.html
However, I expect this version to be included in the upcoming OPNsense 20.7.4 (no ETA!), so feel free to report back when this version is available.

browne-net · 2020-10-27T10:21:50Z

https://opnsense.org/opnsense-20-7-4-released/

Did anyone test if OPNsense 20.7.4 with the latest HAProxy solves the issue?
I am right now unable to update my OPNsense since it is running in a production environment 24/7.

browne-net · 2020-10-27T22:24:46Z

I managed to take that machine off for a few hours.
Updated from 20.7 to 20.7.4 and of course HAProxy stopped working. But this time the plugin didn't even want to start. Switching from LibreSSL to OpenSSL solved the issue.

Here are the errors from a syntax test.

[ALERT] 300/230804 (7671) : parsing [/usr/local/etc/haproxy.conf:24] : unknown keyword 'ssl-default-bind-ciphersuites' in 'global' section
[WARNING] 300/230804 (7671) : parsing [/usr/local/etc/haproxy.conf:64] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[ALERT] 300/230804 (7671) : parsing [/usr/local/etc/haproxy.conf:69] : 'bind 0.0.0.0:443' unknown keyword 'ciphersuites'. Registered keywords :
[ SSL] allow-0rtt
[ SSL] alpn <arg>
[ SSL] ca-file <arg>
[ SSL] ca-ignore-err <arg>
[ SSL] ca-sign-file <arg>
[ SSL] ca-sign-pass <arg>
[ SSL] ciphers <arg>
[ SSL] crl-file <arg>
[ SSL] crt <arg>
[ SSL] crt-ignore-err <arg>
[ SSL] crt-list <arg>
[ SSL] curves <arg>
[ SSL] ecdhe <arg>
[ SSL] force-sslv3
[ SSL] force-tlsv10
[ SSL] force-tlsv11
[ SSL] force-tlsv12
[ SSL] force-tlsv13
[ SSL] generate-certificates
[ SSL] no-ca-names
[ SSL] no-sslv3
[ SSL] no-tlsv10
[ SSL] no-tlsv11
[ SSL] no-tlsv12
[ SSL] no-tlsv13
[ SSL] no-tls-tickets
[ SSL] ssl
[ SSL] ssl-min-ver <arg>
[ SSL] ssl-max-ver <arg>
[ SSL] strict-sni
[ SSL] tls-ticket-keys <arg>
[ SSL] verify <arg>
[ SSL] npn <arg>
[ SSL] prefer-client-ciphers
[STAT] level <arg>
[STAT] expose-fd <arg>
[STAT] severity-output <arg>
[ TCP] mss <arg>
[ TCP] tfo
[ TCP] transparent
[ TCP] v4v6
[ TCP] v6only
[ TCP] defer-accept (not supported)
[ TCP] interface <arg> (not supported)
[ ALL] accept-netscaler-cip <arg>
[ ALL] accept-proxy
[ ALL] backlog <arg>
[ ALL] id <arg>
[ ALL] maxconn <arg>
[ ALL] name <arg>
[ ALL] nice <arg>
[ ALL] process <arg>
[ ALL] proto <arg>
[UNIX] gid <arg>
[UNIX] group <arg>
[UNIX] mode <arg>
[UNIX] uid <arg>
[UNIX] user <arg>
[WARNING] 300/230804 (7671) : parsing [/usr/local/etc/haproxy.conf:109] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[ALERT] 300/230804 (7671) : parsing [/usr/local/etc/haproxy.conf:118] : 'bind 0.0.0.0:5443' unknown keyword 'ciphersuites'.
[ALERT] 300/230804 (7671) : Error(s) found in configuration file : /usr/local/etc/haproxy.conf
[ALERT] 300/230804 (7671) : Fatal errors found in configuration.

I think it is clearly due to the fact that LibreSSL variant of OPNsense still doesn't support TLS v1.3 but the latest version of HAProxy does and makes use of it or at least tries with LibreSSL.

I also found this forum post with an approximate ETA for the LibreSSL variant of OPNsense with support for TLS v1.3.
https://forum.opnsense.org/index.php?topic=18516.msg86648#msg86648

fichtner · 2020-10-29T08:16:58Z

LibreSSL 3.2 has TLSv1.3 support but not the OpenSSL 1.1.1 API parts which might make upstream assume it's not working/refrain from implementing it (properly). So far LibreSSL 3.2 has some minor quirks to sort out in a subsequent stable release for them as well before we can move OPNsense to this version (likely in 1.5 months in preparation for 21.1). So currently we have 3 moving targets/parties involved and this will not resolve instantly for this reason. :)

imidoriya · 2021-01-08T20:05:41Z

Rebuilding a opnsense box on the latest version and imported my haproxy config. It fails to start. I'm also getting the issue that @browne-net reported.
[ALERT] 007/195840 (23438) : parsing [/usr/local/etc/haproxy.conf:47] : 'bind X.X.X.X:443' unknown keyword 'ciphersuites'. Registered keywords :...
It seems the ciphersuites string is failing to be set. I went into haproxy global parameters and tried to resave the Cipher Suites, but if fails.

Changing from LibreSSL to OpenSSL appears to fix it.

imidoriya · 2021-01-08T20:10:16Z

Some additional logging that was a warning on the dashboard.

[08-Jan-2021 20:04:00 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library 'openssl.so' (tried: /usr/local/lib/php/20180731/openssl.so (Shared object "libssl.so.48" not found, required by "openssl.so"), /usr/local/lib/php/20180731/openssl.so.so (Cannot open "/usr/local/lib/php/20180731/openssl.so.so")) in Unknown on line 0
[08-Jan-2021 20:04:00 Etc/UTC] PHP Warning:  Use of undefined constant OCSP_REVOKED_STATUS_NOSTATUS - assumed 'OCSP_REVOKED_STATUS_NOSTATUS' (this will throw an Error in a future version of PHP) in /usr/local/etc/inc/certs.inc on line 33
[08-Jan-2021 20:04:00 Etc/UTC] PHP Warning:  Use of undefined constant OCSP_REVOKED_STATUS_UNSPECIFIED - assumed 'OCSP_REVOKED_STATUS_UNSPECIFIED' (this will throw an Error in a future version of PHP) in /usr/local/etc/inc/certs.inc on line 34
[08-Jan-2021 20:04:00 Etc/UTC] PHP Warning:  Use of undefined constant OCSP_REVOKED_STATUS_KEYCOMPROMISE - assumed 'OCSP_REVOKED_STATUS_KEYCOMPROMISE' (this will throw an Error in a future version of PHP) in /usr/local/etc/inc/certs.inc on line 35
[08-Jan-2021 20:04:00 Etc/UTC] PHP Warning:  Use of undefined constant OCSP_REVOKED_STATUS_CACOMPROMISE - assumed 'OCSP_REVOKED_STATUS_CACOMPROMISE' (this will throw an Error in a future version of PHP) in /usr/local/etc/inc/certs.inc on line 36
[08-Jan-2021 20:04:00 Etc/UTC] PHP Warning:  Use of undefined constant OCSP_REVOKED_STATUS_AFFILIATIONCHANGED - assumed 'OCSP_REVOKED_STATUS_AFFILIATIONCHANGED' (this will throw an Error in a future version of PHP) in /usr/local/etc/inc/certs.inc on line 37
[08-Jan-2021 20:04:00 Etc/UTC] PHP Warning:  Use of undefined constant OCSP_REVOKED_STATUS_SUPERSEDED - assumed 'OCSP_REVOKED_STATUS_SUPERSEDED' (this will throw an Error in a future version of PHP) in /usr/local/etc/inc/certs.inc on line 38
[08-Jan-2021 20:04:00 Etc/UTC] PHP Warning:  Use of undefined constant OCSP_REVOKED_STATUS_CESSATIONOFOPERATION - assumed 'OCSP_REVOKED_STATUS_CESSATIONOFOPERATION' (this will throw an Error in a future version of PHP) in /usr/local/etc/inc/certs.inc on line 39
[08-Jan-2021 20:04:00 Etc/UTC] PHP Warning:  Use of undefined constant OCSP_REVOKED_STATUS_CERTIFICATEHOLD - assumed 'OCSP_REVOKED_STATUS_CERTIFICATEHOLD' (this will throw an Error in a future version of PHP) in /usr/local/etc/inc/certs.inc on line 40

User-Agent Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
FreeBSD 12.1-RELEASE-p11-HBSD #0  74f1f081a1e(stable/20.7)-dirty: Fri Dec  4 13:40:15 CET 2020     root@sensey64:/usr/obj/usr/src/amd64.amd64/sys/SMP amd64
OPNsense 20.7.7_1 729265228
Plugins os-dyndns-1.23 os-haproxy-2.25 os-openconnect-1.4.0 os-vmware-1.5 
Time Fri, 08 Jan 2021 20:06:39 +0000
OpenSSL 1.1.1i  8 Dec 2020
PHP 7.3.25

browne-net · 2021-01-19T14:10:34Z

Did someone test if the bug is resolved on OPNsense 21.1-RC1 which just got released?
https://forum.opnsense.org/index.php?topic=20893.0

I am currently unable to take my machine offline for testing.

cluck · 2021-02-03T15:20:10Z

Landed here while upgrading to 21.1, so the problem still exists.

fichtner · 2021-02-04T07:55:38Z

There was a larger drop of compatibility fixes in OpenBSD 6.8 but no LibreSSL 3.2.4 in sight... https://ftp.openbsd.org/pub/OpenBSD/patches/6.8/common/013_libressl.patch.sig

OPNsense-bot · 2021-03-23T14:23:02Z

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository,
please read https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue,
just let us know, so we can reopen the issue and assign an owner to it.

imidoriya · 2021-03-23T17:30:33Z

The last update had some haproxy changes, but I haven't tested if LibreSSL works or not. Still and issue? We should keep this open if it is.

fichtner · 2021-03-23T18:45:18Z

We have no resources to track cross-upstream issues between LibreSSL and HAProxy. So either we close or we keep it open forever. The result is the same so we might as well close it.

imidoriya · 2021-03-23T20:29:28Z

Understandable - but if anyone does test this and verify it works, please post. Would be great to be notified so I can return to LibreSSL. :)

browne-net · 2021-03-23T21:44:30Z

I just tried it and LibreSSL is still not working with HAProxy.

[NOTICE] 081/222056 (93896) : haproxy version is 2.2.10-6a09215
[ALERT] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:23] : unknown keyword 'ssl-default-bind-ciphersuites' in 'global' section
[WARNING] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:64] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[ALERT] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:69] : 'bind local_ip_1:443' unknown keyword 'ciphersuites'. Registered keywords :
[ SSL] allow-0rtt
[ SSL] alpn <arg>
[ SSL] ca-file <arg>
[ SSL] ca-verify-file <arg>
[ SSL] ca-ignore-err <arg>
[ SSL] ca-sign-file <arg>
[ SSL] ca-sign-pass <arg>
[ SSL] ciphers <arg>
[ SSL] crl-file <arg>
[ SSL] crt <arg>
[ SSL] crt-ignore-err <arg>
[ SSL] crt-list <arg>
[ SSL] curves <arg>
[ SSL] ecdhe <arg>
[ SSL] force-sslv3
[ SSL] force-tlsv10
[ SSL] force-tlsv11
[ SSL] force-tlsv12
[ SSL] force-tlsv13
[ SSL] generate-certificates
[ SSL] no-ca-names
[ SSL] no-sslv3
[ SSL] no-tlsv10
[ SSL] no-tlsv11
[ SSL] no-tlsv12
[ SSL] no-tlsv13
[ SSL] no-tls-tickets
[ SSL] ssl
[ SSL] ssl-min-ver <arg>
[ SSL] ssl-max-ver <arg>
[ SSL] strict-sni
[ SSL] tls-ticket-keys <arg>
[ SSL] verify <arg>
[ SSL] npn <arg>
[ SSL] prefer-client-ciphers
[STAT] level <arg>
[STAT] expose-fd <arg>
[STAT] severity-output <arg>
[ ALL] accept-netscaler-cip <arg>
[ ALL] accept-proxy
[ ALL] backlog <arg>
[ ALL] id <arg>
[ ALL] maxconn <arg>
[ ALL] name <arg>
[ ALL] nice <arg>
[ ALL] process <arg>
[ ALL] proto <arg>
[UNIX] gid <arg>
[UNIX] group <arg>
[UNIX] mode <arg>
[UNIX] uid <arg>
[UNIX] user <arg>
[ TCP] mss <arg>
[ TCP] tfo
[ TCP] transparent
[ TCP] v4v6
[ TCP] v6only
[ TCP] defer-accept (not supported)
[ TCP] interface <arg> (not supported)
[WARNING] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:101] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[WARNING] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:107] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[WARNING] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:131] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[ALERT] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:136] : 'bind local_ip_2:443' unknown keyword 'ciphersuites'.
[WARNING] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:168] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[WARNING] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:172] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[WARNING] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:176] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[ALERT] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:183] : 'bind local_ip_2:8888' unknown keyword 'ciphersuites'.
[ALERT] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:208] : 'bind local_ip_2:8040' unknown keyword 'ciphersuites'.
[ALERT] 081/222056 (93896) : Error(s) found in configuration file : /usr/local/etc/haproxy.conf.staging
[ALERT] 081/222056 (93896) : Fatal errors found in configuration.

…pnsense#2013

fraenki · 2021-03-28T22:22:13Z

[ALERT] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:23] : unknown keyword 'ssl-default-bind-ciphersuites' in 'global' section
[ALERT] 081/222056 (93896) : parsing [/usr/local/etc/haproxy.conf.staging:69] : 'bind local_ip_1:443' unknown keyword 'ciphersuites'. Registered keywords :

I've added a workaround, so the incompatible "ciphersuites" options will be ignored when LibreSSL is used. It will not solve other compatibility issues with LibreSSL, but HAProxy should startup again:

opnsense-patch -c plugins 095740ab393f6e572c359fbedef17d76c3c82cd7

The upcoming os-haproxy 3.2 will include this workaround, but don't be too optimistic for fixes beyond this, it's an upstream issue as already stated. You'll most likely have to make other configuration changes like replacing incompatible ciphers, disabling HTTP/2 and/or TLSv1.3 when using LibreSSL.

Taomyn changed the title ~~HAProxy - one site redirect fails with ERR_HTTP2_SERVER_REFUSED_STREAM~~ HAProxy - site redirects fails with ERR_HTTP2_SERVER_REFUSED_STREAM Sep 8, 2020

fraenki added the support Community support label Sep 10, 2020

fraenki added the help wanted Contributor missing label Sep 29, 2020

fichtner removed the help wanted Contributor missing label Oct 4, 2020

fraenki changed the title ~~HAProxy - site redirects fails with ERR_HTTP2_SERVER_REFUSED_STREAM~~ net/haproxy: site redirects fails with ERR_HTTP2_SERVER_REFUSED_STREAM with LibreSSL Oct 27, 2020

fraenki added upstream Third party issue and removed support Community support labels Oct 29, 2020

OPNsense-bot closed this as completed Mar 23, 2021

OPNsense-bot added the help wanted Contributor missing label Mar 23, 2021

fraenki added a commit to fraenki/plugins that referenced this issue Mar 28, 2021

net/haproxy: ignore incompatible options when LibreSSL is used, refs o…

095740a

…pnsense#2013

net/haproxy: site redirects fails with ERR_HTTP2_SERVER_REFUSED_STREAM with LibreSSL #2013

net/haproxy: site redirects fails with ERR_HTTP2_SERVER_REFUSED_STREAM with LibreSSL #2013

Comments

Taomyn commented Sep 3, 2020 • edited Loading

Taomyn commented Sep 4, 2020 • edited Loading

mimugmail commented Sep 4, 2020

Taomyn commented Sep 4, 2020

mimugmail commented Sep 4, 2020

fichtner commented Sep 4, 2020

sorano commented Sep 4, 2020

Taomyn commented Sep 4, 2020

sorano commented Sep 4, 2020

Taomyn commented Sep 4, 2020

sorano commented Sep 4, 2020 • edited Loading

Taomyn commented Sep 4, 2020

Taomyn commented Sep 4, 2020

sorano commented Sep 4, 2020

Taomyn commented Sep 4, 2020 • edited Loading

Taomyn commented Sep 4, 2020

fraenki commented Sep 4, 2020 • edited Loading

Taomyn commented Sep 4, 2020

Taomyn commented Sep 7, 2020

Taomyn commented Sep 8, 2020

marinbernard-pep06 commented Sep 9, 2020 • edited Loading

Taomyn commented Sep 9, 2020

actual-nsnow commented Sep 14, 2020

Taomyn commented Sep 15, 2020

Taomyn commented Sep 24, 2020

browne-net commented Sep 29, 2020 • edited Loading

fraenki commented Sep 29, 2020

browne-net commented Sep 29, 2020

fraenki commented Sep 30, 2020

browne-net commented Oct 27, 2020 • edited Loading

browne-net commented Oct 27, 2020 • edited Loading

fichtner commented Oct 29, 2020

imidoriya commented Jan 8, 2021

imidoriya commented Jan 8, 2021

browne-net commented Jan 19, 2021

cluck commented Feb 3, 2021

fichtner commented Feb 4, 2021

OPNsense-bot commented Mar 23, 2021

imidoriya commented Mar 23, 2021

fichtner commented Mar 23, 2021

imidoriya commented Mar 23, 2021

browne-net commented Mar 23, 2021 • edited Loading

fraenki commented Mar 28, 2021 • edited Loading

Taomyn commented Sep 3, 2020 •

edited

Loading

Taomyn commented Sep 4, 2020 •

edited

Loading

sorano commented Sep 4, 2020 •

edited

Loading

Taomyn commented Sep 4, 2020 •

edited

Loading

fraenki commented Sep 4, 2020 •

edited

Loading

marinbernard-pep06 commented Sep 9, 2020 •

edited

Loading

browne-net commented Sep 29, 2020 •

edited

Loading

browne-net commented Oct 27, 2020 •

edited

Loading

browne-net commented Oct 27, 2020 •

edited

Loading

browne-net commented Mar 23, 2021 •

edited

Loading

fraenki commented Mar 28, 2021 •

edited

Loading