Getting HTTP 502 Bad Gateway in admin Dashboard when DEBUG is ON

[mikeazores]

Steps to reproduce the issue

After update to Joomla 4.1, having DEBUG ON, when going to the Administration Home Dashboard I get a HTTP 502 Bad Gateway.

Expected result

Display the admin control panel.

Actual result

Got a HTTP 502 Bad Gateway message.

System information (as much as possible)

PHP Built On Linux ae3dedf9ab04 5.4.39-linuxkit #1 SMP Fri May 8 23:03:06 UTC 2020 x86_64
Database Type mysql
Database Version 10.5.6-MariaDB
Database Collation utf8_general_ci
Database Connection Collation utf8mb4_general_ci
Database Connection Encryption None
Database Server Supports Connection Encryption No
PHP Version 8.0.14
Web Server nginx/1.14.2
WebServer to PHP Interface fpm-fcgi
Joomla! Version Joomla! 4.1.0 Stable [ Kuamini ] 15-February-2022 09:30 GMT
User Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.80 Safari/537.36 Edg/98.0.1108.50

=============
PHP Settings

memory_limit: 384M
upload_max_filesize: 100M
post_max_size: 100M
display_errors: true
short_open_tag: true
file_uploads: true
output_buffering: false
open_basedir:
session.save_path:
session.auto_start: 0
disable_functions:
xml: true
zlib: true
zip: true
mbstring: true
gd: true
iconv: true
intl: true
max_input_vars: 1000

=============
Configuration File

offline: false
offline_message: Este sitio está cerrado por tareas de mantenimiento.
Por favor, inténtelo nuevamente más tarde.
display_offline_message: 1
offline_image:
sitename: xxxxxx
editor: tinymce
captcha: 0
list_limit: 20
access: 1
debug: false
debug_lang: false
debug_lang_const: true
dbtype: mysqli
host: xxxxxx
user: xxxxxx
password: xxxxxx
db: xxxxxx
dbprefix: xxxxxx
dbencryption: 0
dbsslverifyservercert: false
dbsslkey:
dbsslcert:
dbsslca:
dbsslcipher:
force_ssl: 0
live_site:
secret: xxxxxx
gzip: false
error_reporting: maximum
helpurl: https://help.joomla.org/proxy?keyref=Help{major}{minor}:{keyref}&lang={langcode}
offset: Europe/Madrid
mailonline: true
mailer: mail
caching: 0
cache_handler: file
cachetime: 15
cache_platformprefix: false
session_handler: database
shared_session: false
session_metadata: true
memcached_persist: true
memcached_compress: false
memcached_server_host: xxxxxx
memcached_server_port: 11211
redis_persist: true
redis_server_host: xxxxxx
redis_server_port: 6379
redis_server_db: 0
cors: false
cors_allow_origin: *
cors_allow_headers: Content-Type,X-Joomla-Token
cors_allow_methods:
behind_loadbalancer: false
proxy_enable: false
proxy_host:
proxy_port:
proxy_user: xxxxxx
massmailoff: false
sitename_pagetitles: 0
session_filesystem_path:
session_memcached_server_host: xxxxxx
session_memcached_server_port: 11211
session_redis_persist: 1
session_redis_server_host: xxxxxx
session_redis_server_port: 6379
session_redis_server_db: 0
frontediting: 1
block_floc: 0
log_everything: 0
log_deprecated: 0

=============
Folder Permissions

administrator/components: writable
administrator/components/com_joomlaupdate: writable
administrator/language: writable
administrator/language/en-GB: writable
administrator/language/es-ES: writable
administrator/language/overrides: writable
administrator/manifests/files: writable
administrator/manifests/libraries: writable
administrator/manifests/packages: writable
administrator/modules: writable
administrator/templates: writable
components: writable
images: writable
images/banners: writable
images/headers: writable
images/media: writable
images/sampledata: writable
language: writable
language/en-GB: writable
language/es-ES: writable
language/overrides: writable
libraries: writable
media: writable
modules: writable
plugins: writable
plugins/actionlog: writable
plugins/api-authentication: writable
plugins/authentication: writable
plugins/behaviour: writable
plugins/captcha: writable
plugins/content: writable
plugins/editors: writable
plugins/editors-xtd: writable
plugins/extension: writable
plugins/fields: writable
plugins/filesystem: writable
plugins/finder: writable
plugins/installer: writable
plugins/media-action: writable
plugins/privacy: writable
plugins/quickicon: writable
plugins/sampledata: writable
plugins/search: writable
plugins/system: writable
plugins/task: writable
plugins/twofactorauth: writable
plugins/user: writable
plugins/webservices: writable
plugins/workflow: writable
templates: writable
configuration.php: writable
administrator/cache: writable
media/cache: writable
log: writable
tmp: writable

Additional comments

This is happening just when going to administrator Home Dashboard. If I go to any other admin menu option, it is working just fine. Also in frontend it seems to be working fine.

[Fedik]

I suspect it caused by one of installed extensions, because on my installation I do not have this problem.

As idea try to disable all non Joomla QuickIcons plugins.

And if you need more help try ask on Joomla forum.

[mikeazores]

@Fedik , thanks for your feedback.

I uninstalled ALL the extensions and still have the problem. Additionally, with DEBUG OFF everything is working as expected.

Do you have the same environment as mine?

Database Version 10.5.6-MariaDB
Database Collation utf8_general_ci
Database Connection Collation utf8mb4_general_ci
Database Connection Encryption None
Database Server Supports Connection Encryption No
PHP Version 8.0.14
Web Server nginx/1.14.2
WebServer to PHP Interface fpm-fcgi

Regards.

[Fedik]

Sorry, I did not noticed you use MariaDB, I have MySQL

Can you please try next:
In System debug plugin set Queries to Off, or disable System debug plugin totally.

Additionally, check php server log, if there any critical error logged.

[mikeazores]

@Fedik Thanks.

I follow your suggestions and the error is happening when Profiling is ON in Debug Plugin.

The Nginx error log is:

A BIG CHUNK OF BINARY DATA AND THEN:
022/02/17 18:38:06 [error] 6#6: *1 upstream sent too big header while reading response header from upstream, client: 172.21.0.1, server: localhost, request: "GET /administrator/index.php HTTP/2.0", upstream: "fastcgi://172.21.0.4:9000", host: "localhost", referrer: "https://localhost/administrator/index.php?option=com_config" 2022/02/17 18:40:25 [error] 6#6: *1 upstream sent too big header while reading response header from upstream, client: 172.21.0.1, server: localhost, request: "GET /administrator/ HTTP/2.0", upstream: "fastcgi://172.21.0.4:9000", host: "localhost" 2022/02/17 18:41:41 [error] 6#6: *1 upstream sent too big header while reading response header from upstream, client: 172.21.0.1, server: localhost, request: "GET /administrator/ HTTP/2.0", upstream: "fastcgi://172.21.0.4:9000", host: "localhost" 2022/02/17 18:41:53 [error] 6#6: *1 upstream sent too big header while reading response header from upstream, client: 172.21.0.1, server: localhost, request: "GET /administrator/ HTTP/2.0", upstream: "fastcgi://172.21.0.4:9000", host: "localhost" 2022/02/17 18:42:04 [error] 6#6: *1 upstream sent too big header while reading response header from upstream, client: 172.21.0.1, server: localhost, request: "GET /administrator/ HTTP/2.0", upstream: "fastcgi://172.21.0.4:9000", host: "localhost" 2022/02/17 18:42:55 [error] 6#6: *1 upstream sent too big header while reading response header from upstream, client: 172.21.0.1, server: localhost, request: "GET /administrator/ HTTP/2.0", upstream: "fastcgi://172.21.0.4:9000", host: "localhost"

I don't know if this something to do with XDEBUG config. My config is:

xdebug.mode=debug
xdebug.client_host=host.docker.internal
xdebug.client_port=9000
xdebug.remote_autostart=off
xdebug.remote_handler = "dbgp"
xdebug.idekey = PHPSTORM`

[Fedik]

You have a loot of modules on Dahsboard? :)

So, this actually the server error, due to too big header:

... upstream sent too big header while reading response header from upstream

The header size increase caused by recent changes #36231
@laoneo is there a way to reduce it?

This error may happen depend from server: ngnix 4K limit, apache 8K.

[mikeazores]

You have a loot of modules on Dahsboard? :)

Not really :) Just 1 extension installed (acymailing).

In Joomla 4.0.x I didn't have this problem. As soon as I updated to Joomla 4.1 this issue started

[laoneo]

@mikeazores can you remove the line 701 in the file /plugins/system/debug/debug.php? Does the error then go away? Just to be sure, it is the following code you have to remove:

$this->app->setHeader('Server-Timing', $metrics);

[alikon]

I can confirm on nginx/1.20.2 4.0.6, all works after update to 4.1.0 got HTTP 502 Bad Gateway when debug plugin is on
removing line 701 solve the issue , i dunno if there is a nginx setting that can help

[mikeazores]

@laoneo After removing that line I can confirm it is working just fine as @alikon pointed out.

[laoneo]

Can you guys test #37087. Perhaps we have to decrease the limit below 4000.

[mikeazores]

Can you guys test #37087. Perhaps we have to decrease the limit below 4000.

@laoneo I tested PR #37087 and is NOT working. However, if I lower the strlen value to say 3600 IT WORKS.

[laoneo]

@mikeazores please make further comments in the PR. Closing this issue.