[logs] content rendered illegible #3430

ghost · 2019-12-23T13:15:15Z

Master 4.19.88 armv7l (mvebu)
libuci/base/2019-12-12
libuci-lua/base/2019-12-12
luci-lib-jsonc/luci/git-19.356.80423
luci-base/luci/git-19.356.80423
luci/luci/git-19.356.80423
luci-i18n-base-en/luci/git-19.356.80423
luci-compat/luci/git-19.356.80423
syslog-ng 3.25.1

tried different themes | browsers but all met with same illegible render however (log files are legible via ssh)

dibdot · 2019-12-23T14:14:40Z

Is this a turris omnia device? If so, not reproducible with latest stable (TOS 4.03) ...

ghost · 2019-12-23T14:20:15Z

I am aware that it does not reproduce with OpenWrt 18.06.x (TOS4.x) or 19.07 (TOS5.x) and until 2 days ago it did not exhibit the issue on Master (TOS6.x) either. But latest changes in the aforementioned LuCI packages seems to have introduced the bug.

dibdot · 2019-12-23T14:24:59Z

Again, which device and what OS are you using? Plain vanilla OpenWrt or a TOS variant?

ghost · 2019-12-23T14:27:24Z

TOS6.x on Turris Omnia (hardware revision CZ11NIC20), however the issue is not caused by the downstream distro since LuCI packages are not being patched in such way to cause this.

jow- · 2019-12-23T16:35:25Z

Not reproducible on OpenWrt master. Maybe something is wrong with cgi-io on "TOS" ?

ghost · 2019-12-23T17:15:33Z

It might be something related to file permissions, considering this been caught by the browser tool box

closing since it does not reproduce on OpenWrt, taking it up with the downstream distro instead

jow- · 2019-12-23T17:23:53Z

@n8v8R - do you maybe use luci-nginx by any chance? It probably needs to be updated to handle the new /cgi-bin/cgi-exec applet as CGI executable. Right now it seems as if it gets downloaded instead.

ghost · 2019-12-23T17:37:34Z

@jow- No, neither that LuCI applet nor nginx at all.

ghost · 2019-12-24T10:44:47Z

@jow- Narrowed it down to luci-mod-status, after upgrading it from git-19.352.74009 to git-19.356.80423 the issue starts to exhibit.

Downstream is not patching this package in their build root and thus this seems to be an issue originating from the OpenWrt code source instead.

During the upgrade it exhibits:

Removing obsolete file /usr/lib/lua/luci/controller/admin/status.lua.
Removing obsolete file /usr/lib/lua/luci/view/admin_status/dmesg.htm.
Removing obsolete file /usr/lib/lua/luci/view/admin_status/iptables.htm.
Removing obsolete file /usr/lib/lua/luci/view/admin_status/syslog.htm.

That correlates with the log rendering issues reported and the observation that the Firewall status is neither working but instead showing a spinner alongside Collecting data...

✅ /cgi-bin/luci/admin/status/overview
✅ /cgi-bin/luci/admin/status/routes
✅ /cgi-bin/luci/admin/status/realtime
✅ /cgi-bin/luci/admin/status/processes
❌ /cgi-bin/luci/admin/status/syslog
❌ /cgi-bin/luci/admin/status/dmesg
❌ /cgi-bin/luci/admin/status/iptables

Probably caused by commits 939b371 and b9bb346

jow- · 2019-12-24T16:43:58Z

You‘re using LuCI on lighttpd and from the symptoms you describe, it‘s CGI setup is broken. It seems as if /cgi-bin/cgi-exec is not executed, but served as file by your webserver. Both the use of lighttpd and the broken CGI config are downstream deviations from OpenWrt master defaults.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>

ghost · 2020-01-08T15:46:38Z

It seems that commit fails to implement the appropriate response header as per https://tools.ietf.org/html/rfc3875#section-6

jow- · 2020-01-09T10:08:15Z

The Content-Type response header is completely unrelated to your issue and sending a Content-Type: application/octet-stream is perfectly valid. Note in your screenshots that the binary garbage you receive starts with ?ELF... which means that lighttpd failed to execute the requested executable but instead opted to send its file contents instead.

This is a bug in lighttpd's configuration. LuCI assumes that executables in /www/cgi-bin/ are executable as CGI programs via http://hostname/cgi-bin/....

ghost · 2020-01-09T10:32:55Z

The advise on the lighttpd forum been different ^[1]

Your CGI (/cgi-bin/cgi-exec) must respond with appropriate response headers.
See https://tools.ietf.org/html/rfc3875 The Common Gateway Interface (CGI) Version 1.1

This is a bug in lighttpd's configuration. LuCI assumes that executables in /www/cgi-bin/ are executable as CGI programs via http://hostname/cgi-bin/....

I do not think so, the config for LuCI in lighttpd reads (and works perfectly fine otherwise)

# Set CGI paths
cgi.assign += (
        "/cgi-bin/luci" => ""
)

## Set aliases to LuCI install directory
alias.url += (
        "/cgi-bin/" => "/www/cgi-bin/",
        "/luci-static/" => "/www/luci-static/"
)

sending a Content-Type: application/octet-stream is perfectly valid

Responding with such header is valid of course but the MIME type octet-stream is not valid for /cgi-bin/cgi-exec

As stipulated by rfc3875 ^[2]

**The Content-Type response field sets the Internet Media Type [6] of
the entity body.

  Content-Type = "Content-Type:" media-type NL

If an entity body is returned, the script MUST supply a Content-Type
field in the response. If it fails to do so, the server SHOULD NOT
attempt to determine the correct content type. The value SHOULD be
sent unmodified to the client, except for any charset parameter
changes.**

^[1] https://redmine.lighttpd.net/boards/2/topics/8893
^[2] https://tools.ietf.org/html/rfc3875#section-6.3.1

jow- · 2020-01-09T10:35:06Z

I am not sure what you're getting at. The script sends a Content-Type header. And the mime-type is application/octet-stream, not octet-stream.

Also your linked bug report does not make sense at all. It appears to me that you're confusing the request content-type and the response content-type. The request is sent encoded as application/x-www-form-urlencoded and the response is sent back as application/octet-stream. This is perfectly legal and does not involve the webserver at all. Neither is lighttpd supposed to parse the request body, which is the CGI program's task, nor is it responsible for handling the CGI response body, which is the requesting clients task.

In the case of simple CGI operation, the webserver is supposed to act as application proxy (or gateway), nothing more.

ghost · 2020-01-09T11:07:11Z

Let us recap for a moment:

f3724e4 introduced in Master

Request.post('/cgi-bin/cgi-exec', postdata, {
			headers: { 'Content-Type': 'application/x-www-form-urlencoded' }
		}

since then the issue started to exhibit in Master
the commit is not present in 19.07 and thus does not exhibit there
other CGI scripts do not exhibit the issue in Master or 19.07
there is no apparent misconfiguration for the CGI exec path in the lighttpd config
the advise from the lighttpd forum and subsequent rfc3875 stipulations are rejected as potential solution
since lighttpd is not vanilla OpenWrt the issue is not being investigated in order to find a solution

jow- · 2020-01-09T11:10:38Z

Debugged it myself now. It indeed is a lighttpd specific problem. Lighttpd does not follow symlinks in /www/cgi-bin/, even when setting server.follow-symlink = "enable", instead it delivers symlinked CGI programs as files which would even be a security issue in some environments.

You need to add the following explicit cgi.assign mappings to make things work:

        "/cgi-bin/luci" => "",
        "/cgi-bin/cgi-backup" => "",
        "/cgi-bin/cgi-download" => "",
        "/cgi-bin/cgi-upload" => "",
        "/cgi-bin/cgi-exec" => ""

since then the issue started to exhibit in Master

Yes, because commit f3724e4 started using a new cgi-exec applet which wasn't used previously. Due to the inflexible nature of lighttpd's CGI configuration, you need to whitelist every new CGI applet which is started to get used by LuCI. This is exactly the reason why non-standard webserver setups are not supported officially, they break whenever there are changes in LuCI because they do not meet the expected environment behavior.

ghost · 2020-01-09T11:13:10Z

Thank you for having taking the time to get to the bottom of this.

jow- · 2020-01-09T11:19:38Z

Apparently the following is the only needed thing to provide the expected behavior of "execute anything in /cgi-bin/ as CGI":

server.modules += ( "mod_cgi" )

$HTTP["url"] =~ "^/cgi-bin" {
    cgi.assign = ( "" => "" )
}

I'd argue that this should even be the default contents of /etc/lighttpd/conf.d/30-cgi.conf

ghost · 2020-01-09T14:38:19Z

For posterity feedback from the lighttpd folks

server.follow-symlink = "enable" follows symlinks, as expected.
The path /cgi-bin is not magical.
Plain and simple: configuring cgi.assign is required for lighttpd to use mod_cgi to handle the request.

ghost closed this as completed Dec 23, 2019

ghost reopened this Dec 24, 2019

jow- closed this as completed Dec 24, 2019

ghost referenced this issue Jan 8, 2020

luci-base: fs.js: add read_direct() and exec_direct() functions

f3724e4

Signed-off-by: Jo-Philipp Wich <jo@mein.io>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[logs] content rendered illegible #3430

[logs] content rendered illegible #3430

ghost commented Dec 23, 2019

dibdot commented Dec 23, 2019

ghost commented Dec 23, 2019

dibdot commented Dec 23, 2019

ghost commented Dec 23, 2019 •

edited by ghost

jow- commented Dec 23, 2019

ghost commented Dec 23, 2019

jow- commented Dec 23, 2019

ghost commented Dec 23, 2019 •

edited by ghost

ghost commented Dec 24, 2019 •

edited by ghost

jow- commented Dec 24, 2019 •

edited

ghost commented Jan 8, 2020

jow- commented Jan 9, 2020 •

edited

ghost commented Jan 9, 2020

jow- commented Jan 9, 2020 •

edited

ghost commented Jan 9, 2020

jow- commented Jan 9, 2020 •

edited

ghost commented Jan 9, 2020

jow- commented Jan 9, 2020 •

edited

ghost commented Jan 9, 2020

[logs] content rendered illegible #3430

[logs] content rendered illegible #3430

Comments

ghost commented Dec 23, 2019

dibdot commented Dec 23, 2019

ghost commented Dec 23, 2019

dibdot commented Dec 23, 2019

ghost commented Dec 23, 2019 • edited by ghost

jow- commented Dec 23, 2019

ghost commented Dec 23, 2019

jow- commented Dec 23, 2019

ghost commented Dec 23, 2019 • edited by ghost

ghost commented Dec 24, 2019 • edited by ghost

jow- commented Dec 24, 2019 • edited

ghost commented Jan 8, 2020

jow- commented Jan 9, 2020 • edited

ghost commented Jan 9, 2020

jow- commented Jan 9, 2020 • edited

ghost commented Jan 9, 2020

jow- commented Jan 9, 2020 • edited

ghost commented Jan 9, 2020

jow- commented Jan 9, 2020 • edited

ghost commented Jan 9, 2020

ghost commented Dec 23, 2019 •

edited by ghost

ghost commented Dec 23, 2019 •

edited by ghost

ghost commented Dec 24, 2019 •

edited by ghost

jow- commented Dec 24, 2019 •

edited

jow- commented Jan 9, 2020 •

edited

jow- commented Jan 9, 2020 •

edited

jow- commented Jan 9, 2020 •

edited

jow- commented Jan 9, 2020 •

edited