lighttpd: do not overwrite /etc/lighttpd/lighttpd.conf (On new installs) #5075

gstrauss · 2022-12-19T08:04:55Z

What does this PR aim to accomplish?:

lighttpd: do not overwrite /etc/lighttpd/lighttpd.conf (on new installations)

How does this PR accomplish the above?:

This PR requires #5065 and subsumes #5066.

This PR shows how pi-hole can avoid overwriting /etc/lighttpd/lighttpd.conf for new installations, while still maintaining lighttpd.conf for existing pi-hole installations.

By submitting this pull request, I confirm the following:

I have read and understood the contributors guide, as well as this entire template. I understand which branch to base my commits and Pull Requests against.
I have commented my proposed changes within the code and I have tested my changes.
I am willing to help maintain this change if there are issues with it later.
It is compatible with the EUPL 1.2 license
I have squashed any insignificant commits. (git rebase)
I have checked that another pull request for this purpose does not exist.
I have considered, and confirmed that this submission will be valuable to others.
I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
I give this submission freely, and claim no ownership to its content.

I have read the above and my PR is ready for review. Check this box to confirm

gstrauss · 2022-12-19T08:58:57Z

On Fedora-based systems, not sure why sed rename is failing in the tests. I'll continue debugging later.

DEBUG    testinfra:base.py:288 RUN CommandResult(command=b'docker exec f1349c07597be5edc1e424f453b46258d32d8af6473eb5f9c2abbe0b5d5ba775 /bin/bash -c \'sed -i \'"\'"\'s/nameserver.*/nameserver 127.0.0.1/\'"\'"\' /etc/resolv.conf\'', exit_status=4, stdout=None, stderr=b'sed: cannot rename /etc/sedtvsuT3: Device or resource busy\n')
DEBUG    testinfra:base.py:288 RUN CommandResult(command=b'docker exec f1349c07597be5edc1e424f453b46258d32d8af6473eb5f9c2abbe0b5d5ba775 /bin/bash -c \'curl -s --head "http://127.0.0.1/admin/" | head -n 1 | grep "HTTP/1.[01] [23].." > /dev/null\'', exit_status=1, stdout=None, stderr=None)

gstrauss · 2022-12-19T09:49:48Z

The sed error is fixed with some docker-foo, but there remains a test error retrieving the /admin/ page for the Fedora-based builds. Will troubleshoot more tomorrow.

Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>

gstrauss · 2022-12-19T20:30:37Z

Tests were failing on Fedora due to the system lighttpd.conf issuing some warnings and the tests treating warnings trace as an error. I'll work out some adjustments...

Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>

gstrauss · 2022-12-19T21:53:04Z

Updated description now that #5065 has been merged. This PR subsumes #5066.

This PR shows how pi-hole can avoid overwriting /etc/lighttpd/lighttpd.conf for new installations, while still maintaining lighttpd.conf for existing pi-hole installations. As discussed in #5066, this PR also removes obsolete pihole block page error handler.

@PromoFaux: tests all pass, including on Fedora 37, newly added to the pi-hole CI pipeline.

gstrauss · 2022-12-19T22:02:29Z

For compatibility with existing installations, the only potential breaking change is removal of the obsolete pihole block page error handler. Most users are not using the obsolete pihole block page error handler; BLOCK_IPV4 and BLOCK_IPV6 are not used in the default pi-hole DNS resolver config.

Existing users can opt-in to convert to the new, less-invasive configuration -- which does not overwrite the system lighttpd.conf -- by uninstalling pi-hole and answering No to question ${QST} Do you wish to go through each dependency for removal? (Choosing No will leave all dependencies installed) [Y/n], and then reinstalling pi-hole.

test/test_any_automated_install.py

remove obsolete pihole block page error handler x-ref: Remove the advanced functionality of the 404 page (Blockpage) pi-hole#3910 Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>

(/var/www/html/pihole/ dir and contents are still removed in uninstall) (/var/www/html/index.lighttpd.orig is still removed in uninstall) Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>

PromoFaux · 2022-12-19T23:35:20Z

by uninstalling pi-hole

This is probably like using a sledgehammer to crack a nut. Simpler:

delete the pi-hole flavoured /etc/lighttpd/lighttpd.conf
One of (untested!):
- uninstall and reinstall lighttpd (will this install a default config?)
- replace /etc/lighttpd/lighttpd.conf with a default config manually
pihole -r to repair the installation, and it will no longer hit this line: https://github.com/pi-hole/pi-hole/pull/5075/files#diff-595630a29a855f6d667a84ca0662042e826bf3ec56322ef61d4a6ef149147d23R1403

gstrauss · 2022-12-20T00:54:23Z

by uninstalling pi-hole

This is probably like using a sledgehammer to crack a nut. Simpler:

* delete the pi-hole flavoured `/etc/lighttpd/lighttpd.conf`

* One of (untested!):
  
  * uninstall and reinstall `lighttpd` (will this install a default config?)
  * replace `/etc/lighttpd/lighttpd.conf` with a default config manually

Haha. Yes.
pi-hole install backs up /etc/lighttpd/lighttpd.conf to /etc/lighttpd/lighttpd.conf.orig, and the uninstall restores that.

        if [[ -f /etc/lighttpd/lighttpd.conf.orig ]]; then
            ${SUDO} mv /etc/lighttpd/lighttpd.conf.orig /etc/lighttpd/lighttpd.conf 
        fi

The uninstall also cleans up and removes /var/www/html/pihole (if empty), which includes removing index.php, so the uninstall would do a bit of cleanup in addition to restoring the original lighttpd.conf.

In any case, there are a small number of steps that can be performed manually for existing installations that wish to migrate.

However, note that the (now-removed) code in basic-install.sh which backed up lighttpd.conf would overwrite an existing backup, so running the pi-hole installer without running the uninstaller could theoretically have overwritten the original backup. In that case, reinstalling the system lighttpd package and then merging the lighttpd.conf.rpmnew or lighttpd.conf-opkg, or ... could restore the system-provided lighttpd.conf.

-        elif [[ -f "${lighttpdConfig}" ]]; then
-            # back up the original
-            mv "${lighttpdConfig}"{,.orig}
-        fi

gstrauss · 2022-12-20T01:08:18Z

Thank you very much for your assistance and guidance here, @PromoFaux

If lighttpd-related questions come up in the future asking how to do something using lighttpd, please tag me.

PromoFaux · 2022-12-20T19:38:07Z

advanced/index.php

-
-// Determine block page type
-if ($serverName === "pi.hole"
-    || (!empty($_SERVER["VIRTUAL_HOST"]) && $serverName === $_SERVER["VIRTUAL_HOST"])) {


@gstrauss I totally forgot to mention/ask - are we able to reproduce this in pihole-admin.conf where the following lines are?

pi-hole/advanced/pihole-admin.conf

Lines 62 to 66 in ec1d4c5

$HTTP["host"] == "pi.hole" {

$HTTP["url"] == "/" {

url.redirect = ("" => "/admin/")

}

}

If my docs reading is correct, is it something like the following?

$HTTP["host"] == "pi.hole" || $HTTP["host"] == env.VIRTUAL_HOST { $HTTP["url"] == "/" { url.redirect = ("" => "/admin/") } }

lighttpd.conf syntax is historically its own custom syntax (for better and worse), and is not shell syntax. If VIRTUAL_HOST is set in the shell environment before starting lighttpd, then this syntax should work:

$HTTP["url"] == "/" { $HTTP["host"] == "pi.hole" { url.redirect = ("" => "/admin/") } $HTTP["host"] == env.VIRTUAL_HOST { url.redirect = ("" => "/admin/") } }

but the above will fail if VIRTUAL_HOST is not set. It should at least be set to VIRTUAL_HOST="" to work
(Constructing a regex using lighttpd.conf syntax is possible but ugly and verbose
e.g. ^(" + env.VIRTUAL_HOST + "|pi.hole)$)
That also requires VIRTUAL_HOST be set in the shell environment, and not missing from the shell env.

Another option, which I know is in wide use, is to use lighttpd.conf include_shell to run a short shell script which generates the simpler lighttpd.conf syntax to do what is desired, which could generate the syntax I posted above for a list of pi-hole admin hosts ("127.0.0.1", "::1", "localhost", "pi.hole", etc)

Lastly, a few lines of lua can be written and used with lighttpd mod_magnet. Lua is a programming language and is much more expressive than lighttpd.conf syntax. Whenever someone asks "can I do this arbitrarily complex thing in lighttpd.conf?", the answer frequently includes: one option is to use lighttpd mod_magnet and a lua script.

Maybe not as simple as you had hoped, but there are solutions available. If you want to extend what is currently checked in, I would suggest include_shell and a shell script to generate the simpler lighttpd.conf directives to do exactly what you want. I think requiring VIRTUAL_HOST be set in the environment will complicate things for various existing scripts in the OS which check lighttpd -f /etc/lighttpd.conf -tt and expect it to work (without knowing that VIRTUAL_HOST needs to be set in the environment)

BTW, the "better" or advantages of lighttpd.conf syntax is realized in how lighttpd uses lighttpd.conf: lighttpd parses and generates a static configuration at startup, with limited conditionals allowed by the syntax. Processing the lighttpd configuration for each and every request at runtime is much faster than runtime config parsing done by many other web servers.

Intercepting and redirecting "/" might be desirable for new installations, but not for existing lighttpd configs which are used for other purposes. If you do use include_shell and create a shell script for this configuration, instead of adding it to pihole-admin.conf, you might consider a new file pihole-admin-redir.conf so that some people can more easily disable just that part.

Alternatively -- and I think the best solution for this very-specific pi-hole use -- you can construct a simple index.html that you drop into the document root if index.* is not already present, and that index.html can be a splash page and/or can do the redirection in the HTML <meta http-equiv="Refresh" content="0; url='/admin/'" />

Thanks for the information.

On reflection, I have decided that we do not need to include the functionality - But I spotted it and realised that I had overlooked it before, so wanted to find a way back in case we needed it.

pralor-bot · 2022-12-21T20:58:51Z

This pull request has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/pi-hole-ftl-v5-20-and-web-v5-18-released/59959/1

pralor-bot · 2023-01-15T20:11:03Z

This pull request has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/pi-hole-ftl-v5-20-1-web-v5-18-1-and-core-v5-15-released/60495/1

pralor-bot · 2023-01-16T21:15:15Z

This pull request has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/yesterdays-update-removed-the-index-html/60531/2

pralor-bot · 2023-11-25T21:04:33Z

This pull request has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/changing-default-listening-port-of-lighttpd-in-raspberry-pi-os-no-longer-honors-external-conf/62235/9

gstrauss force-pushed the lighttpd-simplify-2 branch 2 times, most recently from 39ba72c to 043ad56 Compare December 19, 2022 08:47

gstrauss force-pushed the lighttpd-simplify-2 branch 3 times, most recently from aaec711 to 8de590f Compare December 19, 2022 09:31

gstrauss force-pushed the lighttpd-simplify-2 branch 11 times, most recently from 02bc95b to 5b6d6c1 Compare December 19, 2022 19:55

lighttpd: remove obsolete exclusion of letsencrypt.conf

0d5d3a1

Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>

gstrauss force-pushed the lighttpd-simplify-2 branch from 5b6d6c1 to 4211397 Compare December 19, 2022 20:17

gstrauss mentioned this pull request Dec 19, 2022

lighttpd: remove obsolete pihole block page error handler #5066

Closed

1 task

gstrauss marked this pull request as draft December 19, 2022 20:34

gstrauss force-pushed the lighttpd-simplify-2 branch from 4211397 to 9b53d74 Compare December 19, 2022 21:09

lighttpd: do not overwrite /etc/lighttpd/lighttpd.conf

c6342ed

Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>

gstrauss marked this pull request as ready for review December 19, 2022 21:24

gstrauss force-pushed the lighttpd-simplify-2 branch from 9b53d74 to 5ac671a Compare December 19, 2022 21:24

PromoFaux requested changes Dec 19, 2022

View reviewed changes

test/test_any_automated_install.py Outdated Show resolved Hide resolved

gstrauss added 2 commits December 19, 2022 17:53

lighttpd: remove pihole block page error handler

539f9d4

remove obsolete pihole block page error handler x-ref: Remove the advanced functionality of the 404 page (Blockpage) pi-hole#3910 Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>

lighttpd: do not install obsolete 404 handler

34f45d0

(/var/www/html/pihole/ dir and contents are still removed in uninstall) (/var/www/html/index.lighttpd.orig is still removed in uninstall) Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>

gstrauss force-pushed the lighttpd-simplify-2 branch from 5ac671a to 34f45d0 Compare December 19, 2022 22:53

PromoFaux changed the title ~~lighttpd: do not overwrite /etc/lighttpd/lighttpd.conf~~ lighttpd: do not overwrite /etc/lighttpd/lighttpd.conf (On new installs) Dec 19, 2022

PromoFaux approved these changes Dec 19, 2022

View reviewed changes

PromoFaux merged commit ec1d4c5 into pi-hole:development Dec 19, 2022

PromoFaux mentioned this pull request Dec 19, 2022

Test lighttpd was build with lua support #5073

Closed

1 task

PromoFaux reviewed Dec 20, 2022

View reviewed changes

PromoFaux mentioned this pull request Jan 15, 2023

[Docker Only] Configure lighttpd to redirect http://VIRTUAL_HOST or http://FTLCONF_LOCAL_IPV4 to /admin pi-hole/docker-pi-hole#1271

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lighttpd: do not overwrite /etc/lighttpd/lighttpd.conf (On new installs) #5075

lighttpd: do not overwrite /etc/lighttpd/lighttpd.conf (On new installs) #5075

gstrauss commented Dec 19, 2022 •

edited

Loading

gstrauss commented Dec 19, 2022

gstrauss commented Dec 19, 2022

gstrauss commented Dec 19, 2022

gstrauss commented Dec 19, 2022

gstrauss commented Dec 19, 2022

PromoFaux commented Dec 19, 2022

gstrauss commented Dec 20, 2022

gstrauss commented Dec 20, 2022

PromoFaux Dec 20, 2022

gstrauss Dec 20, 2022

gstrauss Dec 20, 2022

gstrauss Dec 20, 2022

PromoFaux Dec 21, 2022

pralor-bot commented Dec 21, 2022

pralor-bot commented Jan 15, 2023

pralor-bot commented Jan 16, 2023

pralor-bot commented Nov 25, 2023

	$HTTP["host"] == "pi.hole" {
	$HTTP["url"] == "/" {
	url.redirect = ("" => "/admin/")
	}
	}

lighttpd: do not overwrite /etc/lighttpd/lighttpd.conf (On new installs) #5075

lighttpd: do not overwrite /etc/lighttpd/lighttpd.conf (On new installs) #5075

Conversation

gstrauss commented Dec 19, 2022 • edited Loading

gstrauss commented Dec 19, 2022

gstrauss commented Dec 19, 2022

gstrauss commented Dec 19, 2022

gstrauss commented Dec 19, 2022

gstrauss commented Dec 19, 2022

PromoFaux commented Dec 19, 2022

gstrauss commented Dec 20, 2022

gstrauss commented Dec 20, 2022

PromoFaux Dec 20, 2022

Choose a reason for hiding this comment

gstrauss Dec 20, 2022

Choose a reason for hiding this comment

gstrauss Dec 20, 2022

Choose a reason for hiding this comment

gstrauss Dec 20, 2022

Choose a reason for hiding this comment

PromoFaux Dec 21, 2022

Choose a reason for hiding this comment

pralor-bot commented Dec 21, 2022

pralor-bot commented Jan 15, 2023

pralor-bot commented Jan 16, 2023

pralor-bot commented Nov 25, 2023

gstrauss commented Dec 19, 2022 •

edited

Loading