New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Queries are logged if logging is disabled, flushing logs has no effect #2339

Closed
cypris75 opened this Issue Aug 8, 2018 · 16 comments

Comments

Projects
None yet
5 participants
@cypris75
Copy link

cypris75 commented Aug 8, 2018

In raising this issue, I confirm the following:

How familiar are you with the the source code relevant to this issue?:

1


Expected behaviour:

Query logging is disabled. I would expect to not see any queries in "Query Logs" page.

Pushing the button "Flush logs" should flush entries in "Query Logs" page.

Actual behaviour:

Even with Query Logging disabled I still can see new entries coming in to "Query Logs" page.

Pressing the button "Flush logs" has no effect. Entries in "Query Logs" page do not get removed. After pressing the button "Flush logs" I get the following success message: "The Pi-hole log file has been flushed"

Steps to reproduce:

I installed a fresh copy of Raspbian Stretch on a SD Card and installed Pi-hole with standard bash script (curl -sSL https://install.pi-hole.net | bash)

Debug token provided by uploading pihole -d log:

m3syq5oaoo

Troubleshooting undertaken, and/or other relevant information:

When I enable logging and press "Disable query logs and flush logs" this works as expected. "Query Logs" page is empty afterwards.

Tried also:

pi@pihole:~ $ pihole flush
  [✓] Flushed /var/log/pihole.log
  [✓] Deleted  queries from database

Queries re not deleted

Version info:

Linux pihole 4.14.50+
Pi-hole Version v4.0
Web Interface Version v4.0
FTL Version v4.0

@Mcat12

This comment has been minimized.

Copy link
Member

Mcat12 commented Aug 8, 2018

For now, does setting QUERY_DISPLAY to yes in /etc/pihole/pihole-FTL.conf (https://docs.pi-hole.net/ftldns/configfile/#query_display) and then restarting FTL (sudo service pihole-FTL restart) fix this?

@cypris75

This comment has been minimized.

Copy link

cypris75 commented Aug 8, 2018

Added QUERY_DISPLAY=yes to /etc/pihole/pihole-FTL.conf and restared via sudo service pihole-FTL restart but cannot spot any difference in behaviour.

@Mcat12

This comment has been minimized.

Copy link
Member

Mcat12 commented Aug 8, 2018

Sorry, it appears that setting was removed when FTLDNS was added (I will update the documentation for this). It was replaced by privacy levels.

Try adding this setting to the FTL config (and restart FTL):

PRIVACYLEVEL=3
@Mcat12

This comment has been minimized.

Copy link
Member

Mcat12 commented Aug 8, 2018

PR for updating the documentation: pi-hole/docs#44

@Mcat12

This comment has been minimized.

Copy link
Member

Mcat12 commented Aug 10, 2018

@cypris75 Does the privacy level setting help?

@cypris75

This comment has been minimized.

Copy link

cypris75 commented Aug 13, 2018

When the privacy level is set to "Level 3 - Paranoia mode" no logs show up. But when I set it back to < 2 then the previous logs show up again. So logs don't get flushed.

@DL6ER

This comment has been minimized.

Copy link
Member

DL6ER commented Aug 14, 2018

Flushing of logs happens on the hard drive (may also be an SD card, depending on your system). However, pihole-FTL holds all data of the most recent 24 hours in memory to be able to serve any requests without having to rely on heavy I/O operation. This makes it very fast.

Having said this, flushing does only affect the displayed data when you restart the DNS service afterwards as, on restart, pihole-FTL has to fill its memory with data it reads from the database. If the database has been flushed there is nothing to import.

When the privacy level is set to "Level 3 - Paranoia mode" no logs show up.

This is expected.

But when I set it back to < 2 then the previous logs show up again.

This is also expected. To be able to compute the statistics of level 3, we need to record the queries (so we know how many queries have been blocked at a given time, etc.). When you decrease the privacy level, you will, however, only get fully anonymized data. Neither the requested domains nor the requesting clients will be visible whatever you do with the privacy level. This is also expected behavior. Is this not enough for you? You could also just hide away the password of your dashboard and allow SSH login only via (your) keys to be sure that nobody will be able to change the privacy level from 3 to something lower if this is a concern.

@cypris75

This comment has been minimized.

Copy link

cypris75 commented Aug 14, 2018

First I want to mention that I love this product. Kudos to what has been developed so far. Great piece of software.

I think for the average (non-technical) user (which I am) this is all perhaps a bit too confusing. There is a log file and a big red button which says "Flush logs". So the expectation would be that those logs go away if you hit this button. This is currently not happening. At least for my setup.

And my other expectation is, that if I set the log level to "Paranoia mode" then everything gets deleted or not logged at all.

Lets assume you install the software, test it, browse to some sensitive websites and then you want to set it to production, flush the logs and enable paranoia mode. As a user you probably think you are done.

But then an "interested party" comes by, sets the log level to normal and sees the full log history. I think this is something you don't want to happen.

@DL6ER

This comment has been minimized.

Copy link
Member

DL6ER commented Aug 14, 2018

We cannot apply the privacy level retroactively. Each query is affected by the privacy level that was configured while this query has been made. I think this is expected behavior.

The "interested party" should not be allowed to tinker around with your privacy settings, but I do get your point that flushing the logs should also restart the DNS resolver to wipe everything also from its memory. I will work on this today.

@DL6ER DL6ER referenced this issue Aug 14, 2018

Merged

Restart pihole-FTL after log flushing #2358

8 of 8 tasks complete
@samscode

This comment has been minimized.

Copy link

samscode commented Aug 16, 2018

It would be nice to pin a concise workaround for this issue. It took me a while to figure out what this meant exactly...

Try adding this setting to the FTL config (and restart FTL):

PRIVACYLEVEL=3

I noticed I had an empty file called /etc/pihole/pihole-FTL.conf mentioned above that comment and it turned out that was the file referred to as "the FTL config".

During the upgrade, I got no prompt for enabling logs. For me, this change was unexpected behaviour. What if people don't have the web interface enabled or don't check it? They wont even realise they suddenly have query logs enabled. I don't know about other people, but I don't spend a whole like of time ogling DNS analytics. Honestly, the only reason I enabled the web interface was for the nice block page. I'm not actually real keen on having a PHP stack on my Pi just for that and I'm waiting for support to remap port 80 so I can replace it with my own nginx based static block pages.

But aside from that, the main reason I disabled logging throughout my Pi is so I can mount the SD card readonly (which I have to toggle for upgrades, then check everything still runs readonly). I imagine others may have serious privacy concerns and have no indication there's been a change.

@samscode

This comment has been minimized.

Copy link

samscode commented Aug 17, 2018

It seems worse than I thought actually, I've just done a fresh install with the little command line GUI just now and I've specifically selected the options to install the webserver, but not to log queries. When entering into the admin interface I see logs of every DNS query. I think there's either a clear bug here or a serious misunderstanding of the English language. I'm not trying to be rude or anything, it's just that the concept of "expected behaviour" needs a bit more investigation.

@technicalpyro

This comment has been minimized.

Copy link
Contributor

technicalpyro commented Aug 17, 2018

@samscode please refer to This documentation for full explanation we are a bit behind in updating the install process to do this as you seem to expect

@samscode

This comment has been minimized.

Copy link

samscode commented Aug 17, 2018

I've mostly avoided the web interface and didn't notice the other features because I highly valued "no logging". I do appreciate this aspect of the project is the high visibility and high value feature for most people. Sorry if I was a bit confrontational.

In any GUI or documentation can I suggest it would be more respectful terminology to replace "paranoia mode" with just "no logging mode". My use case just equates to "I really don't care so why waste resources" mode. "Paranoia mode" makes it seem like this category of use cases are simply crack pots.

@Mcat12

This comment has been minimized.

Copy link
Member

Mcat12 commented Aug 20, 2018

A full no-logging mode is now in development: pi-hole/FTL#357

I will look into changing the wording for the maximum privacy mode.

@Mcat12

This comment has been minimized.

Copy link
Member

Mcat12 commented Aug 20, 2018

@samscode pi-hole/docs#49 changes the wording of privacy level 3.

@Mcat12

This comment has been minimized.

Copy link
Member

Mcat12 commented Aug 21, 2018

#2384 will let you set the privacy level during install.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment