Really big logs

[manolodd]

Hi JPMens,

I've discovered the mosquitto.log file under /var/log/upstart that, due to its content, I suppose that belongs to mosquitto-auth-plug. It has caused some failures because of my disk is full (it is a virtualized machine). There is not problem as I have already trunked it to 0 bytes, but... ¿Is there any possibility of deactivate logging option for mosquitto-auth-plug? Or at least It could be useful to be able to specify the desired location for it (on another /dev or partition, on /dev/null, on a secondary storage...)

Thanks so much.

[jpmens]

Logging via Mosquitto's logging functions will be possible soon I hope (see #9 ); this will ensure that fprintf to stderr will stop and should take care of your issue.

[manolodd]

Ok, thank you so much. I din't see #9, sorry. I'll wait for it.

[jpmens]

FYI this is taking longer than expected b/c I'm not comfortable yet with how Mosquitto's logging has been surfaced in the plugin; it is possible to crash the server.

[manolodd]

JPMens, thank you for the info.

I'll wait for the solution :-)

[mikini]

Hi @manolodd / @jpmens.
I have today seen what seems to be the exact same problem on a VMware Player virtualized Ubuntu 14.04 using Mosquitto v1.4.5. It had accumulated a ~4GiB log file in /var/log/upstart/mosquitto.log that deprived the system of available diskspace.
We are however not using mosquitto-auth-plugin and if your problem is the same as the one I experienced, I don't think it is at all related to mosquitto-auth-plugin.

I was able to pinpoint the reason to the fact that the mosquitto server is outputting log messages both to the defined "log_dest file" in my configuration file and to stderr. The startup daemon used to control services will by default (since upstart v1.4, see http://upstart.ubuntu.com/cookbook/#id152) save stderr from spawned daemons to files in /var/log/upstart/.

This, although subtly documented but rather strange, mosquitto behaviour can be eliminated by a simple twist in the mosquitto configuration. It is caused by the default log_dest being "stderr" (see man mosquitto.conf). So when not specifying other log_dest clauses than a "log_dest file" (and I think the default Debian/Ubuntu setup is like that) the configured log_dest will be "file" in addition to "stderr". The fix being to turn off the default "stderr" logging before specifying the log method you actually want by inserting a "log_dest none" clause before other "log_dest" clauses.

I intend to bring this up in upstream mosquitto. At least a sane default config on Debian/Ubuntu should not duplicate logs in /var/log/mosquitto.log and /var/log/upstart/mosquitto.log.

Hope this will help you.

[jpmens]

I've tested again with 1.4.11 (OS/X) and I can still crash the server.

[jpmens]

This is now working here, so I will close this (after a very loong time, sorry!)