`local.d` behavior doesn't match FAQ #563

moisseev · 2016-03-18T14:27:59Z

According to the What are local.d and override.d then section local.d/phishing.conf should look like:

phishing {
    redirector_domains = [
        "${CONFDIR}/redirectors.map.local:LOCAL_REDIRECTOR_FALSE"
    ];
}

but it doesn't work.

This works:

    redirector_domains = [
        "${CONFDIR}/redirectors.map.local:LOCAL_REDIRECTOR_FALSE"
    ];

The text was updated successfully, but these errors were encountered:

moisseev · 2016-03-18T15:04:07Z

'local.d/worker-controller.inc' doesn't work.

2016-03-18 17:49:32 #87322(controller) <6fjagz>; controller; rspamd_check_encrypted_password: incorrect or absent password has been specified
2016-03-18 17:49:32 #87322(controller) <6fjagz>; controller; rspamd_check_encrypted_password: incorrect or absent password has been specified

There is no problem with worker-controller.inc.local with the same content.

vstakhov · 2016-03-18T17:15:22Z

worker {
    bind_socket = "*:11333";
    .include "$CONFDIR/worker-normal.inc"
    .include(try=true; priority=1) "$LOCAL_CONFDIR/local.d/worker-normal.inc"
    .include(try=true; priority=10) "$LOCAL_CONFDIR/override.d/worker-normal.inc"
}

worker {
    bind_socket = "localhost:11334";
    .include "$CONFDIR/worker-controller.inc"
    .include(try=true; priority=1) "$LOCAL_CONFDIR/local.d/worker-controller.inc"
    .include(try=true; priority=10) "$LOCAL_CONFDIR/override.d/worker-controller.inc"
}

That's from the default configuration.

moisseev · 2016-03-18T17:40:14Z

Yes, I have that lines in the rspamd.conf

mx# rspamadm configtest
syntax OK
mx# mv worker-controller.inc.local local.d/worker-controller.inc
mx# rspamadm configtest
2016-03-18 20:34:13 #92257(rspamadm) <6fjagz>; cfg; rspamd_config_read: ucl parser error: (NULL)
syntax BAD

mx# cat local.d/worker-controller.inc
# Included from top-level .conf file

type = "controller";
count = 1;
password = "$------------------";
enable_password = "$------------------";
secure_ip = "127.0.0.1";
secure_ip = "::1";
static_dir = "${WWWDIR}";

mx# cat rspamd.conf
# System V init adopted top level configuration

# Please don't modify this file as your changes might be overwritten with
# the next update.
#
# You can modify '$LOCAL_CONFDIR/rspamd.conf.local.override' to redefine
# parameters defined on the top level
#
# You can modify '$LOCAL_CONFDIR/rspamd.conf.local' to add
# parameters defined on the top level
#
# For specific modules or configuration you can also modify
# '$LOCAL_CONFDIR/local.d/file.conf' - to add your options or rewrite defaults
# '$LOCAL_CONFDIR/override.d/file.conf' - to override the defaults
#
# See https://rspamd.com/doc/tutorials/writing_rules.html for details

.include "$CONFDIR/common.conf"

options {
    pidfile = "$RUNDIR/rspamd.pid";
    .include "$CONFDIR/options.inc"
    .include(try=true; priority=1) "$LOCAL_CONFDIR/local.d/options.inc"
    .include(try=true; priority=10) "$LOCAL_CONFDIR/override.d/options.inc"
}

logging {
    type = "file";
    filename = "$LOGDIR/rspamd.log";
    .include "$CONFDIR/logging.inc"
    .include(try=true; priority=1) "$LOCAL_CONFDIR/local.d/logging.inc"
    .include(try=true; priority=10) "$LOCAL_CONFDIR/override.d/logging.inc"
}

worker {
    bind_socket = "*:11333";
    .include "$CONFDIR/worker-normal.inc"
    .include(try=true; priority=1) "$LOCAL_CONFDIR/local.d/worker-normal.inc"
    .include(try=true; priority=10) "$LOCAL_CONFDIR/override.d/worker-normal.inc"
}

worker {
    bind_socket = "localhost:11334";
    .include "$CONFDIR/worker-controller.inc"
    .include(try=true; priority=1) "$LOCAL_CONFDIR/local.d/worker-controller.inc"
    .include(try=true; priority=10) "$LOCAL_CONFDIR/override.d/worker-controller.inc"
}

vstakhov · 2016-03-18T17:41:35Z

Can you put rspamadm configdump output for the relevant sections? That should work as described...

vstakhov · 2016-03-18T17:43:18Z

Oh wait, that's UCL error. That's worse...

vstakhov · 2016-03-18T18:22:34Z

Hm, I can't reproduce this issue. It loads local file just fine.

moisseev · 2016-03-18T18:45:14Z

I've tried on another server. Result is the same.

# mv worker-controller.inc.local local.d/worker-controller.inc
# rspamadm configtest
2016-03-18 21:40:30 #94354(rspamadm) <6fjagz>; cfg; rspamd_config_read: ucl parser error: (NULL)
syntax BAD

# uname -rsv
FreeBSD 10.2-RELEASE-p7 FreeBSD 10.2-RELEASE-p7 #0: Mon Nov  2 14:19:39 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC
# pkg info | grep rspamd
rspamd-devel-1.2.0.b5.17032016 Fast spam filtering system dev version

vstakhov · 2016-03-19T14:29:18Z

I've tried fresh rspamd installation on debian:

mkdir /etc/rspamd/local.d
vim /etc/rspamd/local.d/worker-controller.inc

Then I've added exactly your lines and tried configtest:

# rspamadm configtest
syntax OK

To ensure that file has been loaded, I've tried configdump:

# rspamadm configdump worker
*** Section worker ***
bind_socket = "systemd:0";
type = "normal";
mime = true;
task_timeout = 8.0;
*** End of section worker ***
*** Section worker ***
bind_socket = "systemd:1";
type = "controller";
count = 1;
password = "$------------------";
secure_ip = "127.0.0.1";
secure_ip = "::1";
static_dir = "/usr/share/rspamd/www";
enable_password = "$------------------";
*** End of section worker ***

So that settings have been loaded. I couldn't reproduce your issue again...

moisseev · 2016-03-19T15:56:23Z

I found the problem.
I had this in rspamd.conf.local.override:

worker {
    bind_socket = "localhost:11333";
    .include "$CONFDIR/worker-normal.inc"
}

worker {
    bind_socket = "*:11334";
    .include "$CONFDIR/worker-controller.inc.local"
}

moisseev · 2016-03-19T16:12:47Z

What about local.d/phishing.conf ?

moisseev · 2016-03-19T17:11:57Z

Never mind my question about local.d/phishing.conf. I've just misread the FAQ example. Sorry.

What if you move bind_socket options from rspamd.conf to worker-controller.inc and worker-normal.inc in default configuration? It will be easier to control sockets in local configuration then.

moisseev · 2016-03-20T08:04:16Z

Is there some good reason to include local.d files with priority=1?
Despite local.d and override.d have different priorities, they both redefine complete objects.
It's confusing.

# cat local.d/fuzzy_check.conf
    rule {
        servers = "127.0.0.1:11335";
        symbol = "LOCAL_FUZZY_UNKNOWN";
        mime_types = "*";
        max_score = 20.0;
        read_only = no;
        skip_unknown = yes;
        fuzzy_map = {
            LOCAL_FUZZY_DENIED {
                max_score = 20.0;
                flag = 11;
            }
            LOCAL_FUZZY_PROB {
                max_score = 10.0;
                flag = 12;
            }
            LOCAL_FUZZY_WHITE {
                max_score = 2.0;
                flag = 13;
            }
        }
    }

# rspamadm configdump fuzzy_check
*** Section fuzzy_check ***
rule {
    servers = "127.0.0.1:11335";
    symbol = "LOCAL_FUZZY_UNKNOWN";
    mime_types = "*";
    max_score = 20.0;
    read_only = false;
    skip_unknown = true;
    fuzzy_map {
        LOCAL_FUZZY_DENIED {
            max_score = 20.0;
            flag = 11;
        }
        LOCAL_FUZZY_PROB {
            max_score = 10.0;
            flag = 12;
        }
        LOCAL_FUZZY_WHITE {
            max_score = 2.0;
            flag = 13;
        }
    }
}
min_bytes = 100;
*** End of section fuzzy_check ***

Why not to include local.d with 0 priority? It will merge config elements like rspamd.conf.local :

--- modules.d/fuzzy_check.conf.orig     2016-03-20 10:47:13.093192000 +0300
+++ modules.d/fuzzy_check.conf  2016-03-20 10:47:43.113162000 +0300
@@ -16,7 +16,7 @@
 fuzzy_check {
     # Include dynamic conf for the rule
     .include(try=true,priority=5) "${DBDIR}/dynamic/fuzzy_check.conf"
-    .include(try=true,priority=1) "$LOCAL_CONFDIR/local.d/fuzzy_check.conf"
+    .include(try=true) "$LOCAL_CONFDIR/local.d/fuzzy_check.conf"
     .include(try=true,priority=10) "$LOCAL_CONFDIR/override.d/fuzzy_check.conf"
     min_bytes = 100;
     rule {

# rspamadm configdump fuzzy_check
*** Section fuzzy_check ***
rule {
    servers = "127.0.0.1:11335";
    symbol = "LOCAL_FUZZY_UNKNOWN";
    mime_types = "*";
    max_score = 20.0;
    read_only = false;
    skip_unknown = true;
    fuzzy_map {
        LOCAL_FUZZY_DENIED {
            max_score = 20.0;
            flag = 11;
        }
        LOCAL_FUZZY_PROB {
            max_score = 10.0;
            flag = 12;
        }
        LOCAL_FUZZY_WHITE {
            max_score = 2.0;
            flag = 13;
        }
    }
}
rule {
    servers = "rspamd.com:11335";
    encryption_key = "icy63itbhhni8bq15ntp5n5symuixf73s1kpjh6skaq4e7nx5fiy";
    symbol = "FUZZY_UNKNOWN";
    mime_types [
        "application/*",
    ]
    max_score = 20.0;
    read_only = true;
    skip_unknown = true;
    fuzzy_map {
        FUZZY_DENIED {
            max_score = 20.0;
            flag = 1;
        }
        FUZZY_PROB {
            max_score = 10.0;
            flag = 2;
        }
        FUZZY_WHITE {
            max_score = 2.0;
            flag = 3;
        }
    }
}
min_bytes = 100;
*** End of section fuzzy_check ***

This fixes rspamd#672 and fixes rspamd#563.

moisseev mentioned this issue Jun 26, 2016

How to use adding and overriding of fuzzy rules feature? #672

Closed

moisseev added a commit to moisseev/rspamd that referenced this issue Jul 1, 2016

[Conf] Fix fuzzy_check.conf includes order

ff186e4

This fixes rspamd#672 and fixes rspamd#563.

moisseev mentioned this issue Jul 1, 2016

[Conf] Fix fuzzy_check.conf includes order #691

Merged

vstakhov closed this as completed in #691 Jul 1, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`local.d` behavior doesn't match FAQ #563

`local.d` behavior doesn't match FAQ #563

moisseev commented Mar 18, 2016

moisseev commented Mar 18, 2016

vstakhov commented Mar 18, 2016

moisseev commented Mar 18, 2016

vstakhov commented Mar 18, 2016

vstakhov commented Mar 18, 2016

vstakhov commented Mar 18, 2016

moisseev commented Mar 18, 2016

vstakhov commented Mar 19, 2016

moisseev commented Mar 19, 2016

moisseev commented Mar 19, 2016

moisseev commented Mar 19, 2016

moisseev commented Mar 20, 2016

local.d behavior doesn't match FAQ #563

local.d behavior doesn't match FAQ #563

Comments

moisseev commented Mar 18, 2016

moisseev commented Mar 18, 2016

vstakhov commented Mar 18, 2016

moisseev commented Mar 18, 2016

vstakhov commented Mar 18, 2016

vstakhov commented Mar 18, 2016

vstakhov commented Mar 18, 2016

moisseev commented Mar 18, 2016

vstakhov commented Mar 19, 2016

moisseev commented Mar 19, 2016

moisseev commented Mar 19, 2016

moisseev commented Mar 19, 2016

moisseev commented Mar 20, 2016

`local.d` behavior doesn't match FAQ #563

`local.d` behavior doesn't match FAQ #563