email:domain multimap filter doesn't match when MIME header encoding is used

[drook]

Classification (Please choose one option):

• Crash/Hang/Data loss
• WebUI/Usability
• Serious bug
• Other bug
• Feature
• Enhancement

Reproducibility (Please choose one option):

• Always
• Sometimes
• Rarely
• Unable
• I didn’t try
• Not applicable

Rspamd version:

1.4.1

Operation system, CPU, memory and environment:

FreeBSD 11.0-RELEASE-p5, Intel E5620, 32 GB RAM

Description (Please provide a descriptive summary of the issue):

Assume I have a multimap:

multimap {
  spamsenders_domain {
    type = "header";
    header = "From";
    filter = "email:domain";
    map = "$CONFDIR/maps/spamsenders_domain.map";
    symbol = "SPAMSENDERS_DOMAIN";
    description = "Spam senders domains";
  }
}

metric "default" {
  group {
    name = "local";
    symbol "SPAMSENDERS_DOMAIN" {
      description = "Message from known spam senders domains, additional score applied.";
      weight = 9.5;
    }
  }
}

Assume I have a record mcbaz.ru in this multimap.

Then this message will never match it:

To: emz@norma.perm.ru
From: =?UTF-8?Q?=D0=91=D0=B0=D0=B7=D0=B8=D1=81=2C=20=D1=83=D1=87=D0=B5=D0=B1=D0=BD=D1=8B=D0=B9=20=D1=86=D0=B5=D0=BD=D1=82=D1=80?= <metodist@mcbaz.ru>
MIME-Version: 1.0
Subject: test

test

But this will:

To: emz@norma.perm.ru
From: Someone special <metodist@mcbaz.ru>
Subject: test

test

[drook]

Real message full headers, if needed:

Return-Path: <noreply@mcbaz.ru>
Received: from gw0.qwerty.perm.ru ([unix socket])
         by gw0.qwerty.perm.ru (Cyrus v2.4.18) with LMTPA;
         Thu, 22 Dec 2016 13:12:12 +0500
X-Sieve: CMU Sieve 2.4
Received: from mcbaz.ru (mcbaz.ru [153.92.250.164])
        by gw0.qwerty.perm.ru (8.15.2/8.15.2) with ESMTP id uBM8C8E4065108
        for <osp@qwerty.perm.ru>; Thu, 22 Dec 2016 13:12:09 +0500 (+05)
        (envelope-from noreply@mcbaz.ru)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mcbaz.ru;
 q=dns/txt; s=mail; bh=t70uyBpRQIcG7qG3mmlIvmXXwHN0rhZiPS7m8mz/Gks=;
 h=from:reply-to:subject:date:mime-version:content-type:list-id:list-unsubscribe;
        b=qUFjt2sEoFA0kkK9YFc2C464VrVWSF5CvPQeLk1lfcBE28MhzbHj0kUx/oSFkHFAIKUNcBUf00lK
        E+Sm/Vu5zDgCq/O8+CksrP2TE2d5b9QRuiXt78cRvkSq3ZZxfEB76w0U1++mV37FOddOXE1NxcJz
        +s360Q1dORyFpMunhJs=
To: <osp@qwerty.perm.ru>
Subject:
=?UTF-8?Q?=D0=9E=D1=80=D0=B3=D0=B0=D0=BD=D0=B8=D0=B7=D0=B0=D1=86=D0=B8=D1=8F_=D0=B8_=D0=BF=D1=80=D0=B0=D0=B2=D0=BE=D0=B2=D0=BE=D0=B5_=D1=80=D0=B5=D0=B3=D1=83=D0=BB=D0=B8=D1=80=D0=BE=D0=B2=D0=B0=D0=BD=D0=B8=D0=B5_=D0=B0=D1=83=D0=B4=D0=B8=D1=82=D0=B0?=
From: =?UTF-8?Q?=D0=91=D0=B0=D0=B7=D0=B8=D1=81=2C=20=D1=83=D1=87=D0=B5=D0=B1=D0=BD=D1=8B=D0=B9=20=D1=86=D0=B5=D0=BD=D1=82=D1=80?=
<metodist@mcbaz.ru>
List-Id: MTQ4OTU2OS0xNDk0MTgtODc3 <MTQ4OTU2OS0xNDk0MTgtODc3.list-id.mcbaz.ru>
List-Unsubscribe: <mailto:unsubscribe@mcbaz.ru?subject=unsub-h6g34k7kv8yh3j&body=h6g34k7kv8yh3j>,<http://r.mcbaz.ru/2m9vdygyn8yh3g.html>
Content-Type: multipart/alternative; boundary="-------?=_72496-5586798571877"
MIME-Version: 1.0
Precedence: bulk
Feedback-ID: ded_153.92.250.164:1489569:1489569_920:Sendinblue
X-Mailer: Sendinblue
X-Mailin-Client: 1489569
X-Mailin-Campaign: 920
Reply-To: v2065925@yandex.ru
Message-Id: <201612220912.h6g34k7kv8yh3j@mcbaz.ru>
Date: Thu, 22 Dec 2016 09:12:05 +0100
X-Spamd-Result: default: False [12.10 / 15.00]
 BAYES_SPAM(7.50)[100.00%]
 SPAMRCPT(5.00)[osp@qwerty.perm.ru]
 ONCE_RECEIVED(0.10)[]
 DMARC_POLICY_ALLOW_WITH_FAILURES(-0.50)[]
 R_SPF_SOFTFAIL(0.00)[~all]
 PRECEDENCE_BULK(0.00)[]
 R_DKIM_ALLOW(-0.20)[mcbaz.ru]
 FORGED_SENDER(0.30)[]
 DMARC_POLICY_ALLOW(0.00)[mcbaz.ru, none]
 MIME_GOOD(-0.10)[multipart/alternative, text/plain]
X-Rspamd-Server: localhost
X-Rspamd-Scan-Time: 2.53
X-Rspamd-Queue-ID: uBM8C8E4065108

[vstakhov]

It seems that this issue is related to gmime parser. In 1.5, I have eliminated gmime completely from Rspamd and I cannot thus reproduce your issue.

[drook]

Cool, thanks, waiting for the next release then.

[vstakhov]

In fact, I've updated rspamd-devel port in FreeBSD, so you can make a try before the official release since I have no exact date for it so far.

[drook]

Thanks again, I definitely will.