Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

general purpose spam filter #1

Closed
ghost opened this issue Jul 5, 2013 · 13 comments
Closed

general purpose spam filter #1

ghost opened this issue Jul 5, 2013 · 13 comments
Labels

Comments

@ghost
Copy link

ghost commented Jul 5, 2013

As for now rspamd is used for spam filtering e-mail and html messages.
Is there any possibility to use rspamd for filtering custom plaintext messages?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@vstakhov
Copy link
Member

vstakhov commented Jul 5, 2013

Yes, any normal worker can be switched in non mime mode by adding an option off to its configuration.

@ghost
Copy link
Author

ghost commented Jul 5, 2013

I did this, but rspamc says cannot scan message: Invalid header line:[
with json on feature

@vstakhov
Copy link
Member

vstakhov commented Jul 5, 2013

Rspamc cannot work with json worker yet. But json property is unrelated to option anyway.

@ghost
Copy link
Author

ghost commented Jul 8, 2013

Ok found it in protocol.c thks.

@ghost
Copy link
Author

ghost commented Jul 8, 2013

Reading rspamd sources and thinking about how it can be used I wonder: maybe it is better to create general purpose antispam framework, and create applications on top of it? Which could be used to create spam filtering applications for mail, web based systems(comments, forums and even waf), rss/atoms feeds or any structured or unstructured messages with different fields. WDYT?

@vstakhov
Copy link
Member

vstakhov commented Jul 8, 2013

Actually I tried to follow this logic while writing rspamd. Currently, rspamd has isolated layers of mime parsing, filters, protocol handling, process management and so on. Moreover, it is possible to write custom lua workers that perform different procedures (previously, I've implemented postifx map handler for xmlrpc authorizing, for example). However, I'm unsatisfied with the current state of the configuration system which is absolutely unextendable and limited comparing to SA variance that is perl based. My target is to make a configuration system in lua that could handle all scenarios required including your specific appliances. Nevertheless, I think that librspamdserver and librspamdmime should contain all filtering, configuration and other logic and it should be possible to use a simple linking or dlopen to get access to all rspamd functions. So, which type of framework abstraction did you mean in your suggestion?

@ghost
Copy link
Author

ghost commented Jul 8, 2013

The one that would contain:
classification, learning, dns, event-core, module api,regexp, lua-core, basic filtering capabilities. Just core functions without mime processing or protocol stuff.
In my opinion mime is just a way to extract text for classification. For example mime could be replaced with OCR, to extract text from images to perform classifiction.
The only thing that bothers me is metadata that is passed with messages. In current rspamd protocol it is HEADER values, they are not expandable and strongly tighten with task struct.

@vstakhov
Copy link
Member

vstakhov commented Jul 8, 2013

Actually, rspamdmime is a separate library, however, I do agree that struct worker_task contains too much mime related primitives. Moreover, all mime parsing should be performed abstractly as the specific type of converting the input of rspamd to struct worker_task (which should still include headers, text parts and URLs). In this matter it would be possible to extend rspamd for processing of various types of inputs.

@ghost
Copy link
Author

ghost commented Jul 8, 2013

Maybe we could discuss some kind of roadmap? I think it has great potential and I`d like to help in improving it!

@vstakhov
Copy link
Member

vstakhov commented Jul 8, 2013

Actually, I have the list of tasks that should be done:

  • create a site with actual documentation (github based), rework integration part (describe rmilter and rewrite exim part), lua api (include updated API examples and references), the overall introduction (add information about launchpad packages and yum repositories);
  • rework configuration to use lua more extensively, implement automatic rules update and rsa checks (0.6 milestone);
  • improve all existing filters, such as surbl or fuzzy check to support multiply check sources (to match SA functionality);
  • write abstract statfiles framework to allow storing of statistics in, for example, redis;
  • make an abstract parsing library to decouple messages parsing from filters;
  • add new plugins, such as hashcash or batv to extend rspamd;
  • rework lua API to match modern conventions about lua C extensions.

It is a big list and some items are really extensive, but cool potentially. I personally plan to concentrate on configuration and documentation tasks, as it seems to be the main limiting issue that prevents rspamd from being more popular.

vstakhov added a commit that referenced this issue Dec 4, 2013
Fixed bug in whitelist plugin with getting from address
vstakhov pushed a commit that referenced this issue May 22, 2014
moisseev added a commit to moisseev/rspamd that referenced this issue Feb 17, 2016
2016-02-17 13:00:50 #58424(normal) <a41bdc>; task; lua_metric_symbol_callback: call to (ONCE_RECEIVED) failed: /usr/local/share/rspamd/lua/once_received.lua:82: bad argument rspamd#1 to 'lower' (string expected, got nil); trace: [1]:{[C]:-1 - lower [C]}; [2]:{/usr/local/share/rspamd/lua/once_received.lua:82 - <unknown> [Lua]};
rm-minus-rf pushed a commit to rm-minus-rf/rspamd that referenced this issue Apr 21, 2020
Improve example Exim configuration
citrin added a commit to citrin/rspamd that referenced this issue Sep 21, 2021
If request to /counters returns no symbols then max_len would have
a negative value:

Results for command: counters (0.003 seconds)
=================================================================
==22096==ERROR: AddressSanitizer: negative-size-param: (size=-2147483604)
    #0 0x33ff13 in __asan_memset (/usr/bin/rspamc+0x33ff13)
    rspamd#1 0x383432 in rspamc_counters_output /usr/src/debug/rspamd/src/client/rspamc.c:1064:2
    rspamd#2 0x388c49 in rspamc_client_cb /usr/src/debug/rspamd/src/client/rspamc.c:1600:6
    ...
c-rosenberg pushed a commit to HeinleinSupport/rspamd that referenced this issue Nov 1, 2021
If request to /counters returns no symbols then max_len would have
a negative value:

Results for command: counters (0.003 seconds)
=================================================================
==22096==ERROR: AddressSanitizer: negative-size-param: (size=-2147483604)
    #0 0x33ff13 in __asan_memset (/usr/bin/rspamc+0x33ff13)
    rspamd#1 0x383432 in rspamc_counters_output /usr/src/debug/rspamd/src/client/rspamc.c:1064:2
    rspamd#2 0x388c49 in rspamc_client_cb /usr/src/debug/rspamd/src/client/rspamc.c:1600:6
    ...
vstakhov pushed a commit that referenced this issue Dec 10, 2021
c-rosenberg pushed a commit to HeinleinSupport/rspamd that referenced this issue Feb 27, 2022
If request to /counters returns no symbols then max_len would have
a negative value:

Results for command: counters (0.003 seconds)
=================================================================
==22096==ERROR: AddressSanitizer: negative-size-param: (size=-2147483604)
    #0 0x33ff13 in __asan_memset (/usr/bin/rspamc+0x33ff13)
    rspamd#1 0x383432 in rspamc_counters_output /usr/src/debug/rspamd/src/client/rspamc.c:1064:2
    rspamd#2 0x388c49 in rspamc_client_cb /usr/src/debug/rspamd/src/client/rspamc.c:1600:6
    ...
c-rosenberg pushed a commit to HeinleinSupport/rspamd that referenced this issue Sep 21, 2022
If request to /counters returns no symbols then max_len would have
a negative value:

Results for command: counters (0.003 seconds)
=================================================================
==22096==ERROR: AddressSanitizer: negative-size-param: (size=-2147483604)
    #0 0x33ff13 in __asan_memset (/usr/bin/rspamc+0x33ff13)
    rspamd#1 0x383432 in rspamc_counters_output /usr/src/debug/rspamd/src/client/rspamc.c:1064:2
    rspamd#2 0x388c49 in rspamc_client_cb /usr/src/debug/rspamd/src/client/rspamc.c:1600:6
    ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants