Oct 16, 2018
Release 1.8.1
* [CritFix] Fix options insertion
* [CritFix] Fix words decay one more time (affects long messages)
* [CritFix] Increase default words_decay
* [CritFix] Plug memory leak in redis pool
* [Feature] Add `check_violation` feature to DKIM/ARC signing
* [Feature] Add only unique elements to Clickhouse url arrays
* [Feature] Allow `g+:` and `g-:` composite atoms
* [Feature] Allow dkim domains check in surbl
* [Feature] Allow maps with HTTP auth
* [Feature] Allow to disable actions by users settings
* [Feature] Extend whitelisting options
* [Feature] Store url object in images
* [Feature] Use verdict instead of the plain action in plugins
* [Fix] Allow to call fstring append with NULL string
* [Fix] DCC - luacheck
* [Fix] Do not load torch on each rspamadm invocation
* [Fix] Fix boundaries detection and rework stop words algorithm
* [Fix] Fix dependencies for DNS_SIGNED symbol
* [Fix] Fix errors when dealing with dynamic rates/bursts in Ratelimit
* [Fix] Fix groups mess
* [Fix] Fix groups mess
* [Fix] Fix parsing address with comments
* [Fix] Fix resolving in DMARC reports
* [Fix] Fix various issues with parsing of the received headers
* [Fix] Fix watchers issue in lua_tcp when doing no resolving
* [Fix] Plug memory leak in language detector (affects reloads)
* [Fix] Remove one letter stop words
* [Fix] Slashing: backport chunk logic from libucl
* [Fix] Stop libevent from using cached time in rspamadm
* [Fix] Try to fix watchers chaining
* [Fix] Various fixes in redis sync interface
* [Fix] ip_score - respect check_authed and check_local settings from config
* [Project] Rework passthrough actions
* [Project] Clustering module
* [Rework] Always create result for a task
* [Rework] Completely rewrite DMARC checks logic
* [Rework] Rework and fix whitelist plugin
* [WebUI] Add symbols sorting buttons
* [WebUI] Change symbols order without updating history
* [WebUI] Colorize symbols
* [WebUI] Do not display password form when secure_ip is set
* [WebUI] Fix symbol description tooltips display
* [WebUI] History: add sorting by symbol score value
Sep 24, 2018
Release 1.8.0
* [Feature] Add arguments schemas to processors and extractors
* [Feature] Add functional selectors library
* [Feature] Add generic selector to reputation module
* [Feature] Add more ratelimits: by digest, by attachments data, by filenames
* [Feature] Add preliminary stop words detection support
* [Feature] Add pure Lua debugm function
* [Feature] Add schema validation for Redis settings
* [Feature] Add selectors combine function
* [Feature] Add some recursion protection to lua logger
* [Feature] Add support for Lua API tracing
* [Feature] Allow to apply schema to arguments
* [Feature] Allow to get dkim signing data directly from HTTP headers
* [Feature] Allow to reuse existing authentication results
* [Feature] Cache selectors results in re runtime
* [Feature] Implement new text tokenizer based on libicu
* [Feature] Integrate selectors framework to multimap
* [Feature] Relax FORGED_RECIPIENTS
* [Feature] Support (almost) all html entities
* [Feature] Support adding and deletion of recipients in the milter block
* [Feature] Support gathering HTTP body from fragments in lua_http
* [Feature] Support multi flag in regexp and glob maps
* [Feature] Support selectors in ratelimit module
* [Feature] Support selectors in settings
* [Feature] Use khash in HTML parser
* [Feature] Use pure Lua debugm function
* [Fix] Add fail-safety for destroying sessions
* [Fix] Allow to add result-less fake DNS records
* [Fix] Another try to fix race conditions on config unload
* [Fix] Call Lua callback on DNS timeouts
* [Fix] Deprecate task:inc_dns_req as it is redundant
* [Fix] Do not allow events deletions on cleanup
* [Fix] Do not try to process skipped messages
* [Fix] Fix HTTP requests with no body
* [Fix] Fix another cleanup race condition
* [Fix] Fix bug in processing of pcre regexps
* [Fix] Fix byte array allocation in the pool
* [Fix] Fix crashes on task cleanup
* [Fix] Fix dynamic buckets in ratelimits
* [Fix] Fix endless loop when waiting for Rspamd to stop
* [Fix] Fix lua_util.str_split in case of delimiters set
* [Fix] Fix more issues with watching of async events
* [Fix] Fix stop words detection and loading logic
* [Fix] Fix various corner cases for language detection
* [Fix] Fix watchers in lua_tcp
* [Fix] Fix words decay algorithm
* [Fix] Implement watchers replacement to handle nested calls
* [Fix] Save faked code into fake dns record
* [Fix] Show the proper frame when using lua_util.debugm
* [Fix] Use fake dns records in tests
* [Fix] Use unicode replacements for HTML entities
* [Fix] fixed "cannot find dependency on symbol 1" issue when using replaced symbols in spamassassin rules
* [Fix] partition_id is not available in old versions of CH
* [Project] Add implicit conversion logic to selectors
* [Project] Add initial support for selectors in regexps
* [Project] Add method concept
* [Project] Further changes in unicode operations
* [Project] Implement Clickhouse migrations
* [Project] Implement implicit conversions to userdata
* [Project] Implement insert method
* [Project] Implement selectors registration for regular expressions
* [Project] Implement selectors support in re_cache
* [Project] Improve language detector: cleanup unused files, categorize
* [Project] Migrate CH data to a fat table
* [Project] Rework selectors logic
* [Project] Start Clickhouse utilities library
* [Project] Start unicode rework
* [Project] coroutine threaded model for API calls: thread pool
* [Rework] Move phishtank to a DNS based service
* [Rework] Rework Clickhouse plugin to use the new API
* [Rework] Rework language detector
* [Rework] Rework utf content processing in text parts
* [WebUI] Add progress bar for AJAX requests
* [WebUI] Avoid errors table reinitialization
* [WebUI] Avoid history table reinitialization
* [WebUI] Avoid throughput summary table reinitialization
* [WebUI] Destroy summary table on disconnect
* [WebUI] Fix "auth" request URL
* [WebUI] Fix disabling and hiding controls on page reload
* [WebUI] Fix maps loading from neighbours
* [WebUI] Fix symbols sorting by score
* [WebUI] Fix tables destroying
* [WebUI] Fix throughput data consolidation
* [WebUI] Fix upload buttons disabling
* [WebUI] Notify user on module loading failure
* [WebUI] Update FooTable 3.1.4 -> 3.1.6
Aug 1, 2018
Release 1.7.9
* [CritFix] Fix caseless comparison of equal length strings
* [Feature] Add HTTP basic auth support to elastic and clickhouse plugins
* [Feature] Add SPF selector to reputation
* [Feature] Add support of the fallback backends for HTTP maps
* [Feature] Allow to print full mime structure when extracting mime data
* [Feature] Allow to split symbols in reputation plugin
* [Feature] Check attachments only on AV scanners in attachments_only mode
* [Feature] Disable all SSL checks if ssl_no_verify flag is set
* [Feature] Implement parsing of scoped IPv6 addresses
* [Feature] Improve `rspamc counters` output
* [Fix] Add sanity checks when expanding SPF macros
* [Fix] Allow to parse SA rules with no spaces around =~ (dirty hack)
* [Fix] Avoid one extra byte writing
* [Fix] Deal with direct hash table
* [Fix] Detect empty text part as text, not HTML
* [Fix] Do not reduce map watch timeout for mixed http/file maps
* [Fix] Fix HTML part detection heuristic
* [Fix] Fix double free in redirectors cleanup
* [Fix] Fix legacy history handling in the controller
* [Fix] Fix messages insertion
* [Fix] Fix sending string method
* [Fix] Fix statconver command line arguments
* [Fix] Fixed argument checking for being null
* [Fix] Fixed issues reported by luacheck
* [Fix] Freeze updates queue when do actual storage update
* [Fix] HTTP map hash is per-backend and not per-map
* [Fix] Plug memory leak in fuzzy updates
* [Fix] Prefer 'MTA-Name' when producing authentication results
* [Fix] Replace bad unicode sequences instead of stopping on them
* [Fix] Set classifier version on learning
* [Project] Reworked ratelimits
* [Project] Apply topological sorting for symbols in Rspamd
* [Project] Remove global contexts from C modules
* [Project] Move performance critical hash tables to khash
* [WebUI] Avoid unused indexes
* [WebUI] Do not execute `on_success` callback
* [WebUI] Fix history reset for "All SERVERS" (#2346)
* [WebUI] Fix query URL for selected server
* [WebUI] Fix symbols display in legacy history,
* [WebUI] Hide symbols order selector for legacy history
* [WebUI] Refactor query functions into one
* [WebUI] Remove previously-attached event handlers
* [WebUI] Save symbols to the selected server
* [WebUI] Unify arguments of query functions
* [WebUI] Use common query functions to get graph data
* [WebUI] Use common query functions to save symbols
Jul 12, 2018
Release 1.7.8
* [Feature] Add more extended statistics about fuzzy updates
* [Feature] Add more non-conformant Received headers support
* [Feature] Add preliminary function to get fuzzy hashes from text in Lua
* [Feature] Allow to configure AV module rejection message
* [Feature] Implement fuzzy hashes extraction in mime tool
* [Feature] Improve WHITE_ON_WHITE rule
* [Feature] Improve integer -> string conversion
* [Feature] Reuse maps in multimap module more aggressively
* [Fix] Avoid race condition in skip map as pool lifetime is not enough
* [Fix] Eliminate all specific C plugins pools
* [Fix] Fix DKIM check rule if DNS is unavailable
* [Fix] Fix build where ucontext is defined in ucontext.h
* [Fix] Fix crash in base url handling
* [Fix] Fix descriptors leak in sqlite3 locking code
* [Fix] Fix messages quarantine
* [Fix] Fix padded numbers printing
* [Fix] Fix race condition on maps reinit
* [Fix] Fix regexp functions when no data is passed
* [Fix] Fix specific urls extraction
* [Fix] Fix styles propagation
* [Fix] Improve resetting of the limit buckets
* [Fix] Initialize sqlite3 properly
* [Fix] Work with broken resolvers in resolv.conf
* [Project] Implement HTTP maps caching
* [Project] Refresh fuzzy hashes when matched
* [Project] Add logic to deduplicate fuzzy updates queue
* [WebUI] Add missed declarations
* [WebUI] Avoid using "undefined" property
* [WebUI] Do not accept passwords containing control characters
* [WebUI] Do not redeclare variables
* [WebUI] Enable strict mode,
* [WebUI] Fix variable assignment
* [WebUI] Initialize variables at declaration
* [WebUI] Remove duplicated path from RequireJS config
* [WebUI] Remove unused block
* [WebUI] Remove unused variable
* [WebUI] Remove unused variables
* [WebUI] Use self-explanatory notation
* [WebUI] Use type-safe equality operators
Jul 2, 2018
Release 1.7.7
* [CritFix] Check NM part of pubkey to match it with rotating keypairs
* [CritFix] Do not overwrite PID of the main process
* [CritFix] Fix maps after reload
* [CritFix] Fix maps race conditions on reload
* [CritFix] Fix shmem leak in encrypting proxy mode
* [Feature] Add a concept of ignored symbols to avoid race conditions
* [Feature] Add ability to print bayes tokens in rspamadm mime
* [Feature] Add method to get statistical tokens in Lua API
* [Feature] Add preliminary mime stat command
* [Feature] Add rspamadm mime tool
* [Feature] Add urls extraction tool
* [Feature] Address ZeroFont exploit
* [Feature] Allow rspamadm mime to process multiple files
* [Feature] Allow to extract words in `rspamadm mime`
* [Feature] Allow to print mime part data
* [Feature] Allow to show HTML structure on extraction
* [Feature] Distinguish IP failures from connection failures
* [Feature] Improve output for mime command
* [Feature] Improve styles propagation
* [Feature] Main process crash will now cleanup all children
* [Feature] Preload file and static maps in main process
* [Feature] Print stack trace on crash
* [Feature] Process font size in HTML parser
* [Feature] Propagate content length of invisible tags
* [Feature] Read ordinary file maps in chunks to be more safe on rewrites
* [Feature] Support base tag in HTML
* [Feature] Support more size suffixes when parsing HTML styles
* [Feature] Support opacity style
* [Fix] Another fix for nested composites
* [Fix] Fill nm id in keypairs cache code
* [Fix] Fix colors alpha channel handling
* [Fix] Fix destruction logic
* [Fix] Fix double free
* [Fix] Fix maps preload logic
* [Fix] Fix nested composites process
* [Fix] Fix proxying of Exim connections
* [Fix] Fix reload crash
* [Fix] Fix rspamadm -l command
* [Fix] Update ed25519 signing schema
* [WebUI] Stop using "const" declaration
* [WebUI] Update RequireJS to 2.3.5
Jun 15, 2018
Release 1.7.6
* [CritFix] Fix multiple neural networks support
* [Feature] Add decryption function to keypair command
* [Feature] Add gzip compression for HTTP requests in elastic module
* [Feature] Add gzip methods to lua util
* [Feature] Add maps based on Top Level Domains
* [Feature] Add pubkey checks for dkim_signing
* [Feature] Add support of fake DNS records
* [Feature] Add tool to encrypt files
* [Feature] Allow to add symbols using settings directly
* [Feature] Allow to match private and public keys for DKIM signatures
* [Feature] Allow to set task flags via settings
* [Feature] Allow to specify fake DNS address from the config
* [Feature] Implement signatures verification using rspamadm keypair
* [Feature] Implement signing using `rspamadm keypair`
* [Feature] Improve error reporting for DKIM key access issues
* [Feature] Provide $HOSTNAME variable in UCL
* [Feature] Rework levenshtein distance computation
* [Feature] Split message parsing and processing
* [Feature] Support ED25519 DKIM signatures
* [Feature] Support encrypted configs in UCL
* [Feature] Suppress duplicate warning on very large radix tries
* [Feature] Use OSB to combine header names
* [Fix] Cleanup maps data on shutdown
* [Fix] Fix '~' behaviour in composites
* [Fix] Fix HTTP maps updates
* [Fix] Fix NIST signatures
* [Fix] Fix RFC822 comments when processing a mime address
* [Fix] Fix double free
* [Fix] Fix dynamic settings application
* [Fix] Fix for CommuniGate Pro maillist
* [Fix] Fix keypair creation method to actually create keypair...
* [Fix] Fix matching patterns with no paths
* [Fix] Fix memory leak in parsing comments
* [Fix] Fix parsing of urls with numeric password
* [Fix] Fix plugins intialisation in configwizard
* [Fix] Fix potential crash on reload
* [Fix] Fix potential race condition for a finished HTTP connections
* [Fix] Fix race-condition leak on processes reload
* [Fix] Fix signing in openssl mode
* [Fix] Free language detector structures
* [Fix] Relax alignment requirements
* [Fix] Send DMARC reports compressed
* [Fix] Try to fix leak in dmarc module
* [Fix] Try to plug memory leak in metric exporter
* [Project] Convert rspamadm subcommands to Lua
* [WebUI] Display smtp sender/recipient in history
* [WebUI] Fix elements disabling in "Symbols" tab
* [WebUI] Limit recipients list in history column to 3
* [WebUI] Match envelope and mime addresses following in arbitrary order
* [WebUI] Update column header
* [WebUI] Wrap addresses in history
May 18, 2018
Release 1.7.5
* [Conf] Add MSBL proposed return codes
* [Conf] Add additional groups for policies
* [CritFix] Do not use volatile Lua strings as UCL keys
* [Feature] Add ability to add fuzzy hashes to headers
* [Feature] Add function to extract most meaningful urls
* [Feature] Add rule to block mixed text and encrypted parts
* [Feature] Allow multiple groups for symbols
* [Feature] Allow to disable lua squeezing logic
* [Feature] Allow to get multipart children in Lua
* [Feature] Allow to insert multiple headers from milter headers
* [Feature] Allow to print scores in subject and further extensions
* [Feature] Be more error-prone in squeezed rules
* [Feature] Support multiple return codes in emails module
* [Feature] Use EMA for calculating averages
* [Feature] Use common jit cache for all regexps
* [Feature] support for CommuniGate Pro self-generated messages
* [Fix] Allow to have multiple values for headers as arrays
* [Fix] Do not open sockets for disabled workers
* [Fix] Fix AuthservId
* [Fix] Fix base64 folding in Lua API
* [Fix] Fix build on non-x86 platforms
* [Fix] Fix cached maps logic
* [Fix] Fix compatibility with old maps query logic
* [Fix] Fix crash if skip_map is used
* [Fix] Fix importing static maps from UCL
* [Fix] Fix parsing of unix sockets
* [Fix] Fix raw_mime regexp on HTML part with no text content
* [Fix] Fix tables logging
* [Fix] Fix vertical tab handling in libucl
* [Fix] Try to fix frequency counters
* [Fix] Use better sharding for ip_score
* [Fix] Use multiple results from SURBL DNS reply
* [Fix] When doing AV scan select a different server for retransmit
May 1, 2018
Release 1.7.4
* [Conf] Major stock config updates:
    - Workers are now specified in a new format worker "type" { ... }
    - Enable fuzzy worker to simplify local fuzzy storages configuration
    - Bind all workers to localhost by default to avoid security flaws
* [Conf] Make more sane fuzzy_check default settings
* [CritFix] Fix ucl escape for bad symbols
* [Feature] Add failure symbol for AV module
* [Feature] Add lazy expiration mode for new classifier schema
* [Feature] Add preliminary version of maps stats plugin
* [Feature] Allow to block fuzzy requests from specific networks
* [Feature] Allow to change `expire` of live statistics
* [Feature] Distinguish AV failure from clean result
* [Feature] Further improvements of language detector by using khash
* [Feature] Further optimization of the lang_detection
* [Feature] Implement cluster-aware bayes expiry
* [Feature] Implement exclude patterns in rspamc
* [Feature] Implement glob maps in addition to regexp maps
* [Feature] Implement map statistics function for lua API
* [Feature] Implement stop symbols for Clickhouse collection
* [Feature] Support recipients separated by commas
* [Feature] Try harder to upload scripts to the Redis server
* [Feature] Upgrade t1ha distribution
* [Feature] use_domain_sign_inbound
* [Feature] Use scores from maps if `symbols_set` is not defined
* [Fix] Add resolving version of radix map helper
* [Fix] Check URL before adding implicit prefix
* [Fix] Do not check pid/state when using PRNG
* [Fix] Fix CentOS logrotate script for systemd
* [Fix] Fix slash + dot in urls
* [Fix] Fix systemd version of the logrotate script
* [Fix] Propagate key when import implicit array from Lua
* [Fix] Strip spaces from map keys and values
* [Fix] Try to fix a specific case when processing milter protocol
* [Fix] Try to fix crash when a tcp connection cannot be set
* [Fix] Typo use_domain_local --> use_domain_sign_local
* [Fix] Various fixes to once_received module
* [Project] Store hits counters for map elements
Apr 10, 2018
Release 1.7.3
* [CritFix] Plug bad memory leak in protocol reply
* [Feature] Add avx2 codec for base64
* [Feature] Add method to receive all URL flags from Lua API
* [Feature] Allow to fold headers on stop characters
* [Feature] Allow to set lua_cpath from options
* [Feature] Allow to specify custom rejection message in milter
* [Feature] Deal with unnormalised Unicode obfuscation
* [Feature] Do not detect language twice for relative parts
* [Feature] Implement oversigning feature
* [Feature] Implement silent logging level to minimize noise in logs
* [Feature] Improve URL_IN_SUBJECT rule
* [Feature] Use hashing to reduce redis attack surface
* [Fix] Add oversigning for the most important headers
* [Fix] add 'rewrite subject' to History dropdown
* [Fix] Another fix in folding algorithm
* [Fix] Do not call multimap addr for parts of addr if filter is presented
* [Fix] Do not clean hostname on generic reset
* [Fix] Do not create pid file in no-fork mode
* [Fix] Fix fold_after case to preserve multiple spaces
* [Fix] Fix folding and folding tests
* [Fix] Fix hostname usage in milter mode
* [Fix] Fix lua RSA verify and its tests
* [Fix] Fix metadata exporter send_mail backend (#2124)
* [Fix] Fix processing of '\v' in libucl
* [Fix] Fix shemaless URLs detection
* [Fix] Fix support of multiple headers in sign_header
* [Fix] Fix usage of util.parse_mail_address
* [Fix] Fix weights of dynamic squeezed rules
* [Fix] Leak from bucket before checking the burst
* [Fix] Stop using own localtime as DST could be messy in many cases
* [Fix] Treat unnormalised URLs as obscured
* [Rework] Restore leaky bucket model in ratelimit plugin
* [WebUI] Add messages total to throughput summary
* [WebUI] Add symbols order selector to history
* [WebUI] Config: Load list on demand
* [WebUI] Fix modalBody for maps that appear more than once
* [WebUI] History: Fix Tooltips on paging, filtering and sorting
* [WebUI] Remove a previously-attached event handler
* [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1
Mar 23, 2018
Release 1.7.2
* [Feature] Store emails in Clickhouse
* [Feature] Support single quotes in config
* [Feature] Use templates when publishing CH schema
* [Feature] Improve Docker image
* [Fix] Add rounding when printing a lot of FP variables
* [Fix] Allow to disable certain actions by assigning null to them
* [Fix] Disable results caching
* [Fix] Fix disabling of squeezed symbols
* [Fix] Fix scan time set
* [Fix] Rework logic of actions setting
* [Fix] Try to fix various Lua stack issues
* [WebUI] Add link tag for favicon.ico
* [WebUI] Display hostname:port/path in the page title