Gromox 3.10

Enhancements:

• midb now synchronizes IMAP message flags to and from the mailbox so they survive a deletion of midb.sqlite3.
• midb/imapd now supports IMAP keywords.
• EWS now supports the Outlook-for-Mac smart response draft types (ReplyToItem/ReplyAllToItem/ForwardItem).
• EWS: the FindPeople and GetUserPhoto commands now also search for matches in in personal contacts.
• pff2mt: It is now possible to place a private mailbox dump (.pst) into a public folder. See behavioral changes too.
• mysql_adaptor: A local object cache (TTL 1 minute) was added for recurrent user ID↔name lookups and group membership queries.

Fixes:

• zcore no longer logs the session ... not found anymore message.
• zcore now scans mlist memberships when doing delegate permission checks.
• ruleproc no longer enters single-occurrence updates into user calendars.
• ruleproc now releases a resource's original slot when an occurrence moves.
• ruleproc now drops stale per-occurrence items when the series is re-issued.
• ruleproc: Rule conditions like "is received from" can use multiple internal representations, but only now an equivalence match was added.
• oxcical: Work around broken PidLidGlobalObjectId field values and treat it like no GOID was present to begin with (cures the E-2215 log message).
• exmdb_client: More resilient resubscription after network outages.
• EWS: Restore a (previously dropped) workaround so 32-bit Outlook no longer crashes when showing an OOF dialog that is all greyed out.
• exmdb: Re-added an index for PR_CONVERSATION_ID to speed up conversation-based viewing in OL/g-web.
• exmdb: Fixed a case where a search folder is repopulated over and over again.
• exmdb: Reading a MAPI content table while another thread is trying to close it could led to a SELECT count(idx) FROM t656: SQL logic error log message, which was resolved.
• emsmdb: fix a case of notifications getting stuck in out of memory situations.
• ldap_adaptor: LDAP connections that have timed out at the TCP level are now recognized and reconnected.
• oxcmail: when converting from Internet Mail to MAPI, a HTML body is now correctly detected as the HTML body even if some attachments precede it.

Behavior changes:

• pff2mt, kdb2mt: The root folder from PFF files and KDB mailboxes is now emitted as "unanchored" in the GXMT stream (previously: anchored to IPM_SUBTREE) and is now subject to re-anchoring (gromox-import -B).

Gromox 3.9

Enhancements:

• exmdb: The link_messages EXRPC was sped up, which is used by g-web's indexed search to fill the results folder.
• The meeting processor now sends booking confirmation for auto-accepted requests, now handles incoming cancellations, and permits back-to-back room bookings (previously it wanted a gap of at least 1 second).
• EWS: meetings are now sent with a fair bit more meeting properties, making them actionable meetings rather than just plain appointments.
• EWS: support for some more tags and request types that previously had led to warning messages in the log.
• EWS now sends meeting cancellations when deleting a whole item.

Fixes:

• Synthesize the PR_RTF_IN_SYNC MAPI property, fixing a case where Outlook could show HTML messages (imported from older Exchange/Kopano) as plaintext.
• Fixed a crash when the outgoing_smtp_url directive was used with a smtp+tls: target.
• Fixed a crash when gromox-delivery processed a meeting invitation from a local organizer.
• Fixed a crash when gromox-imap sent keyword declarations to idle sessions.
• The hierarchy synchronization procedure for OL no longer emits out-of-spec error codes for duplicate folders.
• Messages without mandatory ICS properties are no longer advertised to OL Cached Mode for download.
• ruleproc: honor PidLidIsSilent when sending automated meeting responses
• freebusy: skip deleted instances when expanding recurrences
• exmdb_local: suppress OOF replies when not directly addressed
• mbop: make purge-datafiles respect exchange.sqlite3:messages:mid_string
• daemons: a hang on startup when logfile cannot be opened was resolved

Behavior changes:

• Sooner teardown of idle SQL connections to relieve the mariadb/mysql server.
• Events in search folders are no longer propagated to other search folders for performance reasons (since search folder stacking seems to be exceedingly rare).

Gromox 3.8

Enhancements:

• The Information Store now supports a per-store split-process model.
  The gromox.cfg:istore_standalone config directive has turned into a bitmask.
• emsmdb(Outlook): fnevObjectModified notifications for folders now convey the new counts of read/unread messages.
• Support for custom per-message keywords (a.k.a. color categories) in IMAP
• Partial IMAP4rev2 support (ENABLE, NAMESPACE, SELECT/EXAMINE, STATUS SIZE/DELETED, ESEARCH, (UID) MOVE, LIST-EXTENDED, SEARCHRES)
• Conversions between HTML and RTF now support and prefer the use of the external converter Pandoc, if installed.
• Conversions from HTML to plaintext now support the use of Pandoc as a fallback if chawan/w3m is not present.
• mbop: new "exaddrxlat" command
• ruleproc: implemented OP_FORWARD handling for TwoStep processor
• exmdb, zcore: new link_message RPCs for bulk linking, speeds up external search folder filling agents (e.g. grommunio-web)

Fixes:

• ab_tree: restore/implement time-based purging of the cache
• imap: fixed BODY[HEADER]/[TEXT]/[HEADER.FIELDS] returning NIL
• Server side rule and meeting processor: On reception of meeting updates, the prior calendar entry is replaced (rather than left untouched, which caused duplicates).
• ruleproc now ignores stale meeting updates (e.g. if an older update was held up in SMTP queues and was delivered late past another)
• imap: searching by Bcc field is now supported
• imap: STORE commands no longer fail when unrecognized flags are used
• midb: the SENTBEFORE/SENTON/SENTSINCE imap search keywords now test against the Date header rather than PR_LAST_MODIFICATION_TIME
• midb: the SENT* keywords now disregard timezone as mandated
• imap: FETCH/STORE/SEARCH no longer produce EXPUNGE replies
• ews: no longer loses the attendee list of an appointment when saved

Behavioral changes:

• The default value for exmdb_provider.cfg:cache_interval has been reduced from 15min to 1min, closing sqlite handles earlier when they are idle.
• exmdb_client: back off exponentially on notify reconnect
• ews: the SyncFolderItems and GetItem response sizes are now capped for reasons of memory use (clients should paginate as needed)
• ews: a limit is imposed on the streaming event backlog size now (clients should reconnect/resync as needed if they get kicked)

Gromox 3.7

Fixes:

• Outlook no longer loses appointsments when a shared calendar is opened
• The "percentage complete" of an appointment was mangled when passing through EWS, which has been fixed. (EWS uses [0, 100], but MAPI uses [0, 1.0])
• EWS now synthesizes a value for the "Received Time" field.
• AutoDiscover no longer erroneously advertises OWA API support. This had made it impossible for Thunderbird to connect.
• EWS: Setting the read flag on a message no longer generates the E-2901/E-3418 error message on the server.
• The EWS implementation now performs retrieval/update of Out Of Office settings such that it works in multi-server environments.
• The use of Qmail as an outgoing SMTP server is now supported by the message sending routines.
• Fix a case of Outlook notifications getting potentially suppressed.
• Fix a case of sqlite_busy on /var/lib/gromox/user/xyz: held by unknown log message appearing in the exmdb daemon (gromox-http/istore).
• Conversion from MAPI to iCal appoinments used the wrong UID for exceptions in a recurring series, which has been fixed.
• Reading iCal timezone definitions mangled the daylightbias, which was fixed.
• Logins with Outlook 2010 now work when Gromox is running with OpenSSL 3. (OpenSSL 1.1 & LibreSSL were fine, but OSSL3 changed behavior.)
• SMIME messages signed multiple times are unwrapped recursively and will now show the body. (Applies to just g-web; OL worked previously.)

Enhancements:

• Added conversion routines to zcore so that g-web can export GAL items to vCard

Gromox 3.6

Enhancements:

• oab: Gromox now serves Offline Addressbook files
• ews: implement task creation, update, and property retrieval
• ews: implement ExpandDL, GetPersona, CreateUserConfiguration, UpdateUserConfiguration, DeleteUserConfiguration, DeleteAttachment, AddDelegate, UpdateDelegate and RemoveDelegate operations
• ews: implement calendar occurrence operations (create, update, delete) with timezone support
• ews: implement lifecycle for meetings: invitation sending on UpdateItem, cancellations on DeleteItem, and responses to organizer on accept/decline
• ews: generate read receipts when marking messages as read, honor SuppressReadReceipts on UpdateItem
• ews: add partial matching to ResolveNames via ab_tree
• ews: add delegate permission level support and cross-mailbox permission checks
• ews: add Flag write conversion, ConversationIndex/ConversationTopic write, and attendee field handling for UpdateItem
• ruleproc: update calendar on incoming meeting response, setting PR_RECIPIENT_TRACKSTATUS on the matching attendee
• oxdisco: support for impersonation username syntax with EAS
• oxcical: treat zero-length PidLidAppointmentTimeZoneDefinitionStartDisplay as if it were absent
• eml2mt: output GXMT data as soon as practical
• htmltotext: support and prefer Chawan as a new external htmltotext renderer
• rtftohtml: support a few more RTF commands

Fixes:

• oxcmail: resolve a case of bad_function_call getting thrown
• midb: retrieve midb_sqlite_busy_timeout from the right config file, avert a "config key .. has no default and was not set either" log message
• exmdb: close cursors before modifying tables in purge
• exmdb: fix use-after-free when exmdb_provider utilizes exmdb_client in multiserver
• exmdb: proper symmetric unlocking for db_base_rd_ptr class
• exmdb: unlock giant_lock before erasing the same from hash table
• midb: release lock and reference when me_sync_mailbox experiences a sync failure
• lib: avoid emitting colors to stderr when that is not a tty
• exmdb: strip redundant notifications from notify_new_mail()
• exmdb: stop returning garbage in MAPI content table cells when the table sort order contains a PT_MV_UNICODE | MV_INSTANCE column.
• oxcical: on import, do set the PidLidAppointmentStateFlags, PidLidResponseStatus, PidLidMeetingType properties
• ews: Meeting requests that created with the EWS SendOnlyToAll flag had the wrong iCal METHOD and lacked the attendees, which has been fixed.

Changes:

• exmdb: read multi-server map exclusively from SQL rather than exmdb_list.txt
• oxcmail: on INET-to-MAPI conversion, set senders/receiver properties to use SMTP rather than EX addressing
• imap: remodeled midb_agent to require a less memory in practice
• ews: strip all characters that are illegal in XML when e.g. sending the bodytext to the client

Gromox 3.5

Enhancements:

• Finer-grained control over listening socket creation, including new config directives (http_listen, imap_listen_tls, etc.) for specifying these. This makes it possible not having to use the wildcard address.
• importer: make -B option usable with public stores

Fixes:

• emsmdb: async notification connections were not marked active and would not deliver a signal of new pending events.
• emsmdb: repair EcDoAsyncWaitEx to immediately return when notifications are already pending in the session queue
• emsmdb: fix UAF/crash when notifications are sent over RPCH
• oxcical: upon reception of non-recurring appointments, the PidLidRecurring=0 property will be set now, as some Outlook versions fail to show appointments in the daily/weekly/monthly view if the property is absent.
• oxcmail: messages with timezones ±08xx/09xx were mistreated as UTC
• oxvcard: fixed a field shift in the ADR line's parsing and emission
• ews: trim all C0 control codes from XML responses
• exmdb: limit production of PR_RTF_COMPRESSED variants of PR_BODY/PR_HTML when saving messages to IPM.Task objects, as OL only depends on it for those message classes.
• Plugged a memory leak that occurred when a multi-HTML Internet Mail is converted to MAPI
• Plugged a memory leak that occurred when HTML is converted to RTF
• Plugged a logical memleak stemming from never shrinking the buffer of FETCH RFC822 and related commands

Changes:

• php_mapi can now tolerate the presence of the PHP opcache module, though opcache must still be disabled due to miscompilation.

Gromox 3.4

Enhancements:

• mbop: new "zaddrxlat" command
• delivery: add pre-delivery junk shelving and lda_junk_rules config directive
• exmdb_local: replace direct disk I/O with imapfile_read/write EXRPCs

Fixes:

• mrautoproc: server-side processed meeting requests automatically entered into the calendar had lacked the flags asfMeeting+asfReceived, which was fixed.
• nsp: seeking backwards with the seekEntries routine jumped too far backwards, causing the last screenful of the Outlook GAL to be misrepresented.
• oxcmail: the "Keywords:" RFC5322 header (Categories) used to get filled with text garbage, which is now fixed.
• zcore: plug a memory leak occuring when importing vCards
• Any data, when converted from windows-1255 or 1258 character set to Unicode, sometimes lost the last character in the conversion, which has been fixed.
• exmdb: abort purge-datafiles if there is a database error midway
• imap: release potentially-large APPEND buffers much earlier
• exporter: plug two memory leaks

Changes:

• oxcmail: limit nesting depth of attachments during export to 7
• oxcmail: RFC 5322 header fields are now treated as US-ASCII as mandated, and no longer magically assumed to be in the same charset as the body.
• exporter: FAI messages are no longer emitted by default and explicitly need to be requested with the -a option.
• ruleproc: treat not only "Busy" as a collision, but also "Tentative" and "Out-Of-Office"
• ruleproc: evaluate not just PR_START_DATE but also PidLidAppointmentStartWhole (and their end counterparts)
• ab_tree: PR_COMPANY_NAME is no longer synthesized from the title of the domain a user belongs to
• daemons: deleted the oxcical_allday_ymd config directive

Gromox 3.3

Enhancements:

• oxvcard: include photo when converting MAPI contact objects to VCARD
• exporter: support export of multiple messages to GXMT
• exporter: support for folder export to GXMT
• importer: support a read-only mode
• exmdb: include RTF in the cross-body format synchronization when the message is saved, making Outlook Notes or Outlook Contact notes display something in grommunio-web and other clients
• exmdb: new read_delegate/write_delegate EXRPCs so that delegates.txt is no longer accessed via filesystem
• exch: allow user sending mail with From: line set to own aliases
• mysql_adaptor: regard aliases for user_ids and displayname
• midb: log when incoming connection count has been reached
• ews: recognize elements located in the SOAP header

Fixes:

• ews: avoid sending newly created message items when message invitation flag is set
• zcore: obtain freebusy information for own mailbox using owner mode, so that setting e.g. the "default" ACL default for one's calender does not deprive the user of his own freebusy view.
• exmdb: the event volley when the last row of a MAPI table with categories got deleted had wrong event data
• nsp: results from nspiGetMatches were not always capped as requested by clients (potentially crashing OL)
• zcore: make openstream treat MAPI_BEST_ACCESS as documented
• emsmdb: calls over RPCH with large outputs would sometimes just report an error due to insufficient buffer sizes, which has been fixed.
• mapi_lib: repair botched html_to_rtf with non-UTF-8 HTML input
• rtf_to_html and html_to_plain misconverted U+007F,U+07FF,U+7FFF,U+7FFFF to faulty UTF-8, which has been fixed
• mbsize: NTS error rate computation had an unsigned underflow and was misreported

Changes:

• tools: renamed gromox-{mt2exm,exm2eml} to gromox-{import,export} and added the old aliases for the tools
• nsp: make resolvenames skip over empty strings in line with the specification

Gromox 3.2

Enhancements:

• ews: implement GetRoomLists and GetRooms handlers
• ews: implement tItemAttachment, tUserConfigurationName types
• ews: implement types related to CategoryList
• ews: load ReplyTo recipients
• ews: map item flag/reminder and message sender fields
• istore: standalone exmdb_provider process launcher (splits the gromox-http process into two for better debuggability)
• oxcmail: on export (MAPI-to-IM), recognize EX-in-ONEOFF and EMSAB (GAB) entryids in the Reply-To field and substitute them accordingly
• kdb2mt: entries in the user map file are now allowed to lack the sv and id fields
• kdb2mt: entryids and search keys for senders/recipients are now translated with the help of user maps

Fixes:

• mt2exm: avoid generating zero-sized /var/lib/gromox/user/*/eml/* files (and thus bogus imapstructure ext files) in the internal mailbox directory when a message object to import has no RFC5322 representation
• midb: ignore incomplete imapstructure files in /var/lib/gromox/user/**/ext
• delivery: stop treating non-existing users as a temporary condition
• oxcmail: ONEOFF_ENTRYIDs did not have their email address/name set properly after some refactoring in 3.0, which was fixed
• ab_tree: users with HIDE_FROM_GAL or HIDE_FROM_AL were not hidden in all cases, which has been rectified
• imap: suppress AUTH=LOGIN advertisement before TLS established and instead emit LOGINDISABLED "capability" in accordance with RFC 2595
• ews: avoid dereference of unenganged std::optional, which had led to spurious use-after-free/crash
• ews: include PR_CONTAINER_CLASS in response when obtaining folder properties
• ews: add missing locking for concurrent EWSPlugin::unsubscribe invocation
• ews: avoid sending out emClient's draft messages
• emsmdb: ropOpenStream with MAPI_CREATE did not truncate the result to the right length, which was fixed
• html2rtf: map some CSS font-size keywords to sensible point sizes for RTF
• ab_tree: rework our Minimal EntryID encoding to avoid values >= 0x80000000, solving a hypothetical problem reading the addressbook data for the 589821th domain in the user database
• exmdb: MAPI table row deletion events were not being delivered, which was fixed
• oxvcard: fix crash when importing/exporting PR_HOBBIES property

Changes:

• imap: the combination of the config directive imap_force_tls=yes with imap_support_tls=no is now rejected on startup
• delivery: the OOF autoresponder was rewritten so it does not use direct filesystem access anymore (since the mailbox might not be present on the same host where the LDA runs)

Gromox 3.1

Enhancements:

• http: support for SPNEGO authentication (Kerberos-in-SPNEGO or NTLMSSP-in-SPNEGO)
  with the HTTP "Authorization: Negotiate" header.
• dscli: try all oxdisco URLs until one succeeds
• exmdb: support repeated import of permission data (e.g. from kdb2mt)
• ews: create calendar item after accepting a MR with MacMail

Fixes:

• emsmdb: the total mail count in a contents view was not updated
• emsmdb,zcore: Send-As mail now correctly has the delegator in Envelope-From
• email_lib: deal with MIME parts with zero header lines
• tools: reinstate submit.php for delayed sending
• nsp: avoid buffer overruns in nsp_interface_fetch_property
• emsmdb: Partial message change tracking was bug-ridden and deleted. Standard transfers are now used instead. (E.g. an IPM.Task object where only the percentage-completed field was changed would be mis-synchronized to another Cached Mode client as "delete start/end dates".)
• exmdb: Public folders were missing timeindex queries and their content tables might have shown fewer messages.
• http: A workaround was added for OpenSSL 3.0 so that connections from Outlook 2010 once again succeed.
• mysql_adaptor: Improve the time needed to compute the composite mailbox permission for user in a case of a mailbox with 100K ACL entries.

Changes:

• exmdb: deactivate movecopy/deletemessages event storm compaction
• emsmdb: outgoing lzxpress compression in the EMSMDB protocol is now disabled as it does not compress well for the time invested.
• http: the ntlm_program_helper config directive was removed; your gss_program simply needs to handle both GSS and NTLM.
• event: support for reading the old event_acl.txt was deleted. The replacement is the event_hosts_allow config directive.
• timer: support for reading the old timer_acl.txt was deleted. The replacement is the timer_hosts_allow config directive.