Skip to content

Releases: mjl-/mox

v0.0.13

06 Nov 23:11
v0.0.13
68c130f
Compare
Choose a tag to compare

This release fixes TLS interoperability with incoming deliveries from Microsoft
servers by disabling TLS session tickets. If you have MTA-STS and/or DANE
enabled, TLS is required for successful delivery, and updating to v0.0.13 is
required to receive messages from Microsoft again. TLS session tickets may be
enabled again in a future release, possibly per port/service.

Improvements

  • In the IMAP server, for the "bodystructure" response item to a "FETCH"
    command, add the content-type parameters for multiparts so IMAP clients will
    get the MIME boundary without having to parse the message themselves. (issue
    #217, 8fa197b)
  • Add an HTTP handler for the acme http-01 validiation mechanism to all plain
    http (non-tls) webservers (ports), not only to the one listening on port 80.
    (#issue 218, 0fbf241)
  • Properly link to matrix room so users can find it. (issue #226, 76f7b9e)

Bug fixes

  • Disable session tickets for tls to workaround deliverability issues with
    incoming email from Microsoft over smtp with starttls. Without this fix,
    email from Microsoft is no long coming in. (issue #237, 22c8911)
  • In the SMTP server, when logging about problems with recipients, actually
    show which recipients were present in the session. (issue #232,
    598c5ea).
  • Webmail: During "send and archive", don't fail with error message when
    message that is being responded to is already in archive folder. (issue #233,
    879477a)
  • Webmail: if we don't have loaded account settings yet, abort loading the
    popup after showing an error that the settings aren't available yet. (issue
    #218, 0430572)

Update instructions

Before upgrading, do a dry-run first.

  • Make a temporary backup with the old mox version:
    mox-v0.0.12 backup data/tmp/testupgrade
  • Verify that all is well with the old version:
    mox-v0.0.12 verifydata data/tmp/testupgrade
  • Verify the state with the new version:
    mox-v0.0.13 verifydata data/tmp/testupgrade

With a successful dry-run, the upgrade should go smoothly. Make a new backup
with mox-v0.0.12 backup data/tmp/backup (the previous backup was modified by
the dry-run, so couldn't be used to restore!), replace the binary and restart.
For further details, see
https://www.xmox.nl/faq/#hdr-how-do-i-upgrade-my-mox-installation

If you run into any problems, please create a bug report.

Thanks

Thanks for all the contributions/bug reports/feedback/discussions, much
appreciated! Special thanks to mdavids, danieleggert, startup-001-steve,
snabb, mattfbacon, mgkirs, exander77.

Development on mox is funded through the NLnet NGI0 Entrust Fund,
https://nlnet.nl/entrust/, with financial support from the European
Commission's Next Generation Internet programme.

Downloading & compiling

See https://www.xmox.nl/install/#hdr-download.

v0.0.12

06 Oct 12:11
v0.0.12
354b9f4
Compare
Choose a tag to compare

Improvements

  • webmail: Change many inline styles to using css classes, and add dark mode.
    (#163 by mattfbacon, a16c086)
  • webmail: In compose window, merge close & cancel button, and align buttons on
    the right. (4d28a02)
  • mox backup: Add hint about systemd ReadWritePaths if hardlinking fails on
    linux due to cross-device link. (#170 by rdelaage, 44a6927)
  • mox backup: Clarify behaviour with destination directory, and exit code.
    (#172 by RobSlgm, 1fc8f16)
  • When removing account, remove its data directory instead of leaving it
    around. (#162 by RobSlgm & x8x, 30ac690)
  • Give more helpful pointers for dns-related setup, such as troubleshooting dns
    resolving, and multi-line dkim dns records. (#158, #164, vipas84, RobSlgm,
    83004bb)
  • Improve http request handling for internal services (web interfaces) and
    multiple domains. The handler for /admin/ is now only enabled on the listener
    (machine) host name by default, no longer all hosted domains. The internal
    handlers (for admin, account, webmail, webapi), can now also be explicitly
    configured in the webserver section, for additional/custom endpoints to serve
    those services on. (#160 by TragicLifeHu, 614576e)
  • During DNS self-check, if the SRV records with just a dot (for a non-existent
    service), is missing, show as warning, not as error. (#184 by morki,
    e350af7)
  • During DNS self-check, warn when DANE is not configured (through static host
    keys), instead of showing "OK". (#185 by morki, 73373a1)
  • Systemd service file now syslogs as facility "mail". (by kiekerjan,
    151bd1a)
  • Add favicon to web interfaces. Admins can use the webserver config to serve a
    different file. (#186 by morki, c629ae2)
  • Attempts at improving interoperability with SMTP clients and the "login" SASL
    authentication mechanism. (#51 by hmfaysal, #223 by gdunstone & wneessen,
    aead738, 7ecc3f6)
  • Recognize more charsets than utf-8/iso-8859-1/us-ascii when parsing message
    headers with addresses. (#204 by morki, 5678b03)
  • webapi: Implement adding "alternative files" to messages sent with the Send
    method. (#188 by morki, 6c488ea)
  • webmail: Add setting to show html version of a message by default, instead of
    text version. (#196 by GildedHonour, b77f44a)
  • When login sessions to admin/account/webmail interfaces expiry or are no
    longer valid, explain the reason in the message above the login form. (#202
    by ally9335, a977082)
  • webapi: Add "RcptTo" to webapi MessageGet result. (mattanja on matrix,
    b0c4b09)
  • webadmin: At managing aliases, mention an alias member won't receive a
    message if the member address is in the message From header. (#220 by
    wneessen, bbc419c)
  • In ACME port config option, explain why using a HTTPS reverse proxy will not
    work for ACME tls-alpn-01 verification. (#218 by mgkirs, 7d3f307)
  • Add more details to X-Mox-Reason message header added during delivery, for
    understanding why a message is accepted/rejected. (#179 by Fell, #157 by
    mattfbacon, 32b549b)
  • Many small improvements.

Bug fixes

  • webadmin: Propagate error when quota size cannot be parsed, improve parsing
    and hint in error message. (#115 by pmarini-nc, 72be3e8)
  • webadmin: Don't show js runtime typecheck errors for invalid values in DMARC
    and TLS reports. (#161 by RobSlgm, a2c9cfc)
  • webmail: In list of From address to use in compose window, don't add the
    catchall address. (1a0a396)
  • webmail: Only show "edit" button on drafts, and similar for "e" shortcut.
    (8254e9c)
  • webadmin: Show correct host TLSRPT record in dns selfcheck, and make all
    suggested dns records absolute. (#182 by mdavids, 9bab312)
  • Show the same SPF record for a domain in the dnsrecords and dnscheck
    output/pages. (#176 by rdelaage & RobSlgm, 7e54280)
  • Fix parsing message headers with addresses that need double quotes. (#199 by
    gene-hightower, 016fde8)
  • Reject attempts at STARTTLS for SMTP & IMAP when no TLS config is present.
    Instead of dereferencing a nil pointer, which is caught by the go runtime,
    with fallback error handling gracefully closing the SMTP connection.
    (a7bdc41)
  • For certain errors during SCRAM authentication, handle errors more gracefully
    instead of aborting the connection. (#222 by wneessen, c7315cb)
  • For messages retired from the delivery queue, set "success" field properly,
    and include the SMTP code/enhanced code on success too (not only on failure).
    (fdc0560)
  • webmail: Fix loading a "view" (messages in a mailbox) when the "initial"
    message cannot be parsed. (#219 by wneessen, fb65ec0)
  • smtpclient: Handle server closing connection after writing its response to
    RCPT TO in pipelined mode. (#198 by soheilpro, 17346d6)
  • imapserver: Prevent unbounded memory allocations when handling a command
    (solves same problem as in CVE-2024-34055). (aef99a7)
  • For incoming SMTP deliveries with STARTTLS, use certificate of hostname if
    SNI hostname is unknown. Instead of failing the connection because no
    certificates are available. Fixes interoperability with SMTP clients that do
    opportunistic SMTP without sending a hostname. (#206 by RobSlgm, 62bd2f4)

Update instructions

Before upgrading, do a dry-run first.

  • Make a temporary backup with the old mox version:
    mox-v0.0.11 backup data/tmp/testupgrade
  • Verify that all is well with the old version:
    mox-v0.0.11 verifydata data/tmp/testupgrade
  • Verify the state with the new version:
    mox-v0.0.12 verifydata data/tmp/testupgrade

With a successful dry-run, the upgrade should go smoothly. Make a new backup
with mox-v0.0.11 backup data/tmp/backup (the previous backup was modified by
the dry-run, so couldn't be used to restore!), replace the binary and restart.
For further details, see
https://www.xmox.nl/faq/#hdr-how-do-i-upgrade-my-mox-installation

If you run into any problems, please create a bug report.

After upgrading, you may want to run "mox reparse" to parse the message headers
of all messages in all accounts. Message headers for addresses with character
sets other than us-ascii/utf-8/iso-8859-1 will be fixed.

Thanks

Thanks for all contributions, bug reports, feedback and discussions. It improves
mox, keep it coming!

Development on mox is funded through the NLnet NGI0 Entrust Fund,
https://nlnet.nl/entrust/, with financial support from the European
Commission's Next Generation Internet programme.

Downloading & compiling

See https://www.xmox.nl/install/#hdr-download.

v0.0.11

30 Apr 19:38
v0.0.11
195c57f
Compare
Choose a tag to compare

New features

  • Improve queue management (40ade99)
    • Add option to put messages in the queue "on hold", preventing delivery
      attempts until taken off hold again.
    • Add "hold rules", to automatically mark some/all submitted messages as "on
      hold", e.g. from a specific account or to a specific domain.
    • Add operation to "fail" a message, causing a DSN to be delivered to the
      sender. previously we could only drop a message from the queue.
    • Update admin page & add new cli tools for these operations, with new
      filtering rules for selecting the messages to operate on. In the admin
      interface, add filtering and checkboxes to select a set of messages to operate
      on.
  • Add a webapi and webhooks for a simple HTTP/JSON-based API, helps with
    sending transactional email. (for issue #31 by cuu508, 09fcc49)
    • Webapi allows submitting messages without having to compose the message
      yourself, and without having to know SMTP.
    • Webhooks makes it easy to process delivery failure/success updates, without
      needing IMAP and process DSNs.
    • History about outgoing/sent messages can be kept for a configurable interval,
      per account. Also for the new webhook queue.
    • Messages can be delivered with a "unique SMTP MAIL FROM" address, using a
      unique id after the localpart catchall separator, e.g.
      you+<unique>@example.org.
    • Automatic suppression list management, protecting server reputation.
    • Extra metadata can be attached through the webapi, or through
      X-Mox-Extra-: headers during SMTP submission.
    • Most settings are per-account, configurable through config file and
      account web interface. The webapi must be enabled in mox.conf through field
      WebAPIHTTP(s).
    • Gopherwatch.org was created to validate this functionality, and it can now
      operate either with SMTP/IMAP or webapi/webhooks.
  • Add aliases/lists: when sending to an alias, the message gets delivered to all
    members. (for issue #57 by hmfaysal, issue #99 by naturalethic, feedback by
    damir & marin, 960a512).
  • IMAP quota extension (RFC 9208), so mail clients can show disk usage (issue
    #115 by pmarini, 4dea2de)
  • Webmail: when moving a single message out of/to the inbox, ask if user wants
    to create/remove a rule to automatically do that server-side for future
    deliveries, either based on list-id header if present, or message-from address.
    (6c0439c)
  • Webmail: add server-side stored settings, initially for signature, top/bottom
    reply and showing the security indicator bars below address input fields.
    (for issue #102 by nixigaj, 70adf35)
  • Webmail: for replies/forwards, add button "send and archive thread" next to
    the "send" button, and give it a control+shift+Enter shortcut. (for issue #135
    by mattfbacon, 5229d01)
  • Webmail: store composed message as draft until send, ask about unsaved changes
    when closing compose window. (9529ae0)
  • Webmail: remember server-side per from-address whether we should show the
    text/html/html-with-external-resources version of a message. (0f735a1)

Improvements

  • Add account config option to skip first-time sender delay for incoming
    messages over SMTP. (8b2c978)
  • In quickstart, check if outgoing SMTP connection on port 25 can be made. New
    cloud machines tend to have the port blocked, early warning is helpful.
    (reported by arnt, 0262f46)
  • In quickstart, use "postmaster@" for the contact address with the ACME account
    for Let's Encrypt if the initial address has a non-ASCII localpart, to prevent
    account registration from failing. (reported by arnt, f4b6e14)
  • Add a "direct" transport, that allows influencing behaviour of the normal
    delivery mechanism of dialing MX records: The IP address family can be limited
    to IPv4 or IPv6 (either may not have proper reverse DNS set up). (for issue
    #149, PR #153 by lmeunier, be570d1)
  • Also give delivery another try with 5xx response when it happens during MAIL
    FROM/RCPT TO, not only EHLO as before. (related to #149, PR #152 by lmeunier,
    feb8e6c)
  • Make error messages around syntax errors in config files related to
    spurious spaces more helpful/understandable. (reported by arnt, 6516a27)
  • In SMTP server, reevaluate if SMTPUTF8 extension is needed for delivery, to
    prevent potential delivery problems to servers that don't implement SMTPUTF8.
    (issue #145, by lmeunier, 9c5d234)
  • When importing maildir, use file mtime as received timestamp if no timestamp
    is available in filename. (based on message from abdul h, 6d38a1e)
  • If webauth login cookie is missing, and forwarding was configured, hint that
    reverse proxy may be stripping path. (for issue #151 by naturalethic,
    afc47c8)
  • Webmail: When adding submitted message to Sent mailbox, keep any Bcc address
    in Bcc header. (c9451d4)
  • Make more of the dynamic config options (in domains.conf) configurable through
    the web interfaces, instead of requiring editing the config file. (baf4df5,
    a69887b, e702f45)
  • Webmail: Allow resizing of compose window, and remember width/height for
    viewport dimension. (e8bbaa4)
  • Webmail: Show all images (inline and attachment) below the text part (for the
    text view, not for html view). (3a58b2a)
  • Webmail: Add export functionality, similar to existing option in account web
    interface, but not also possible per mailbox or hierarchy. (bf5cfca)
  • Webmail: ctrl+Backspace on empty address input field removes the field (b54e903)
  • Localserve: delivery from queue now goes through the smtp server instead of
    directly from queue to local accounts. To go through the full regular delivery
    paths. (1cf7477)
  • Many small improvements.

Bug fixes

  • Always properly escape values in Authentication-Results header added to
    incoming messages. Some generated values could have characters that weren't
    escaped. (2c9cb5b)
  • Fix logging in on account and webmail interface for account names (not email
    addresses) with non-ASCII characters. (reported by arnt, 666f84e)
  • Update to latest bstore with a fix for ordering of certain database results.
    Only a single query in mox would have triggered the issue, with no expected
    impact. (d34dd8a)
  • For incoming TLS connections (HTTPS, but also SMTP and IMAP) for unrecognized
    domain names (for which we don't have a TLS certificate), don't respond with an
    "internal error" TLS alert, but with an "unrecognized name" alert. More helpful
    to user figuring out what's going on. (reported by arnt, 89a9a8b).

Update instructions

Before upgrading, do a dry-run first.

  • Make a temporary backup with the old mox version:
    mox-v0.0.10 backup data/tmp/testupgrade
  • Verify that all is well with the old version:
    mox-v0.0.10 verifydata data/tmp/testupgrade
  • Verify the state with the new version:
    mox-v0.0.11 verifydata data/tmp/testupgrade

With a successful dry-run, the upgrade should go smoothly. Make a new backup
with mox-v0.0.10 backup data/tmp/backup (the previous backup was modified by
the dry-run, so couldn't be used to restore!), replace the binary and restart.
For further details, see
https://www.xmox.nl/faq/#hdr-how-do-i-upgrade-my-mox-installation

If you run into any problems, please create a bug report.

Thanks

Thanks for contributions and/or feedback from: pmarini, tabatinga0xffff,
lmeunier, alex, arnt, abdul h, cuu508, naturalethic, nixigaj, mattfbacon,
jsfan3, hmfaysal, damir & marin from sartura, RobSlgm, daftaupe, vipas84,
TragicLifeHu, manaus0xff, jdlawrie, Bloomers7577, kbrgmn (and all those I
missed).

Feedback, feature requests, bug reports, contributions (start small!) are all
welcome. An easy way to help mox is to use it and spread the word!

Development on mox is funded through the NLnet NGI0 Entrust Fund,
https://nlnet.nl/entrust/, with financial support from the European
Commission's Next Generation Internet programme.

v0.0.10

09 Mar 19:29
v0.0.10
f3501b4
Compare
Choose a tag to compare

New features

  • Implement SMTP "FUTURERELEASE" extension in SMTP server. For submitting a
    message for delivery from the queue at a time in the future, e.g. tomorrow
    morning 9:00. (93c52b0)
  • Delivery from the queue to multiple recipients in a single SMTP transaction,
    transferring data only once. This is only done for recipients with the same
    recipient domain. The maximum recipients limit from the RFC 9422 SMTP LIMITS is
    honored and mox now announces its own limit. (47ebfa8)
  • Allow configuring DNS blocklists (DNSBLs) only for monitoring, without using
    them for incoming email. Previously, mox would only monitor DNSBLs that are
    used for incoming connections. But it is useful to know if your IPs are on a
    blocklist, unrelated to whether you're using the blocklist. (15e450d)

Improvements

  • Mox has a separate website now, https://www.xmox.nl. It should be more
    friendly for first-time visitors than the github page. It also has an initial
    video, with more to come. Content, except images/videos, is in the mox git
    repository, so functionality and documentation can be changed together.
    (0bc3072)
  • Fix interpreting a per-account negative total size quota as "no limit",
    overriding the global/default setting. (issue #115 by pmarini-nc)
  • Webmail: When Q/B-word-decoding attachment filenames, recognize more
    character set encodings. (issue #113 by jsfan3)
  • Webmail: show unicode for internationalized email addresses by default. Not
    the xn-- names, which are not user-friendly. A hover still shows the xn--
    names. After talking to arnt at FOSDEM.
  • Localparts of addresses are now normalized to Unicode NFC throughout mox,
    including for incoming deliveries. Previously, incoming SMTPUTF8 deliveries
    with non-NFC-normalized unicode localparts (e.g. with separate code points for
    accents and such (NFD)) would not be accepted. (8e6fe74)
  • The PRECIS (RFC 8265) password profile is now applied when setting/using
    passwords. It prevents confusing users who use unusual unicode whitespace
    codepoints in their passwords. (c57aeac)
  • Webmail: Implement registering "mailto:" links. Click the new button in the
    Help popup to register. From Hans-Jörg. (ee1db2d)
  • In DSNs, show the full (multiline) SMTP response, not just the first line. It
    often has helpful details. (50c1396)
  • More DSN improvements: put the full SMTP reply in field Diagnostic-Code
    (1c934f0), when delivering a local DSN add Delivered-To header
    (79da4fa), when parsing an incoming DSN set the Action field (e.g. failed,
    delayed) (f6497b1), match DSN to threads based on Referenced/In-Reply-To
    only, not subject (13923e4).
  • In quickstart and self-check, improve the check whether resolver verifies
    DSNSEC. We were looking up NS on ".", but some DNSSEC-verifying resolvers
    respond to that with unauthentic data. (issue #139 by triatic)
  • More helpful instructions about setting up a DNSSEC-verifying resolver, and
    how to test it. (issue #131 by romner-set)
  • Relevant for reusable components: the "slog" package from golang.org/x/exp is
    now replaced with slog from the standard library. Function signatures have
    changed. (d1b87cd)

Bug fixes

  • smtpserver: Spurious \r were sometimes injected when fixing up crlf line
    endings for incoming messages. The check could look at wrong a buffer,
    seemingly randomly incorrectly concluding a \r was missing before a \n.
    Messages with the extra \r added shouldn't cause any trouble.
    You can find these messages with grep -rn $'\r\r$' data/accounts/$youraccount/msg/,
    remove one \r manually and run mox fixmsgsize $youraccount to
    reparse the message. (issue #117, by haraldrudell)
  • Deliveries could seemingly randomly but consistencly fail with an incorrect
    diagnostic about the message containing a bare newline. It would happy due to
    crlf handling around buffer starts/ends. (issue #129 by x8x)
  • When adding a message to the queue for delivery, set the correct local
    account, so DSNs about delivery failures go to the correct user, not the
    postmaster account. (dc83ad1)
  • imapserver: In a sequence/uid pattern, the "*" would in some cases be interpreted
    as the first message, but it should always be the last message in a mailbox.
    (14aa854).
  • Fix displaying DMARC reports with empty values for some fields. The
    TypeScript checks would reject them for being invalid enums, empty strings
    are now part of the enum. (20812dc)
  • In outgoing TLS reports, always use DNS ASCII A-labels, not unicode U-labels.
    (62be829)
  • For domains configured only for TLS/DMARC reporting, don't reject messages to
    that domain during submission, but deliver them as normal. (1d9e80f)
  • smtpclient: Treat server's size limit of SIZE=0 as "no limit", instead of
    failing to deliver. (39bfa43)
  • Accept TLS reports that reference multiple domains, and that mix reports
    about hosts and recipient domains. (e0c36ed)
  • Admin: Prevent writing out an invalid domains.conf that cannot be parsed
    again. Happened when the last address of an account was removed through the
    admin web interface. (issue #133 by ally9335)
  • Webmail: sending to invalid addresses could result in (failing) attempts to an
    empty address instead of returning an error message to the user. (63cef8e)

Update instructions

Before upgrading, do a dry-run first.

  • Make a temporary backup with the old mox version:
    mox-v0.0.9 backup data/tmp/testupgrade
  • Verify that all is well with the old version:
    mox-v0.0.9 verifydata data/tmp/testupgrade
  • Verify the state with the new version:
    mox-v0.0.10 verifydata data/tmp/testupgrade

With a successful dry-run, the upgrade should go smoothly. Make a new backup
with mox-v0.0.9 backup data/tmp/backup (the previous backup was modified by
the dry-run, so couldn't be used to restore!), replace the binary and restart.
For further details, see
https://www.xmox.nl/faq/#hdr-how-do-i-upgrade-my-mox-installation

If you run into any problems, please create a bug report.

Thanks

Thanks for contributions and/or feedback from: haraldrudell, x8x, romner-set,
triatic, mteege, Hans-Jörg, arnt, jsfan3, pmarini-nc, ArnoSen, andreasheil,
theduke, daluntw, lmeunier, ally9335, p-rintz, daftaupe (and everyone at the
FOSDEM email devroom, and all those I missed).

Feedback, feature requests, bug reports, contributions (start small!) are all
welcome. An easy way to help mox is to use it and spread the word!

Development on mox is funded through the NLnet NGI0 Entrust Fund,
https://nlnet.nl/entrust/, with financial support from the European
Commission's Next Generation Internet programme.

Download or compiling

See https://www.xmox.nl/install/#hdr-download.

v0.0.9

09 Jan 11:06
v0.0.9
dda0a4c
Compare
Choose a tag to compare

New features

  • Per-account disk space quota. Mox now tracks the total size of messages in an
    account (overhead from the message index database or file system is not
    included). A maximum disk usage can be configured globally and/or per account.
    If configured, the maximum allowed disk usage is enforced. By default, no
    maximum is configured. Setting quota for accounts can prevent a single account
    from filling up the disks. (d73bda7)
  • When suggesting CAA DNS records (specifying which Certificate Authorities are
    allowed to sign certificates for a domain), suggest variants that bind to the
    ACME account ID, and restricts the validation methods to those used by mox.
    Should prevent MitM close to a machine from requesting TLS certificates through
    ACME as seen on the internet recently. (db3fef4)
  • Add config file fields for ACME external account binding (EAB). Some ACME
    providers require EAB to link an ACME account with a non-ACME account at the
    provider. With EAB, more ACME providers can be used with mox. (ee1094e)
  • Implement the PLUS-variants of the SCRAM authentication mechanisms:
    SCRAM-SHA-256-PLUS and SCRAM-SHA-1-PLUS. The PLUS variants add TLS channel
    binding: Authentication only succeeds if the client and server are on the same
    TLS connection. Authentication will fail if there is a MitM (that has a valid
    TLS certificate). (e7478ed)
  • Use a mail.<domain> CNAME for the SMTP (submission) and IMAP servers of a
    domain, pointing to the mail server host name. Before, clients were instructed
    to configure the mail server host name directly, but that makes it harder to
    migrate the domain to another mail server in the future: All clients would need
    to update their settings. A CNAME can be pointed to a new server without
    requiring changes to client settings. (da3ed38)

Improvements

  • The admin, account and mail web interfaces now use session cookie-based
    authentication (with csrf) instead of HTTP authentication. These interfaces
    now have a "logout" button (not possible with HTTP authentication). (#58,
    0f8bf2f)
  • Webmail: Don't automatically mark unclassified messages in the Rejects mailbox
    as non-junk when reading them. For all other mailboxes the behaviour is
    unchanged. (416113a)
  • Webmail: Ask user to reload the application when the server version has
    changed. (8e37fad)
  • Webmail: In the message view, show the DMARC status of the domain of the
    message "From" address. (fb81eff)
  • Webmail: When composing, leave out our own address when replying. (7c1879d)
  • Junk filtering: Make content-based filtering for first-time senders more
    strict for messages delivered over non-TLS connections, or when the addressee
    isn't in a To/Cc message header. Common for junk, uncommon for non-junk.
    (2ff87a0)
  • SMTP server: after "MAIL FROM:" and "RCPT TO:" commands, allow a space (which
    is invalid syntax) also for delivery. We only allowed it for submission,
    assuming only (submitting) mail clients or spammers had sloppy SMTP
    implementations. In practice, also legitimate delivering mail servers have
    sloppy implementations. (#101, af5da17)
  • When generating a Authentication-Results message header (with results for
    SPF/DKIM/DMARC/etc), put each result on a new line for better readability.
    (2710a5b)
  • Make many non-server Go packages more easily reusable. Package imports were
    changed so more packages can be imported without pulling in mox internals. See
    https://github.com/mjl-/moxtools for a tool that reuses Go packages. It is
    deployed publicly at https://tools.xmox.nl/. As part of this change, mox now
    uses Go's slog package for logging. Changes in the API's between releases are
    tracked at https://github.com/mjl-/mox/tree/main/apidiff. (5b20cba,
    72ac1fd, f3a35a6)
  • SMTP server: for submission, if a message has a Return-Path header, only fail
    in pedantic mode. (#103, 57fc37a)
  • Webmail: For messages in the Sent mailbox, show To/Cc/Bcc in italic, and show
    all correspondents in collapsed threads. (#104, 802dcef)
  • The admin and account web interfaces were changed from JavaScript to
    TypeScript, making it easier to maintain. (a9940f9).
  • Implement IMAP-UTF-7 more fully, and allow creating mailboxes with "special"
    characters: "&" (the IMAP-UTF-7 escape character), "#" (the IMAP namespace
    character), "*" and "%" (matching characters). These were not allowed out of
    caution, but occur in real-world mailbox names. Mox now uses IMAP-UTF-7 when
    sending mailbox names for clients that did not enable IMAP4rev2 or UTF8=ACCEPT.
    Before, mox would always send UTF-8, but not all clients understand that, and it
    can cause confusion with IMAP-UTF-7 and "&" escaping. (#110, d84c96e)
  • IMAP server: Add STATUS=SIZE as capability. It was already implemented as
    part of IMAP4rev2, but older clients won't recognize that. (59bffa4)
  • And more smaller improvements.

Bug fixes

  • SMTP server and SMTP smuggling: Mox was itself not vulnerable to SMTP
    smuggling, treating only "\r\n.\r\n" as end of transaction. But two
    improvements have been made: (1f9b640)
    1. Bare carriage returns are no longer accepted during SMTP transactions. Bugs
      in other mail servers can lead them to accept other sequences as
      end-of-transaction, notably "\r.\r". Mox would accept submitted messages with
      that sequence for delivery. Such messages could trigger bugs in other mail
      servers causing them to materialize non-existent messages. By no longer
      accepting bare carriage returns in submitted messages, mox can no longer be used
      to trigger the "\r.\r"-bug in other mail servers. SMTP transactions with a bare
      carriage return now result in an error mentioning SMTP smuggling. Mox can still
      store messages with bare carriage returns, e.g. from imports. Mox already added
      missing carriage returns to bare newlines.
    2. A bug in mox caused sequences of "\nX\n" for any X (including "\n.\n") to
      result in a temporary processing error. For "\n.\n" this accidentially was fine
      behaviour, for other characters the bug has been fixed. Any sequence of
      "\r\n.\r\n" where one or both carriage returns are missing now result in an
      error mentioning SMTP smuggling.
  • IMAP server: The on-disk message size was not correctly calculated for messages
    added with the APPEND command (typically used for imported messages and when a
    mail client sends a message) when bare newlines ("\n") got a missing carriage
    return added ("\r\n"). This would cause errors when attempting to read the
    message. If you are affected by this, run "mox fixmsgsize " to fix up
    incorrect message sizes. Reported by daftaupe. (02eb7b5)
  • SMTP server: When writing "slow responses" (when a message is deemed junk),
    ensure the total response time isn't too long (slightly less than 30 seconds).
    Slow responses were writing 1 byte per second. With a long response (e.g. long
    error message), a sending mail server may not consume a full response. If mox
    was the sending server, it would report a timeout after 30 seconds. Report by
    naturalethic. (fbc18d5)
  • IMAP server: Only send "OLDNAME" in a response to the LIST command when
    IMAP4rev2 is enabled. IMAP4rev1 clients (most common) don't understand it. From
    duesee with imap-flow. (41e3d1a)

Update instructions

Before upgrading, do a dry-run first.

  • Make a temporary backup with the old mox version:
    mox-v0.0.8 backup data/tmp/testupgrade
  • Verify that all is well with the old version:
    mox-v0.0.8 verifydata data/tmp/testupgrade
  • Verify the state with the new version:
    mox-v0.0.9 verifydata data/tmp/testupgrade

With a successful dry-run, the upgrade should go smoothly. Make a new backup
with mox-v0.0.8 backup data/tmp/backup (the previous backup was modified by
the dry-run, so couldn't be used to restore!), replace the binary and restart.
For further details, see
https://github.com/mjl-/mox#how-do-i-upgrade-my-mox-installation

If you run into any problems, please create a bug report.

After upgrading, you may want to:

  • Run "mox fixmsg " if you've imported messages over IMAP that have
    bare newlines ("\n" instead of "\r\n").
  • Configure your email clients to use authentication mechanism
    SCRAM-SHA-256-PLUS for SMTP (submission) and IMAP, if they support it. If mail
    clients have trouble logging in after upgrading, they may be picking a SCRAM
    PLUS variant without properly supporting it. Explicitly configuring the non-PLUS
    authentication mechanism should fix the problem and ensures a MitM cannot
    downgrade the chosen authentication mechanism by altering the list of supported
    authentication mechanisms.
  • Change the CAA records for your domains to include the ACME account id and
    allowed validation methods. See the suggested DNS records for each configured
    domain.
  • Set disk usage quota, either globally for all accounts or per account. See
    QuotaMessageSize in mox.conf, https://pkg.go.dev/github.com/mjl-/mox/config.
  • Add ClientSettingsDomain: mail.<yourdomain> to each domain in domains.conf,
    add the CNAME record as afterwards suggested in the DNS records page, and
    update client account settings to use the new host name.
  • For mox setups configured behind an existing webserver, add "Forwarded: true"
    to the (Admin|Account|Webmail)HTTP(S) sections in mox.conf. It causes them to
    use X-Forwarded-* headers for determining if HTTPS was active (for secure
    cookies), and for the IP used for rate limiting.

Thanks

Thanks for contributions and/or feedback from: Fell, duesee (and
https://github.com/duesee/imap-flow/), daftaupe, naturalethic, jsfan3, Halyul,
mattfbacon, jsaponara, pmarini (and those I missed).

Fee...

Read more

v0.0.8

22 Nov 22:23
v0.0.8
9d2e761
Compare
Choose a tag to compare

New features:

  • DNSSEC-awareness throughout the code base, based on
    https://github.com/mjl-/adns, a fork of Go's DNS resolver. DNSSEC
    is a requirement for DANE (see below). If you don't have a
    DNSSEC-verifying stub resolver configured, DNS lookups are regarded
    as unverified. Installing unbound and and is still the recommended
    action.
  • DANE for incoming and outgoing delivery (RFCs 7672, 6698 and 7671).
    DANE is a mechanism to require verified TLS (with STARTTLS) for delivery
    over SMTP. Verification with DANE does not use the global WebPKI/PKIX
    pool of Certificate Authorities. With DANE, verification is done based
    on DNS records of type TLSA. These records specify (hashes of) public
    keys to allow (DANE-EE), ignoring expiration/hostname-match/issuing
    party, and/or they specify (hashes of) certificates of allowed
    certificates authorities (DANE-TA), regardless of whether those
    authorities are in the globally trusted WebPKI/PKIX CA pool.
    DANE requires that DNS records are DNSSEC-protected, both to protect
    the MX records and the TLSA records. MTA-STS (already implemented)
    has similar goals, but does use the WebPKI/PKIX Certificate Authorities
    pool, both to verify TLS certificates and to protect MX records.
    DANE and MTA-STS can coexist: In the default configuration, mox
    generates private keys, then retrieves certificates from Let's Encrypt
    for these private keys (through https://github.com/mjl-/autocert, a
    fork of golang.org/x/crypto/acme/autocert). These certificates are
    valid for MTA-STS, and TLSA records are generated for the keys for
    verification with DANE. For inbound delivery with DANE protection,
    your DNS records must be DNSSEC-protected. For outbound delivery with
    DANE protection, a trusted DNSSEC-verifying stub resolver is required.
  • Mox now compiles on Windows, so "mox localserve" and most other
    commands to work, but "mox serve" (the actual mail server) does not
    yet work.
  • "SMTP Require TLS Option" (RFC 8689), consisting of two mechanisms:
    1. A REQUIRETLS SMTP extension to require verified TLS along each hop
      in message delivery, either through MTA-STS or DANE.
    2. A message header "TLS-Required: No", that overrides any TLS
      requirement along the way as specified by any MTA-STS or DANE
      policy.
      These mechanisms can be used to ensure secure delivery, or to work
      around delivery issues due to TLS requirements. Mox remembers whether
      an SMTP server offered the REQUIRETLS extension. Webmail automatically
      selects it if all recipients support it. Webmail also lets the user
      select the "TLS-Required: No" header.
  • Outgoing DMARC reports (RFC 7489). Mox now stores the results of DMARC
    evaluations for inbound messages. These results can be viewed in the
    admin web pages. Reports are typically sent every 24 hours (covering a
    24 UTC day), but will be sent for up to 1 hour intervals if requested
    by a domain. Sending DMARC reports is enabled by default, but can
    be disabled through new option NoOutgoingDMARCReports in mox.conf.
    Reporting addresses can be added to a suppression list, to reduce
    noise due to deliverability issues. Incoming DMARC reports were
    already implemented.
  • Outgoing SMTP TLS reporting (RFC 8460). When delivering outbound
    messages, the SMTP client will look up MTA-STS and/or DANE policies
    for TLS requirements, with a fallback to opportunistic TLS.
    The evaluated security policies, (TLS) connection success/failure
    counts, and any failure details, are stored. Reports are sent once
    per day to reporting addresses in the TLSRPT DNS record of a domain,
    over a 24 hour UTC day period. By default, reports are only sent
    if there was a failure. The pending results can be viewed in the
    admin web pages. Sending reports can be disabled with new option
    NoOutgoingTLSReports in mox.conf. Reports with only successes can be
    enabled through OutgoingTLSReportsForAllSuccess. Reporting addresses
    can be added to a suppression list to reduce noise due to delivery
    failures.

Improvements:

  • Webmail: Recognize encoded file names in message attachments. Either with
    RFC2231-encoding (as specified) or Q/B-word encoding (as used in practice).
    (#82)
  • Webmail: For portait images, don't let image extend beyond window height.
  • Webmail: Wrap long header lines, instead of showing horizontal scrollbar.
  • Webmail: Replying without having text selected now starts a top-post
    with an "On ... wrote:"-line. Replying with text selected still starts
    a bottom-post containing only the selected text, quoted. (#83)
  • Webmail: In the compose window, autoresize address input fields to
    match the content.
  • Webmail: When composing a message, show security properties of recipient
    addresses: Whether STARTTLS is known to be offered by the SMTP server
    (historically), whether MTA-STS is implemented, whether MX records are
    DNSSEC-signed, whether DANE is implemented, and whether REQUIRETLS is
    offered by the SMTP server (historically).
  • Webmail: Add clear marker between message header and body, so an
    HTML message cannot fake being part of the UI.
  • Webmail: If a "display name" of an address contains address-like
    characters ("@" or "<" or ">"), only display the actual email address
    in the message listing, not the display name. Should prevent confusion
    attacks with messages specifying an unrelated email address in the
    display name.
  • The suggested SRV DNS record for autodiscovery now points directly to
    the host name, not to a CNAME (which is technically invalid, but seems
    to work in practice).
  • When ACME-validation for a new TLS certificate fails, log error messages that
    may explain the reason. E.g. "your CAA record forbids Let's Encrypt from
    issuing certificates".
  • SMTP server: workaround for Windows Mail that has invalid additional space in
    its "AUTH PLAIN" command.
  • Fix delivery to recipient domains with an MX host containing an underscore,
    such as "_dc-mx.." as apparently used by cloudflare. From
    richard g.
  • When generating a DSN message (for delivery failure), try harder to DKIM-sign
    it: With a configured domain, also when sending from
    postmaster@mailhost..
  • For incoming messages, track whether TLS and REQUIRETLS was used during
    delivery, and whether the message matched a forwarding or mailing list rule,
    and show it in the webmail.
  • In logging, change "fatal io error" to just "io error". The "fatal" sounds
    too serious, it's just the connection that will be closed. (#39)
  • Add rfc/xr.go to generate HTML pages with cross-referenced code and
    RFC. These HTML pages are published at https://www.xmox.nl/xr/dev/
  • Webmail: In case of long lists of addresses in To/Cc/Bcc headers, only show
    the first 4 addresses along with a "More" button. (#98)
  • Clarify documentation on importing messages from the command-line,
    which can be unintuitive due to systemd service file mount points. (#79)
  • Implement obsolete SASL LOGIN for submission, for interoperability with the
    new cloud Outlook.
  • Fix IMAP ESEARCH response for clients before IMAP4rev2, notably cloud
    Outlook.
  • Many small improvements.

Bug fixes:

  • Security: When looking up MTA-STS policies, don't follow CNAME records
    for the recipient domain. A single unauthenticated CNAME response
    could redirect policy lookup to another domain.
  • Webmail: When replying to selected text consisting of characters in multiple
    unicode blocks, don't loose some of the selected text in the reply.
  • Don't parse DKIM "selectors" as IDNA domains. They are just DNS
    labels. Based on email from richard g.
  • Update to latest bstore (database library) to fix a bug with
    deleting/updating records. Problem found during development of new
    features, behaviour not seen in any committed version.
  • Webmail: Fix the date shown in the message headers. It was off by the timezone.
  • Fix concurrency bug with accessing a math/rand PRNG with Read. Mostly
    replaced with crypto/rand. Found during development and tests.
  • The queue page on the webadmin would fail with a JS error when a message was
    in the queue and no transport was configured (which is the default).
  • For domains configured only to accept DMARC reports, don't request an
    autoconfig TLS certificate through ACME at startup.
  • For incoming messages, convert bare newlines to carriage
    return+newline. The import code already did this. Having bare newlines
    could cause imapserver's fetch command to fail with a (connection)
    panic in some cases.

Update instructions:

Before upgrading, you should do a dry-run first:

  • Make a temporary backup with the old mox version:
    mox-v0.0.7 backup data/tmp/testupgrade
  • Verify that all is well with the old version:
    mox-v0.0.7 verifydata data/tmp/testupgrade
  • Verify the state with the new version:
    mox-v0.0.8 verifydata data/tmp/testupgrade

With a successful dry-run, the upgrade should go smoothly. Make a new backup
with mox-v0.0.7 backup data/tmp/backup (the previous backup used for the
dry-run has been modified, so couldn't be used to restore!), replace the binary
and restart.

If you are upgrading from v0.0.6, see its upgrade instructions for commands to
execute. It's better to immediately upgrade to v0.0.8 (see issue #71).

If you run into any problems, please create an issue.

After upgrading, you may want to configure DANE:

To make use of DANE for outbound deliveries, make sure you have a
trusted DNSSEC-verifying stub resolver. Unbound is recommended. Don't
use systemd-resolved, its DNSSEC support is not ready for use.

To make use of DANE for inbound deliveries, first make sure your
DNS records are DNSSEC signed, and your DNS operator supports TLSA
records. The SMTP TLS private keys ("host keys) should be added to
the...

Read more

v0.0.7

24 Sep 12:20
v0.0.7
91140da
Compare
Choose a tag to compare

version: v0.0.7
date: 2023-09-24

Update instructions:

Due to a bug with "expunging" (deleting) messages that were
junk-filter-trained, messages that were removed from disk could be resurrected,
causing errors when mox would later try to open such messages again. Before
upgrading, you should first check and resolve this problem:

  • Find missing files by running: mox-v0.0.6 fixmsgsize
  • Create empty replacements for the missing files (use "touch data/accounts/...").
  • Update the message metadata in the database for the newly created files:
    mox-v0.0.6 fixmsgsize
    (this will fix the previously missing files, but should no longer find new
    missing files)
  • Fix per-mailbox message/unread counts for each affected account, run:
    mox-v0.0.6 recalculatemailboxcounts
    ("account" is the name as it appears in the data/accounts/ directory,
    not necessarily an email address)
  • The affected accounts will see the recreated empty messages, which
    can be deleted.

Now with the storage consistency resolved, you can start with the upgrade. Mox
v0.0.7 adds message threading, and all messages will be read and assigned a
message thread. This is done in the background, in two steps, and may take a
while. The first step adds Message-ID and a "thread base subject" to each
message in the database. The second step reads through all messages and uses
their References/In-Reply-To/Subject message headers to match threads. In
testing, upgrading took approximately 1 minute per 100k messages, but it will
depend on the hardware. Accounts are available for reading and delivery during
the upgrade, but the webmail may tell you that threading is not yet available.

You should do a dry-run of the upgrade first:

  • Make a temporary backup with the old mox version:
    mox-v0.0.6 backup data/tmp/testupgrade
  • Verify that all is well with the old version:
    mox-v0.0.6 verifydata data/tmp/testupgrade
    NOTE: If you still get an error about a message with the wrong size, see
    https://github.com/mjl-/mox/issues/71 for a fix.
  • Trigger the threading upgrade using the new version:
    mox-v0.0.7 openaccounts data/tmp/testupgrade
    (it prints nothing until done; the live upgrade prints progress)
  • Verify the new state:
    mox-v0.0.7 verifydata data/tmp/testupgrade

With a successful dry-run, the upgrade should go smoothly. Make a new backup
with mox-v0.0.6 backup data/tmp/backup (the previous backup used for the
dry-run has been modified, so couldn't be used to restore!), replace the binary
and restart.

After the upgrade, any accounts affected by the "missing file" problem should
get their "uid validity" increased, so IMAP clients will resynchronize. Run
"mox-v0.0.7 bumpuidvalidity " for each affected account. The change is
made directly in the database file, so no IMAP/webmail sessions should be
active for the account. If any sessions are active, the database file is locked
and "bumpuidvalidity" will print a timeout error. Briefly shutting down mox is
an option, the bumpuidvalidity command operates quickly.

Apologies for the inconvenience, mox aims to make administrating a mail server
easier than this.

If you run into any problems, please create an issue.

New features:

  • Keep track of message threading, and add a threaded view to the webmail. Newly
    delivered messages are matched against threads, as are imported messages. The
    message threading is currently only exposed through the webmail client, not yet
    through IMAP. The webmail has new keyboard shortcuts to navigate between
    threads, collapse/expand them (with state remembered), or mute them (so new
    deliveries to the thread are marked as read).
  • Add option to accept DMARC and TLS reports for other domains. You would add
    the other domain in mox, and specify an alternative domain (of your regular
    dmarc/tls reporting address) for the recipient of the reports. The webadmin
    DNS-check page will show the required DNS records for delegated reporting.
    Incoming reports will be delivered to the configured reporting address, and
    processed by mox.
  • Easier account setup on Apple devices without device management profiles
    (.mobileconfig files). Download a profile from the account web interface, at an
    email address. Or open the QR-code to easily get the profile on a mobile device.
    For issue #65 by x8x.
  • Transparent gzip compression when webserving files for selected content-types
    that are likely compressible. Both for static files (with a cache of compressed
    files, max 512MB stored in $datadir/tmp/httpstaticcompresscache), and for
    forwarded requests (gzip level "fast"). Compression can be enabled per handler.

Improvements:

  • In SMTP server, for submission (with authenticated clients), don't fail on a
    bad domain/IP address in the EHLO command. With submission the domain/IP is
    irrelevant, and clients often fill in something that isn't strictly correct. No
    need to prevent those users from submitting email (except in pedantic mode).
    For issue #55 reported by gimpf.
  • At top of config files, mention the config file format, and hints to prevent
    likely mistakes (given how sconf is different from what admins may be used to),
    including a pointer to the sconf documentation page. For issue #56 reported by
    kikoreis.
  • Recognize when quickstart is probably run behind a NAT, possibly a container,
    and set the "NATIPs" field of the "public" listener in mox.conf accordingly.
    This triggers when you set up mox for a public domain, but only have
    private/loopback IPs on the machine. For issue #59 reported by pmarini.
  • When moving a message out of Rejects mailbox, mark it as unread. For issue #63
    by x8x.
  • The "mox setaccountpassword" subcommand now takes an account name as
    parameter instead of an email address. The email address could be confusing
    in the face of wildcard addresses and a typo/non-explicitly-created address:
    The account holding the wildcard address would get a new password. For issue
    #68 by x8x.
  • Make Mac OS X Mail use the special-use mailbox attributes so it finds the
    correct "Sent" and "Trash" mailboxes (instead of creating its own "Sent
    Messages" and "Deleted Messages" mailboxes). The IMAP server now
    unconditionally sends the special-use flags, even if the mail client doesn't
    request it. For issue #66 by x8x.

Bug fixes:

  • Fix expunging of messages marked junk/nonjunk. The messages would be marked
    as expunged, then the junkfilter would retrain and clear the expunged field
    again. This would cause the message to be resurrected while the on-disk message
    file was already removed. Trying to read such messages would fail. The update
    instructions should help fix the problem.
  • Fix "mox sendmail" when submitting over a TLS connection by setting the remote
    host name to verify the certificate of. Due to a logic bug the name wasn't set
    and the connection would fail due to the missing setting.
  • Don't generate duplicate suggested SPF record if hostname is equal to domain
    name, e.g. postmaster@mail.domain.example. Fixes issue #46 reported by x8x.
  • Fix showing attachments of type text/plain in webmail, they weren't shown at
    all because they were skipped when parsing the message during webmail message
    processing.
  • Fix parsing the List-Post header in messages, for use in webmail when replying
    to a mailing list message.
  • When moving a message to the mailbox that has the special-use "junk" flag,
    mark the message as junk and retrain. This should have already worked just like
    the "AutomaticJunkFlags" config option, and the default account config already
    handles marking messages as junk based on that option, but the special-use flag
    should be recognized independently and now also takes precedence.
  • Set the correct special-use mailbox flag "\Drafts" instead of "\Draft" on the
    draft mailbox. Mail clients may have not found the correct drafts mailbox
    before. For issue #66 by x8x.
  • Fix "mox bumpuidvalidity" to not create a (mostly harmless) uidvalidity
    inconsistency that "mox verifydata" will warn about. For issue #61 by x8x.
  • And quite a few smaller tweaks/improvements/fixes.

Special thanks for contributions and/or feedback from: x8x, gimpf, kikoreis,
pmarini, fairking, gedw99, hmfaysal (and those I missed).

Feedback, requests, bug reports, contributions (start small!) are all welcome.

Good news: Mox is now being funded for a year of continued development through
the NGI0 Entrust Fund, a fund established by NLnet with financial support from
the European Commission's Next Generation Internet programme,
https://nlnet.nl/project/Mox/!

v0.0.6

16 Aug 15:24
v0.0.6
0b94752
Compare
Choose a tag to compare

Update instructions:

Make a backup, replace the binary and restart.

After the upgrade, the first time an account is opened with this new version,
new message indexes are created and mailbox message count statistics are
calculated. For large mailboxes, the time and memory this takes can be
noticable. The upgrade tests take about 15 seconds on 570k messages on a
thinkpad x1 from 2018, and run with a max memory data size of 768MB.

Don't forget to make a backup of the data directory with your currently running
mox before upgrading (e.g. "mox-v0.0.5 backup data/tmp/backup"). You can
dry-run the upgrade by making a separate backup ("mox-v0.0.5 backup
data/tmp/testupgrade") and running the "verifydata" command with the new mox
version ("mox-v0.0.6 verifydata data/tmp/testupgrade"). Running "verifydata"
with a newer mox will make changes to the database files, so don't run it on a
backup you may need to restore.

It is recommended to run the new "mox reparse" command after upgrading. It will
reparse all messages with the improved message parsing code.

For existing installations, the new webmail must be enabled manually in mox.conf
with config options "WebmailHTTP" and/or "WebmailHTTPS", similar to
"AccountHTTP(s)". See the example config printed by "mox config
describe-static".

If you are forwarding email to an address hosted with mox, you may want to
configure the new "IsForward" and possibly "AcceptRejectsToMailbox" options in
a delivery ruleset for better junk handling/analysis.

If your mox is behind a NAT, and your mox.conf uses config option IPsNATed, you
should switch to new config option NATIPs, and specify the public IPs that are
NATed.

New features:

  • Webmail, for reading/writing messages. It is similar to other regular and
    webmail clients, with a decent set of features for a first version (with more to
    come in the future). Webmail is enabled for new installs created with the
    quickstart. For more details about the implementation, see
    849b4ec.
  • IMAP extensions CONDSTORE and QRESYNC have been implemented. With CONDSTORE,
    changes to messages are tracked with a modification sequence, "modseq".
    This is used by mail clients both to efficiently find changes when reconnecting,
    and for conditionally storing updated message flags (only if modseq is still as
    expected). QRESYNC is an additional extension for faster full mailbox
    synchronization. These are useful by themselves, but keeping track of the
    "modseq" will also help with JMAP.

Improvements:

  • Faster IMAP "STATUS" command (which checks mailbox status), because mox now
    keeps track of total/unseen/deleted number of messages in mailboxes. Noticable
    in larger mailbox (>50k messages).
  • Be less strict by default when parsing messages, and handle non-ascii/utf-8
    encodings in message subjects. We already accepted messages with problems, but
    stored them mostly unparsed. IMAP clients would parse the message themselves so
    users wouldn't notice anything, but now that we have webmail it is more
    important to have parsed forms of messages for problematic messages.
  • Properly decode character encodings other than ASCII and UTF-8 when
    returning/searching text in messages.
  • Be more lenient for (authenticated) submission of email over SMTP (but not
    during regular SMTP delivery). Before, we were strict about certain protocol
    violations, but that wasn't helpful behaviour for legitimate users. In pedantic
    mode, mox still rejects these violating commands.
  • New config options "AcceptRejectsToMailbox" and "IsForward" in Rulesets, for
    handling incoming messages that are forwards from another address. By setting
    "AcceptRejectsToMailbox", if the junk analysis says a message is junk, the
    message is accepted during SMTP instead of rejected, but delivered to the
    configured mailbox. It isn't always a good idea to reject forwarded messages
    that are junk. By setting "IsForward", the junk analysis and future
    classification based on such a message is done differently. See the config
    option for details. (PR #50 by bobobo1618)
  • Add a KeepRejects options to account configurations, so the Rejects mailbox
    isn't automatically cleaned up. (PR #49 by bobobo1618)
  • Add option NATIPs to a listener in mox.conf for better handling when mox is
    configured behind a NAT.
  • "mox verifydata" now checks for more potential issues, and mox gained a few
    subcommands to fix those issues.

Bug fixes:

  • Fix showing the progress while importing messages. Messages were still
    imported, but the SSE connection with progress updates was broken.
  • Fix potential delays in propagating changes to IMAP IDLE connections.
  • IMAP compliance: add missing space after continuation line ("+").
  • IMAP compliance: add missing empty untagged response for SEARCH in case of no
    matches.
  • Fix the -existing-webserver flag for "mox quickstart". Its TLS key/cert check
    tripped it up.
  • And many more small fixes and improvements.

Special thanks for contributions and/or feedback from: Mendel, bobobo1618,
hmfaysal, x8x, kikoreis, gerben, andrii, liesbeth, morki, gedw99 and everyone I
forgot. More feedback/bug reports welcome!

To download, see https://github.com/mjl-/mox#download

v0.0.5

03 Jul 09:14
v0.0.5
785a38c
Compare
Choose a tag to compare

v0.0.5

Update instructions: No special steps required other than updating the binary.

Fixes/Improvements:

  • Fix "mox localserve", and some commands that talk to a running mox instance
    over the ctl socket. Broken in previous release.
  • During quickstart, warn if the host name resolves to a loopback IP, as it
    will likely prevent email delivery to local accounts from working. (#37 by
    dmikushin)
  • In case of DNS resolve errors on Linux likely due to missing name server in
    /etc/resolv.conf, point user to systemd-resolved manual page, the likely
    cause. (#38 by ArnoSen)
  • Redirect requests for /admin to /admin/, to prevent confusing users about
    failed credentials when they try to open the admin web interface. (#43 by
    FieryCod, lemos1235)
  • Fix race condition when handling forwarded websocket connections. Could show
    up as calling Close on a nil connection. (found by existing tests)
  • Fix parsing bugs in the IMAP server. One of them may cause correct commands
    to be rejected, but this probably isn't commonly triggered. (found by new
    tests)
  • Fix bug interpreting UID sets in the IMAP server, potentially causing a
    SEARCH not to return all matching messages, and potentially not EXPUNGE-ing
    (removing) all messages marked as deleted. (found by new tests)
  • Fix bug with assigning UIDs when renaming Inbox. Renaming Inbox is a special
    case and its special handling wasn't renumbering UIDs of moved messages,
    potentially leading to errors on later operations on the destination mailbox.
    Command "mox verifydata" has been updated to check for this inconsistency. Two
    new unlisted commands are introduced that can correct these inconsistencies:
    "mox reassignuids" and "mox fixuidmeta". This will only affect you if you have
    renamed your Inbox, not a common operation. (found by new tests)
  • Fix importing messages with a date with a timezone >= 24 hours. (found while
    importing old email archives)

New features:

  • Implement user-defined flags/keywords in the IMAP server. Previously, only
    the standard keywords like \Seen, \Draft, etc were supported.
  • Configurable routes for outgoing email, through configurable transports. The
    default transport is regular direct delivery to MX hosts. Other transport
    options: 1. delivery through another mail server with SMTP (with various
    TLS/STARTTLS/authentication options), to a relay/smarthost or 3rd party
    account; 2. direct delivery but making connections through a SOCKS proxy.
    Routes can be configured for accounts, domains, or globally. A matching route
    selects the transport. If no route matches (the default), the default transport
    (direct delivery) is selected. Routes are evaluated for each delivery attempt.
    They are typically matched on destination address, and can also start matching
    after a configurable number of failed delivery attempts (with another
    transport). (#36 by dmikushin, and earlier discussions on HN)
  • Support more authentication mechanisms in sendmail: SCRAM-SHA-256,
    SCRAM-SHA-1, CRAM-MD5.

Thanks to all who provided feedback, bug reports, patches. You've helped mox
become a better mail server!

To download, see https://github.com/mjl-/mox#download

v0.0.4

04 Jun 20:32
v0.0.4
0187fa0
Compare
Choose a tag to compare

Update instructions: No special steps required other than updating the binary.

Changes:

  • Bugfix: When DKIM-signing messages, take the Date and Message-Id
    header into account if we added them just before. If you submit a message
    without these headers, they are added automatically, but the signature was
    calculated over the message as if those headers were not added. The receiving
    side would evaluate such DKIM signature as invalid.
  • Bugfix: SMTP transactions with SMTP from/to addresses with double quotes
    would fail.
  • Bugfix: Delivery of messages to addresses where the domain does not have an MX
    record would fail. In these cases, e.g. where CNAME records need to be followed
    or where we need to connect directly to the host, we were aborting immediately
    after starting because of a reused canceled Go context.
  • Bugfix: Use MTASTSHTTPS.NonTLS from the config file for the MTASTS config,
    not the similar field for Autoconfig. (#29)
  • Bugfix: If we encountered an error fetching an mta-sts policy as part of a
    delivery attempt, we weren't trying to continue in strict mode, which is safe to
    do in this case.
  • Bugfix: Dropping a message from the queue (with "mox queue drop " or
    through the admin web interface) would be removed from the database, but left in
    the file system. If you still have old message files in data/queue/, they can be
    removed manually.
  • Update to latest bstore database library and add a new index on the Message
    database table, for fast indexed lookups of DKIM reputation for incoming
    messages.
  • Add "mox backup" and "mox verifydata" subcommands. Backup makes a consistent
    snapshot of the databases, message files, and the other files in the data
    directory. Simply copying the DB files while they are in use will lead to
    corrupted files sooner or later, so a consistent snapshot is important. The
    message files are hardlinked when possible, so a snapshot is fast and disk
    space consumption is limited to the database files. The verifydata subcommand
    verifies the database files and messages, useful to verify backups and
    restores. See documentation for these new commands for details.
  • Add support for websockets to the reverse proxy webserver. (#25)
  • After "logout" of an IMAP connection, close the connection. (#30)
  • Open pre existing TLS certificates/keys as the root user, not as the
    unprivileged user. Makes it easier to use these files when set up by other
    tools. (#30)
  • Various smaller technical and user-experience improvements.

Thanks for the feedback, issues and PR's you sent in.
naturalethic, 154pinkchairs, cuu508, inigoserna, kou029w, belst, and those I
forgot. Keep it coming!

To download, see https://github.com/mjl-/mox#download