Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/mailman: make Postfix support optional (provided you configure the MTA yourself) #105397

Merged
merged 1 commit into from Dec 14, 2020

Conversation

@kisik21
Copy link
Contributor

@kisik21 kisik21 commented Nov 29, 2020

Motivation for this change

Fixes #102641. Backward compatibility is provided: merging this PR will not result in any changes to the system besides the manual.

To Do
  • Documentation
    • We need more awareness around cfg.settings.mta for this PR to be useful
    • The new cfg.enablePostfix option should clearly explain how to configure Mailman with other MTAs
  • Error messages may need some refining
  • Tests for Mailman?
    • I think we need to cover Mailman with NixOS tests, including integration with other MTAs. It may be out of scope for this PR though.
How to test this

The following is a configuration.nix file that works the same with or without this PR being merged. You can build it using the tip of this branch as Nixpkgs or the master at the time of filing this PR - the derivation hash should stay EXACTLY the same.

# Build the test configuration from the base branch (pinned for convenience here)
nix-repl> old = ((builtins.getFlake "github:NixOS/nixpkgs/23a5371532aed94099834e46058a7e307c3dda87").lib.nixosSystem { modules = [ ./test-configuration.nix ]; }).config.system.build.vm
# Build from this PR
nix-repl> new = ((builtins.getFlake "github:kisik21/nixpkgs/mailman-other-mta-support").lib.nixosSystem { modules = [ ./test-configuration.nix ]; }).config.system.build.vm
# Ensure the produced output is the same
nix-repl> old == new
true
configuration.nix listing for testing backwards compatibility
{ config, pkgs, lib, ... }:
{
  imports = [
    # Unify flake and non-flake suffixes
    # Taken from https://github.com/NixOS/nixpkgs/issues/101622#issuecomment-716516561
    ({ config, lib, ... }: with lib; {
      system.nixos.revision = mkForce null;
      system.nixos.versionSuffix = mkForce "pre-git";
    })
    # Disable the manual to prevent it from breaking the comparison
    # We only need to compare the functional differences between systems
    # Not the range of available options
    ({ config, lib, ... }: with lib; {
      documentation.nixos.enable = false;
    })
  ];
  services = {
    postfix = {
      enable = true;
      relayDomains = [
        "hash:/var/lib/mailman/data/postfix_domains"
      ];
      config = {
        transport_maps = [
          "hash:/var/lib/mailman/data/postfix_lmtp"
        ];
        local_recipient_maps = [
          "hash:/var/lib/mailman/data/postfix_lmtp"
        ];
      };
    };
    mailman = {
      enable = true;
      siteOwner = "postmaster@fireburn.ru";
    };
  };
}

The following is a rough example of a minimal working Exim with Mailman config. Warning: this is untested, use at your own risk! :3

mailman-with-exim.nix
{ config, lib, pkgs, ... }:
{
  services = {
    mailman = {
      enable = true;
      siteOwner = "postmaster@fireburn.ru";
      enablePostfix = false;
      settings.mta = {
        incoming = "mailman.mta.exim4.LMTP";
        outgoing = "mailman.mta.deliver.deliver";
        lmtp_host = "localhost";
        lmtp_port = "8024";
        smtp_host = "localhost";
        smtp_port = "25";
        configuration = "python:mailman.config.exim4";
      };
    };
    exim = {
      enable = true;
      config = ''
        primary_hostname = mail.fireburn.ru
        domainlist local_domains = fireburn.ru : *.fireburn.ru
        domainlist relay_to_domains = :
        domainlist mm_domains = lists.fireburn.ru
        hostlist relay_from_hosts = <; localhost

        acl_smtp_rcpt = acl_check_rcpt
        .ifdef _HAVE_PRDR
        acl_smtp_data_prdr = acl_check_prdr
        .endif
        acl_smtp_data = acl_check_data

        never_users = root : nobody
        host_lookup = *
        dns_dnssec_ok = 1

        .ifdef _HAVE_PRDR
        prdr_enable = true
        .endif

        qualifyDomain = fireburn.ru
        log_selector = +smtp_protocol_error +smtp_syntax_error +tls_certificate_verified
        ignore_bounce_errors_after = 2d
        timeout_frozen_after = 7d
        check_rfc2047_length = false
        rfc1413_hosts =

        begin acl
        acl_check_rcpt:
          accept hosts = :
                 control = dkim_disable_verify

          deny message = Restricted chars in address
               domains = +local_domains
               local_parts = ^[.] : ^.*[@%!/|]

          deny message = Restricted chars in address
               domains = !+local_domains
               local_parts = ^[./|] : ^.*[@%!] : ^.*/\\.\\./

          accept local_parts = postmaster
                 domains = +local_domains
          require verify = sender

          accept hosts = +relay_from_hosts
                 control = submission
                 control = dkim_disable_verify

          require message = nice hosts say HELO first
                  condition = ''${if def:sender_helo_name}

          require message = relay not permitted
                  domains = +local_domains : +relay_to_domains

          require verify = recipient
          accept

        .ifdef _HAVE_PRDR
        acl_check_prdr:
          warn set acl_m_did_prdr = y
          accept
        .endif

        acl_check_data:
          deny message = max line length exceeded
               condition = ''${if > {$max_received_linelength}{998}}

          deny message = header syntax
               log_message = header syntax ($acl_verify_message)
               !verify = header_syntax

          accept

        begin routers
        dnslookup:
          driver = dnslookup
          domains ! +local_domains
          transport = remote_smtp
          ignore_target_hosts = <; 0.0.0.0 ; 127.0.0.0/8 ; ::1
          dnssec_request_domains = *
          no_more

        system_aliases:
          driver = redirect
          allow_fail
          allow_defer
          data = ''${lookup{$local_part}lsearch{/etc/aliases}}
          user = exim
          file_transport = address_file
          pipe_transport = address_pipe

        mailman3_router:
          driver = accept
          domains = +mm_domains
          require_files = /var/lib/mailman/lists/''${local_part}.''${domain}
          local_part_suffix_optional
          local_part_suffix = \
            -bounces   : -bounces+* : \
            -confirm   : -confirm+* : \
            -join      : -leave     : \
            -owner     : -request   : \
            -subscribe : -unsubscribe
          transport = mailman3_transport

        userforward:
          driver = redirect
          check_local_user
          local_part_suffix = +* : -*
          local_part_suffix_optional
          file = $home/.forward
          allow_filter
          no_verify
          no_expn
          check_ancestor
          file_transport = address_file
          pipe_transport = address_pipe
          reply_transport = address_reply

        localuser:
          driver = accept
          check_local_user
          local_part_suffix = +* : -*
          local_part_suffix_optional
          transport = local_delivery
          cannot_route_message = This user isn't known in these lands

          begin transports
          remote_smtp:
            driver = smtp
            message_size_limit = ''${if > {$max_received_linelength}{998} {1} {0}
          .ifdef _HAVE_DANE
            dnssec_request_domains = *
            hosts_try_dane = *
          .endif
          .ifdef _HAVE_PRDR
            hosts_try_prdr = *
          .endif

          local_delivery:
            driver = appendfile
            directory = $home/Maildir/
            delivery_date_add
            envelope_to_add
            return_path_add
            create_directory
            maildir_format

          address_pipe:
            driver = pipe
            return_fail_output

          address_file:
            driver = appendfile
            delivery_date_add
            envelope_to_add
            return_path_add

          address_reply:
            driver = autoreply

          mailman3_transport:
            driver = smtp
            protocol = lmtp
            allow_localhost
            hosts = localhost
            port = MM3_LMTP_PORT
            rcpt_include_affixes = true

          begin retry
          # Address or Domain    Error       Retries
          # -----------------    -----       -------
          *                      *           F,2h,15m; G,16h,1h,1.5; F,4d,6h
          begin rewrite
          begin authenticators
      '';
    };
  };
}
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS (evaluated only, did not build nor test)
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@kisik21
Copy link
Contributor Author

@kisik21 kisik21 commented Nov 29, 2020

cc @lheckemann as the maintainer of the Mailman module

Copy link
Member

@lheckemann lheckemann left a comment

Just one little nitpick. Otherwise LGTM :)

@kisik21
Copy link
Contributor Author

@kisik21 kisik21 commented Nov 30, 2020

Oh, how about release notes though? I think this kind of a big improvement should be reflected in the Release Notes! Should I do it in this PR too?

@kisik21 kisik21 force-pushed the mailman-other-mta-support branch from 7e0b05e to 9b06527 Nov 30, 2020
@kisik21 kisik21 changed the title WIP: nixos/mailman: make Postfix support optional (provided you configure the MTA yourself) nixos/mailman: make Postfix support optional (provided you configure the MTA yourself) Nov 30, 2020
@kisik21 kisik21 marked this pull request as ready for review Nov 30, 2020
@kisik21 kisik21 requested a review from peti as a code owner Nov 30, 2020
@@ -250,6 +250,18 @@
<title>Other Notable Changes</title>

<itemizedlist>
<listitem>
<para>
The Mailman NixOS module (<literal>services.mailman</literal>)
Copy link
Member

@lheckemann lheckemann Nov 30, 2020

Use <option> for option names, that way they should get linked AFAIU

Copy link
Contributor Author

@kisik21 kisik21 Nov 30, 2020

oh well, I'm running a bisect now, I should make separate worktrees for when I bisect 🤣

don't worry I'll make a new worktree in RAM real quick

@kisik21
Copy link
Contributor Author

@kisik21 kisik21 commented Nov 30, 2020

Oh no, I forgot about the Mailman manual section, it seems! Converting back to draft.

@kisik21 kisik21 marked this pull request as draft Nov 30, 2020
@kisik21 kisik21 force-pushed the mailman-other-mta-support branch from 9b06527 to 45b1f16 Nov 30, 2020
@kisik21
Copy link
Contributor Author

@kisik21 kisik21 commented Nov 30, 2020

The manual fails to build, and I can't debug why. I'm starting to hate Docbook 🤣

@lheckemann
Copy link
Member

@lheckemann lheckemann commented Dec 13, 2020

./release-notes/rl-2103.xml: Not feasibly valid:
Line 256:
   253     <listitem>
   254      <para>
   255       The Mailman NixOS module (<literal>services.mailman</literal>) has a new option
   256       <xref linkend="opt-services.mailman.enablePostfix">services.mailman.enablePostfix</xref>,
   257       defaulting to true, that controls integration with Postfix.
   258      </para>
   259      <para>
     error: text not allowed here;
    expected the element end-tag

An <xref /> has its text generated automatically, if you want to specify it yourself use a <link>…</link> :)

This (I think) helpful error message came from going into nixos/doc/manual and running nix-shell --run make.

@kisik21
Copy link
Contributor Author

@kisik21 kisik21 commented Dec 13, 2020

@lheckemann Indeed you are correct. Magical. Absolutely amazing! I'll push it real quick, and this PR will be perfectly ready to merge.

…the MTA yourself)

Mailman can now work with MTAs other than Postfix. You'll have to configure
it yourself using the options in `services.mailman.settings.mta`.

This addition is reflected in the release notes for 21.03.
@kisik21 kisik21 force-pushed the mailman-other-mta-support branch from 45b1f16 to ad023b0 Dec 13, 2020
@kisik21 kisik21 marked this pull request as ready for review Dec 13, 2020
@lheckemann lheckemann merged commit cc786ac into NixOS:master Dec 14, 2020
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment