Support for detecting running container mount

[callmemagnus]

This modification proposes to support the running configured mounted volume for configuration.

This removes the need to pass a path from the command line when the container is running and configured in another way that yours.

[johansmitsnl]

Can you rebase to master?

[callmemagnus]

Rebased.

[johansmitsnl]

Tests fail, can you verify them?

[callmemagnus]

Will do

[callmemagnus]

Ok, I'm working on this.

I have successfully corrected all the tests.

I have 2 issues:

1. docker exec mail doveadm auth test -x service=smtp add@localhost.localdomain 'may be \a `p^a.*ssword' | grep 'auth succeeded' fails although it works perfectly when ran on the command line.
2. delete user does not work as expected

I'll push my change to this branch later today.

[johansmitsnl]

@callmemagnus the master merged a commit 59ce9d0 that fixes this issue. Could you rebase again on master to see if it solves the errors?

[callmemagnus]

Thanks for prompt answer @johansmitsnl, but I already had that commit.

The command is failing in the test runner but not manually. I wonder if it is not a strange case of value escaping between my shell, the test suite, docker and the shell in the container.

[callmemagnus]

I don't get it.

Travis returns errors on portions of code I haven't done anything on.

On my laptop, with the current code I have 2 failures, both related to the doveadm working in cli but not in test suite:

 ✓ checking configuration: hostname/domainname
 ✓ checking configuration: hostname/domainname override
 ✓ checking configuration: hostname/domainname override: check container hostname is applied correctly
 ✓ checking configuration: hostname/domainname override: check overriden hostname is applied to all configs
 ✓ checking configuration: hostname/domainname override: check hostname in postfix HELO message
 ✓ checking configuration: hostname/domainname override: check headers of received mail
 ✓ checking process: postfix
 ✓ checking process: clamd
 ✓ checking process: new
 ✓ checking process: opendkim
 ✓ checking process: opendmarc
 ✓ checking process: fail2ban (disabled in default configuration)
 ✓ checking process: fail2ban (fail2ban server enabled)
 ✓ checking process: fetchmail (disabled in default configuration)
 ✓ checking process: fetchmail (fetchmail server enabled)
 ✓ checking process: clamav (clamav disabled by ENABLED_CLAMAV=0)
 ✓ checking process: saslauthd (saslauthd server enabled)
 ✓ checking process: postgrey (disabled in default configuration)
 ✓ checking postgrey: /etc/postfix/main.cf correctly edited
 ✓ checking postgrey: /etc/default/postgrey correctly edited and has the default values
 ✓ checking process: postgrey (postgrey server enabled)
 ✓ checking postgrey: there should be a log entry about a new greylisted e-mail user@external.tld in /var/log/mail/mail.log
 ✓ checking postgrey: there should be a log entry about the retried and passed e-mail user@external.tld in /var/log/mail/mail.log
 ✓ checking postgrey: there should be a log entry about the whitelisted and passed e-mail user@whitelist.tld in /var/log/mail/mail.log
 ✓ checking process: dovecot imaplogin (enabled in default configuration)
 ✓ checking process: dovecot imaplogin (disabled using SMTP_ONLY)
 ✓ checking imap: server is ready with STARTTLS
 ✓ checking imap: authentication works
 ✓ checking imap: added user authentication works
 ✓ checking pop: server is ready
 ✓ checking pop: authentication works
 ✓ checking pop: added user authentication works
 ✓ checking sasl: doveadm auth test works with good password
 ✓ checking sasl: doveadm auth test fails with bad password
 ✓ checking sasl: sasl_passwd exists
 ✓ checking logs: mail related logs should be located in a subdirectory
 ✓ checking smtp: authentication works with good password (plain)
 ✓ checking smtp: authentication fails with wrong password (plain)
 ✓ checking smtp: authentication works with good password (login)
 ✓ checking smtp: authentication fails with wrong password (login)
 ✓ checking smtp: added user authentication works with good password (plain)
 ✓ checking smtp: added user authentication fails with wrong password (plain)
 ✓ checking smtp: added user authentication works with good password (login)
 ✓ checking smtp: added user authentication fails with wrong password (login)
 ✓ checking smtp: delivers mail to existing account
 ✓ checking smtp: delivers mail to existing alias
 ✓ checking smtp: delivers mail to existing alias with recipient delimiter
 ✓ checking smtp: delivers mail to existing catchall
 ✓ checking smtp: delivers mail to regexp alias
 ✓ checking smtp: user1 should have received 6 mails
 ✓ checking smtp: rejects mail to unknown user
 ✓ checking smtp: redirects mail to external aliases
 ✓ checking smtp: rejects spam
 ✓ checking smtp: rejects virus
 ✓ checking smtp_only: mail send should work
 ✓ checking accounts: user accounts
 ✓ checking accounts: user mail folders for user1
 ✓ checking accounts: user mail folders for user2
 ✓ checking accounts: user mail folders for added user
 ✓ checking accounts: comments are not parsed
 ✓ checking postfix: vhost file is correct
 ✓ checking postfix: main.cf overrides
 ✓ checking postfix: master.cf overrides
 ✓ checking dovecot: config additions
 ✓ checking spamassassin: should be listed in amavis when enabled
 ✓ checking spamassassin: should not be listed in amavis when disabled
 ✓ checking spamassassin: docker env variables are set correctly (default)
 ✓ checking spamassassin: docker env variables are set correctly (custom)
 ✓ checking spamassassin: all registered domains should see spam headers
 ✓ checking clamav: should be listed in amavis when enabled
 ✓ checking clamav: should not be listed in amavis when disabled
 ✓ checking clamav: should not be called when disabled
 ✓ checking opendkim: /etc/opendkim/KeyTable should contain 2 entries
 ✓ checking opendkim: /etc/opendkim/KeyTable dummy file generated without keys provided
 ✓ checking opendkim: /etc/opendkim/keys/ should contain 2 entries
 ✓ checking opendkim: generator creates default keys size
 ✓ checking opendkim: generator creates key size 2048
 ✓ checking opendkim: generator creates key size 1024
 ✓ checking opendkim: generator creates keys, tables and TrustedHosts
 ✓ checking opendkim: generator creates keys, tables and TrustedHosts without postfix-accounts.cf
 ✓ checking opendkim: generator creates keys, tables and TrustedHosts without postfix-virtual.cf
 ✓ checking opendkim: generator creates keys, tables and TrustedHosts using domain name
 ✓ checking ssl: generated default cert works correctly
 ✓ checking ssl: lets-encrypt-x3-cross-signed.pem is installed
 ✓ checking ssl: letsencrypt configuration is correct
 ✓ checking ssl: letsencrypt cert works correctly
 ✓ checking ssl: manual configuration is correct
 ✓ checking ssl: manual configuration copied files correctly 
 ✓ checking ssl: manual cert works correctly
 ✓ checking SRS: main.cf entries
 ✓ checking SRS: postsrsd running
 ✓ checking fail2ban: localhost is not banned because ignored
 ✓ checking fail2ban: fail2ban-fail2ban.cf overrides
 ✓ checking fail2ban: fail2ban-jail.cf overrides
 ✓ checking fail2ban: ban ip on multiple failed login
 ✓ checking fail2ban: unban ip works
 ✓ checking postscreen
 ✓ checking fetchmail: gerneral options in fetchmailrc are loaded
 ✓ checking fetchmail: fetchmail.cf is loaded
 ✓ checking system: freshclam cron is enabled
 ✓ checking amavis: virusmail wiper cron exists
 ✓ checking amavis: VIRUSMAILS_DELETE_DELAY override works as expected
 ✓ checking amavis: old virusmail is wipped by cron
 ✓ checking amavis: recent virusmail is not wipped by cron
 ✓ checking system: /var/log/mail/mail.log is error free
 ✓ checking system: /var/log/auth.log is error free
 ✓ checking system: sets the server fqdn
 ✓ checking system: sets the server domain name in /etc/mailname
 ✓ checking system: postfix should not log to syslog
 ✓ checking system: amavis decoders installed and available
 ✓ checking sieve: user1 should have received 1 email in folder INBOX.spam
 ✓ checking manage sieve: server is ready when ENABLE_MANAGESIEVE has been set
 ✓ checking manage sieve: disabled per default
 ✓ checking sieve: user2 should have piped 1 email to /tmp/
 ✓ checking sieve global: user1 should have gotten a copy of his spam mail
 ✓ checking accounts: user_without_domain creation should be rejected since user@domain format is required
 ✓ checking accounts: user3 should have been added to /tmp/docker-mailserver/postfix-accounts.cf
 ✓ checking accounts: auser3 should have been added to /tmp/docker-mailserver/postfix-accounts.cf
 ✓ checking accounts: a.ser3 should have been added to /tmp/docker-mailserver/postfix-accounts.cf
 ✓ checking accounts: user3 should have been removed from /tmp/docker-mailserver/postfix-accounts.cf but not auser3
 ✓ checking user updating password for user in /tmp/docker-mailserver/postfix-accounts.cf
 ✓ checking accounts: listmailuser
 ✓ checking accounts: no error is generated when deleting a user if /tmp/docker-mailserver/postfix-accounts.cf is missing
 ✓ checking accounts: user3 should have been added to /tmp/docker-mailserver/postfix-accounts.cf even when that file does not exist
 ✓ checking PERMIT_DOCKER: can get container ip
 ✓ checking PERMIT_DOCKER: opendmarc/opendkim config
 ✓ checking PERMIT_DOCKER: my network value
 ✓ checking amavis: config overrides
 ✗ checking user login: predefined user can login
   (in test file test/tests.bats, line 1181)
     `run docker exec mail doveadm auth test -x service=smtp pass@localhost.localdomain 'may be \a `p^a.*ssword' | grep 'auth succeeded'' failed
 ✓ checking setup.sh: Without arguments: status 1, show help text
 ✓ checking setup.sh: Wrong arguments
 ✗ checking setup.sh: setup.sh email add
   (in test file test/tests.bats, line 1210)
     `run docker exec mail doveadm auth test -x service=smtp setup_email_add@example.com 'test_password' | grep 'auth succeeded'' failed
 ✓ checking setup.sh: setup.sh email list
 ✓ checking setup.sh: setup.sh email update
 ✓ checking setup.sh: setup.sh email del
 ✓ checking setup.sh: setup.sh email restrict
 ✓ checking setup.sh: setup.sh alias list
 ✓ checking setup.sh: setup.sh alias add
 ✓ checking setup.sh: setup.sh alias del
 ✓ checking setup.sh: setup.sh config dkim
 ✓ checking setup.sh: setup.sh debug fetchmail
 ✓ checking setup.sh: setup.sh debug inspect
 ✓ checking setup.sh: setup.sh debug login ls
 ✓ checking setup.sh: setup.sh debug fail2ban
 ✓ checking setup.sh: setup.sh relay add-domain
 ✓ checking setup.sh: setup.sh relay add-auth
 ✓ checking setup.sh: setup.sh relay exclude-domain
 ✓ checking postfix: ldap lookup works correctly
 ✓ checking postfix: ldap custom config files copied
 ✓ checking postfix: ldap config overwrites success
 ✓ checking postfix: remove privacy details of the sender
 ✓ checking dovecot: ldap imap connection and authentication works
 ✓ checking dovecot: ldap mail delivery works
 ✓ checking dovecot: ldap mail delivery works for a different domain then the mailserver
 ✓ checking dovecot: ldap config overwrites success
 ✓ checking dovecot: postmaster address
 ✓ checking spoofing: rejects sender forging
 ✓ checking spoofing: accepts sending as alias
 ✓ checking saslauthd: sasl ldap authentication works
 ✓ checking saslauthd: ldap smtp authentication
 ✓ checking dovecot: ldap rimap connection and authentication works
 ✓ checking saslauthd: sasl rimap authentication works
 ✓ checking saslauthd: rimap smtp authentication
 ✓ checking postfix-lmtp: virtual_transport config is set
 ✓ checking postfix-lmtp: delivers mail to existing account
 ✓ checking pflogsum delivery
 ✓ checking dovecot: only A grade TLS ciphers are used
 ✓ checking dovecot: nmap produces no warnings on TLS ciphers verifying
 ✓ checking postfix submission: only A grade TLS ciphers are used
 ✓ checking postfix submission: nmap produces no warnings on TLS ciphers verifying
 ✓ checking postfix smtps: only A grade TLS ciphers are used
 ✓ checking postfix smtps: nmap produces no warnings on TLS ciphers verifying
 ✓ checking restart of process: postfix
 ✓ checking restart of process: clamd
 ✓ checking restart of process: amavisd-new
 ✓ checking restart of process: opendkim
 ✓ checking restart of process: opendmarc
 ✓ checking restart of process: fail2ban (fail2ban server enabled)
 ✓ checking restart of process: fetchmail
 ✓ checking restart of process: clamav (clamav disabled by ENABLED_CLAMAV=0)
 ✓ checking restart of process: saslauthd (saslauthd server enabled)
 ✓ checking relay hosts: default mapping is added from env vars
 ✓ checking relay hosts: custom mapping is added from file
 ✓ checking relay hosts: ignored domain is not added
 ✓ checking relay hosts: auth entry is added
 ✓ checking relay hosts: default auth entry is added

186 tests, 2 failures
Makefile:260: recipe for target 'tests' failed
make: *** [tests] Error 1

I also added my fork to travis and the errors are the same than the ones I get on my machine: https://travis-ci.org/callmemagnus/docker-mailserver/jobs/366510534.

[callmemagnus]

I updated the branch but travis does not start this build. Can you trigger it ? I have the tests passing on my branch (see #950).

[johansmitsnl]

Restarted jenkins

[17Halbe]

Did you check that initialpass is not empty? I had timing problems with it before. Also -c mail doesn't have an effect as the addmailuser script is running on a temporary container.

[callmemagnus]

What you say about -c mail is not correct. My change is exactly changing that as it guesses the mounted volume of a container.

But you are right about the value of initialpass. I need to check if setup.sh is returning when the task is done or not.

Anyway, the job on travis does not start on this PR :-(

[johansmitsnl]

Restarted the build

[callmemagnus]

hmmm... Broken build again, this is getting frustrating. I think Travis is definitely showing its limit with this build.