[POTENTIAL BUG] sed errors in logs after ctrl+c and then up with docker-compose #2241

NorseGaud · 2021-10-05T11:49:35Z

Miscellaneous first checks

I checked that all ports are open and not blocked by my ISP / hosting provider.
I know that SSL errors are likely the result of a wrong setup on the user side and not caused by DMS itself. I'm confident my setup is correct.

Affected Component(s)

Unsure

What happened and when does this occur?

Using master branch (pending 10.2.0) #2217

On the first run of the DMS, logs look fine. I run with docker-compose up --build and become attached to the logs which I then watch for problems as I test the basic sending and receiving.

Things are fine.

Then, I ctrl+c to stop the server so I can start it again detatched:

Stopping deliver ... done
[ec2-user@ip-172-31-3-247 docker-mailserver]$ docker-compose up -d
Starting deliver ... done

But, watching the logs, I see sed errors:

[ TASKLOG ]  Initializing setup
[ TASKLOG ]  Checking configuration
[ TASKLOG ]  Configuring mail server
sievec(root)<43><>: Debug: Loading modules from directory: /usr/lib/dovecot/modules
sievec(root)<43><>: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so
sievec: Debug: Effective uid=0, gid=0, home=/root
sievec: Debug: Quota root: name=User quota backend=count args=
sievec: Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0
sievec: Debug: Quota rule: root=User quota mailbox=Trash bytes=+52428800 messages=0
sievec: Debug: Quota warning: bytes=0 (95%) messages=0 reverse=no command=quota-warning 95 root 
sievec: Debug: Quota warning: bytes=0 (80%) messages=0 reverse=no command=quota-warning 80 root 
sievec: Debug: Quota warning: bytes=0 (100%) messages=0 reverse=yes command=quota-warning below root 
sievec: Debug: Quota grace: root=User quota bytes=0 (10%)
Nameservers 127.0.0.11
Error: sed -i -r s|^(smtpd_tls_chain_files =).*|\1 /etc/letsencrypt/live/xkr.email/privkey.pem /etc/letsencrypt/live/xkr.email/fullchain.pem| /etc/postfix/main.cf
Error: sed -i -r -e s|^(ssl_key =).*|\1 </etc/letsencrypt/live/xkr.email/privkey.pem| -e s|^(ssl_cert =).*|\1 </etc/letsencrypt/live/xkr.email/fullchain.pem| /etc/dovecot/conf.d/10-ssl.conf
[ WARNING ]  Spamassassin is disabled. You can enable it with 'ENABLE_SPAMASSASSIN=1'
[ WARNING ]  Clamav is disabled. You can enable it with 'ENABLE_CLAMAV=1'
sed: can't read /etc/cron.daily/spamassassin: No such file or directory
[ TASKLOG ]  Post-configuration checks
rm: cannot remove '/etc/logrotate.d/clamav-*': No such file or directory
rm: cannot remove '/etc/cron.d/clamav-freshclam': No such file or directory
[  ERROR  ]  Failed to remove ClamAV configuration
rm: cannot remove '/etc/cron.daily/spamassassin': No such file or directory
[  ERROR  ]  Failed to remove SpamAssassin configuration
[ TASKLOG ]  Starting daemons & mail server
cron: started
rsyslog: started
dovecot: started
update-check: started
opendkim: started
opendmarc: started
postfix: started
fail2ban: started
changedetector: started
amavis: started
[ TASKLOG ]  xkr.email is up and running



### What did you expect to happen?

```Markdown
No errors

How do we replicate the issue?

See above

DMS version

master branch (10.2.0)

How much RAM is available to DMS explicitly?

less than 4GB

How many CPU cores are available?

less than 4 Cores

Is DMS running in a virtualized environment?

... a virtual private server (VPS) (with virtual CPU cores)

What operating system is DMS running on?

Linux

What instruction set architecture is DMS running on?

x86_64 / AMD64

I/O - Persistent memory

Docker volume using EFS mount

What container orchestration tool are you using?

Docker Compose

Docker version

N/A

Docker Compose version

No response

The output of `uname -a`

Linux ip-172-31-3-247.us-west-2.compute.internal 4.14.232-176.381.amzn2.x86_64 #1 SMP Wed May 19 00:31:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Important environment variables

version: '3.8'

services:
  mailserver:
    build:
      context: .
    hostname: xkr.email
    domainname: xkr.email
    container_name: deliver
    ports:
      - "25:25"
      - "143:143"
      - "587:587"
      - "993:993"
      - "5002:5002"
    volumes:
      - /mnt/efs-us-west-2/maildata:/var/mail
      - /mnt/efs-us-west-2/mailstate:/var/mail-state
      - ../maillogs:/var/log/mail
      - /mnt/efs-us-west-2/config/:/tmp/docker-mailserver
      - /mnt/efs-us-west-2/config/10-custom.conf:/etc/dovecot/conf.d/10-custom.conf:ro
      - /etc/letsencrypt:/etc/letsencrypt
      - /etc/localtime:/etc/localtime:ro
    environment:
      - SPOOF_PROTECTION=1
      - ENABLE_SPAMASSASSIN=0
      - SPAMASSASSIN_SPAM_TO_INBOX=1
      - ENABLE_CLAMAV=0
      - ENABLE_FAIL2BAN=1
      - ENABLE_POSTGREY=0
      - ENABLE_SASLAUTHD=0
      - ONE_DIR=1
      - DMS_DEBUG=0
      - SSL_TYPE=letsencrypt
      - POSTMASTER_ADDRESS=connarpierce@gmail.com
      - LOGWATCH_INTERVAL=daily
    restart: always
    stop_grace_period: 1m
    cap_add:
      - NET_ADMIN
      - SYS_PTRACE

Relevant log output

No response

Other relevant information

No response

What level of experience do you have with Docker and mail servers?

No response

Code of conduct

I have read this project's Code of Conduct and I agree
I have read the README and the documentation and I searched the issue tracker but could not find a solution

Improvements to this form?

No response

The text was updated successfully, but these errors were encountered:

NorseGaud · 2021-10-05T11:52:57Z

Additionally, some of the stuff has been incorrectly wrapped with code blocks... The new forms we have to fill out seems to need a bit more work.

casperklein · 2021-10-05T12:29:45Z

Then, I ctrl+c to stop the server so I can start it again detatched:

This is not the same as docker-compose down. To get a new container, you have to issue docker-compose down after CTRL-C.

NorseGaud · 2021-10-05T12:31:46Z

Then, I ctrl+c to stop the server so I can start it again detatched:

This is not the same as docker-compose down. To get a new container, you have to issue docker-compose down after CTRL-C.

I don't want a new container though. Is that a requirement?

casperklein · 2021-10-05T12:32:57Z

rm: cannot remove '/etc/logrotate.d/clamav-*': No such file or directory
rm: cannot remove '/etc/cron.d/clamav-freshclam': No such file or directory
rm: cannot remove '/etc/cron.daily/spamassassin': No such file or directory

I am wondering about these errors. Are you sure, you are testing with the latest master branch? This has been fixed with: #2233

casperklein · 2021-10-05T12:34:25Z

I don't want a new container though. Is that a requirement?

No, just best practice. There are probably some more issues when restarting the container, that should be addressed.

casperklein · 2021-10-05T12:36:46Z

Regarding your initial sed errors. They can be ignored. They are just telling you, the files got already altered on "first" container start.

georglauterbach · 2021-10-05T12:52:36Z

I find

sed: can't read /etc/cron.daily/spamassassin: No such file or directory
[ TASKLOG ]  Post-configuration checks
rm: cannot remove '/etc/logrotate.d/clamav-*': No such file or directory
rm: cannot remove '/etc/cron.d/clamav-freshclam': No such file or directory
[  ERROR  ]  Failed to remove ClamAV configuration
rm: cannot remove '/etc/cron.daily/spamassassin': No such file or directory
[  ERROR  ]  Failed to remove SpamAssassin configuration

especially troubling as we already saw this in a previous issue... Why are these files removed, and where?

georglauterbach · 2021-10-05T12:53:51Z

Regarding the forms @NorseGaud, they're still in beta and sometimes not rendered correctly. If you visit https://github.com/docker-mailserver/docker-mailserver/blob/master/.github/ISSUE_TEMPLATE/bug_report.yml you can see that actually as is well with out forms - when GitHub parses them, sometimes the Markdown is malformed.

NorseGaud · 2021-10-05T12:54:38Z

Regarding your initial sed errors. They can be ignored. They are just telling you, the files got already altered on "first" container start.

This seems like a problem if I'm scanning the logs for failures in my monitoring tools (which looks for "error" and other keywords typical to failures). You're right though, the sed error doesn't seem, thus far, to have broken anything. Just seems like doing a check before executing the sed to see if it even needs to run would be better for log monitoring and cleanliness.

georglauterbach · 2021-10-05T12:58:05Z

I'm curious, where do these sed errors come from? I cannot find these lines in our scripts...

casperklein · 2021-10-05T13:15:22Z

These all are no new errors. They have been there for ages. Due to the use of sedfile, now this errors are just made visible 😉

I'm curious, where do these sed errors come from? I cannot find these lines in our scripts...

docker-mailserver/target/scripts/startup/setup-stack.sh

Lines 812 to 818 in cd7677b

    
           sedfile -i -r "s|^(smtpd_tls_chain_files =).*|\1 ${POSTFIX_KEY_WITH_FULLCHAIN}|" "${POSTFIX_CONFIG_MAIN}" 
        
           # Dovecot configuration 
        
           sedfile -i -r \ 
        
             -e "s|^(ssl_key =).*|\1 <${DOVECOT_KEY}|" \ 
        
             -e "s|^(ssl_cert =).*|\1 <${DOVECOT_CERT}|" \ 
        
             "${DOVECOT_CONFIG_SSL}"

NOT sedfile related, but due container restart (file already got removed on first start):

docker-mailserver/target/scripts/startup/setup-stack.sh

Line 1521 in cd7677b

    
           sed -i -e 's|invoke-rc.d spamassassin reload|/etc/init\.d/spamassassin reload|g' /etc/cron.daily/spamassassin

@NorseGaud I am curious, why do you prefer to keep the used container, instead of creating a new one?
PS: See also #2241 (comment) and ensure, you are really using the latest master branch.

NorseGaud added kind/bug meta/needs triage This issue / PR needs checks and verification from maintainers priority/medium labels Oct 5, 2021

NorseGaud changed the title ~~[POTENTIAL BUG]~~ [POTENTIAL BUG] sed errors in logs on down and up docker-compose Oct 5, 2021

NorseGaud changed the title ~~[POTENTIAL BUG] sed errors in logs on down and up docker-compose~~ [POTENTIAL BUG] sed errors in logs after ctrl+c and then up with docker-compose Oct 5, 2021

NorseGaud closed this as completed Oct 6, 2021

casperklein mentioned this issue Oct 6, 2021

release: version v10.2.0 #2217

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[POTENTIAL BUG] sed errors in logs after ctrl+c and then up with docker-compose #2241

[POTENTIAL BUG] sed errors in logs after ctrl+c and then up with docker-compose #2241

NorseGaud commented Oct 5, 2021 •

edited

NorseGaud commented Oct 5, 2021

casperklein commented Oct 5, 2021

NorseGaud commented Oct 5, 2021

casperklein commented Oct 5, 2021

casperklein commented Oct 5, 2021

casperklein commented Oct 5, 2021

georglauterbach commented Oct 5, 2021 •

edited

georglauterbach commented Oct 5, 2021

NorseGaud commented Oct 5, 2021 •

edited

georglauterbach commented Oct 5, 2021

casperklein commented Oct 5, 2021 •

edited

[POTENTIAL BUG] sed errors in logs after ctrl+c and then up with docker-compose #2241

[POTENTIAL BUG] sed errors in logs after ctrl+c and then up with docker-compose #2241

Comments

NorseGaud commented Oct 5, 2021 • edited

Miscellaneous first checks

Affected Component(s)

What happened and when does this occur?

How do we replicate the issue?

DMS version

How much RAM is available to DMS explicitly?

How many CPU cores are available?

Is DMS running in a virtualized environment?

What operating system is DMS running on?

What instruction set architecture is DMS running on?

I/O - Persistent memory

What container orchestration tool are you using?

Docker version

Docker Compose version

The output of uname -a

Important environment variables

Relevant log output

Other relevant information

What level of experience do you have with Docker and mail servers?

Code of conduct

Improvements to this form?

NorseGaud commented Oct 5, 2021

casperklein commented Oct 5, 2021

NorseGaud commented Oct 5, 2021

casperklein commented Oct 5, 2021

casperklein commented Oct 5, 2021

casperklein commented Oct 5, 2021

georglauterbach commented Oct 5, 2021 • edited

georglauterbach commented Oct 5, 2021

NorseGaud commented Oct 5, 2021 • edited

georglauterbach commented Oct 5, 2021

casperklein commented Oct 5, 2021 • edited

NorseGaud commented Oct 5, 2021 •

edited

The output of `uname -a`

georglauterbach commented Oct 5, 2021 •

edited

NorseGaud commented Oct 5, 2021 •

edited

casperklein commented Oct 5, 2021 •

edited