Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

You should not run your mail server because mail is hard. #16

Open
poolpOrg opened this issue Aug 31, 2019 · 34 comments

Comments

@poolpOrg
Copy link
Owner

commented Aug 31, 2019

No description provided.

@poolpOrg poolpOrg self-assigned this Aug 31, 2019

@goloroden

This comment has been minimized.

Copy link

commented Sep 1, 2019

Hey there 😊

First of all, thanks for your article, and thanks for cleaning up with a few myths! That's a good thing, and please don't take what follows as criticism on your article, but rather as some further thoughts on it…

As you said, running the mail server itself actually isn't that hard. What is hard is all the things around it:

  • You have to backup it regularly.
  • You have to care about updates and security updates.
  • You have to run two servers for failover and keep them in sync.
  • You have to care about updates and security updates for the underlying OS.
  • And so on…

As long as running a mail server is not your core business, I would avoid doing all that on my own. I'm happily paying for Office 365 (or any other hosted solution out there) that let's me focus on my actual job, which I get paid for.

Because, running your own git server isn't hard, too. Or running your own messaging service. Or running … but it all sums up.

And if in the end I have to spend hours per month or per week to manage my infrastructure, I'd rather pay for it – as said, unless it's my core competency.

So yes, running a mail server isn't hard. Anyway, it's nothing I would want to do if I can avoid it.

Just my 2 cents 😉

Golo

@dubvfan87

This comment has been minimized.

Copy link

commented Sep 1, 2019

I 100% agree with @goloroden. I worked for a company previously with an IT admin who thought he can run our email in house. Our email server would go down at least once a month and on the outside it just seemed like he was incompetent. I knew better (that email is hard), but the CEO just fired the guy. The next guy, who wasn't nearly as knowledgeable just bought O365 and everyone was happy to not have emails sent to them bounced.

Also you never touched on reputation. Without it most of your mail will end up in receivers spam folders. Good luck figuring out why outlook SMTP servers are not accepting your mail. Did someone somewhere in the world flag an email from your IP as spam?

I'm not convinced. Sure it's easy to actually install postfix. Howver, not being able to provide 99.9999% uptime on email could be your ass. Why risk it?

@cayblood

This comment has been minimized.

Copy link

commented Sep 1, 2019

I care enough about decentralization, data privacy and sovereignty, that I would gladly pay for a turnkey self-deployed solution in the form of an AWS marketplace offering or DigitalOcean droplet. BUT it seems to me that an even bigger problem than hosting mail is the lack of clients with Gmail's capabilities. If someone developed a Gmail clone frontend for such a solution, I would gladly pay a hefty monthly fee for it. So much of my life is on Gmail, and Google is so secretive about when and how they suddenly shut off accounts, that I would much rather manage my own email, but the alternatives lack feature parity.

@myfirstnameispaul

This comment has been minimized.

Copy link

commented Sep 1, 2019

@cayblood What feature of GMail has keeps you tethered to their service? Is it just the webmail interface?

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

commented Sep 1, 2019

@golorden

Hey there 😊

Hey,

First of all, thanks for your article, and thanks for cleaning up with a few myths! That's a good thing, and please don't take what follows as criticism on your article, but rather as some further thoughts on it…

Thanks, and don't worry, I wouldn't write and allow comments if I didn't want people to interact and contradict :-)

As you said, running the mail server itself actually isn't that hard. What is hard is all the things around it:

  • You have to backup it regularly.
  • You have to care about updates and security updates.
  • You have to run two servers for failover and keep them in sync.
  • You have to care about updates and security updates for the underlying OS.
  • And so on…

Yes, however backups, updates, failover, these are all not mail specific and you supposedly have to handle them for any service you run.

I'm not saying mail is easy, I'm saying it's not hard, it's not harder than running other services, it's just... work.

I have daily backups, they cover user home directories, mail, websites, databases, etc... mail is not handled any different. I monitor all services equally when it comes to updates and security, mail does not get a more special treatment than my web server which people will never mention as being "hard".

Yes it is harder to maintain your services and ensure they are up than to outsource them, but harder doesn't equate to hard in my opinion.

As long as running a mail server is not your core business, I would avoid doing all that on my own. I'm happily paying for Office 365 (or any other hosted solution out there) that let's me focus on my actual job, which I get paid for.

I'm not saying you shouldn't, if this is the best option to you then go for it, I'm not against Big Mailer Corps.

What I'm against is the trend that because mail is considered something hard, the immediate solution is to move everyone to Big Mailer Corps without assessing if they would be just fine part of a smaller provider, hosted on a shared server, self-hosted on the family server, etc...

This gives them so much power that it's akin to giving them the power to decide what they want to do with that protocol disregarding what's in the interest of the community.

Because, running your own git server isn't hard, too. Or running your own messaging service. Or running … but it all sums up.

Yes, people have to make choice, some would rather outsource everything, others will want to host everything, and others will want to pick what they will work on, but my point remains:

By claiming that it's hard, which it is not, people are discouraging others from even attempting and seeing by themselves, and what bothers me is when this is done by hearsay.

And if in the end I have to spend hours per month or per week to manage my infrastructure, I'd rather pay for it – as said, unless it's my core competency.

There are also many reasons that aren't technical, why this isn't a good thing for all.

I'm sure the Iranians that got kicked out of Github because US decided that sanctions should apply aren't that thrilled about a world where e-mail is fully controlled by 3 or 4 US companies that could essentially not only kill their e-mail but also prevent them from communicating with most of the world.

I dislike that idea profoundly and it's not far fetched.

So yes, running a mail server isn't hard. Anyway, it's nothing I would want to do if I can avoid it.

Just my 2 cents 😉

Well, thanks for your comment :-)

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

commented Sep 1, 2019

@dubvfan87

I 100% agree with @goloroden. I worked for a company previously with an IT admin who thought he can run our email in house. Our email server would go down at least once a month and on the outside it just seemed like he was incompetent. I knew better (that email is hard), but the CEO just fired the guy. The next guy, who wasn't nearly as knowledgeable just bought O365 and everyone was happy to not have emails sent to them bounced.

On the other end of the spectrum is a company I worked for who bought O365 and switched to Gmail, and another who bought Gmail and switch to self-hosting.

Heck, a month or so ago I helped a team unbreak their mail setup which was broken by proofpoint in front of O365, I got all of the pain from third-party hosting without any of the benefits.

Different experiences, but in all of them I still don't thing mail is hard: it is work, yes, but work is not necessarily hard, it is not necessarily constant and it is not necessarily time consuming on the long run once you know what you're doing.

Also you never touched on reputation. Without it most of your mail will end up in receivers spam folders. Good luck figuring out why outlook SMTP servers are not accepting your mail. Did someone somewhere in the world flag an email from your IP as spam?

I didn't touch on reputation because by experience it is irrelevant to most people, reputation is only an issue when you send mail to larger volumes of people and my post wasn't about bulk sending.

Outlook is a different beast, I accepted that it takes time to inbox from them.

It is when you try hard to work around their spam that you actually make things worse, once you accept that for some time you're going to warn people about looking in their spam folders, it'll eventually get better.

Other Big Mailer Corps are essentially no problem for small senders, I can inbox any Big Mailer Corps with a basic setup and, to be transparent, when I used to work in a borderline industry, I could easily inbox pretty much any Big Mailer Corps with not so much work even if they were actively blocking me. If I could do that with the volumes and kind of trafic I was sending, I think most people should easily inbox everywhere (but outlook, who will spambox for a while) given they do the minimum work.

I'm not convinced. Sure it's easy to actually install postfix. Howver, not being able to provide 99.9999% uptime on email could be your ass. Why risk it?

Because I don't have uptime issues and I'd really rather have control over my mail and not depend on a company that can terminate my account the next day if they so wanted.

@gwlperl

This comment has been minimized.

Copy link

commented Sep 1, 2019

First, thank you poolpOrg for you contribution to the open source internet.

I ran my own mail system from 2002-2015. Started out with just my own domains and then hosted domains for friends and family. Variations of Qmail plus postfix plus ASSP plus custom scripts and DoveCot IMAP etc. Ya'll know the drill. I've a few friends that have done the same thing over the years, and all of them have quit as well. They all say the same thing -- it got to be more work that it was worth. And as the base price of email is "zero" (even though we're now finding out that "free" isn't so "free) it's hard to make money on it.

Then came protonmail. They do everything better (and I mean everything) than I could ever and for not much $ at all. Do I regret all those years of running an email system? No, the experience of keeping up with the technology and the internet kept me on my technical toes, a constant learning experience. That experience helped my career.

So yes, you can run an email system. If you don't know how, but are interested in it, then by all means set one up. (The cloud is cheap). I have no regrets.

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

commented Sep 1, 2019

@gwlperl

First, thank you poolOrg for you contribution to the open source internet.

Thanks for reading and commenting ;-)

I ran my own mail system from 2002-2015. Started out with just my own domains and then hosted domains for friends and family. Variations of Qmail plus postfix plus ASSP plus custom scripts and DoveCot IMAP etc. Ya'll know the drill. I've a few friends that have done the same thing over the years, and all of them have quit as well. They all say the same thing -- it got to be more work that it was worth. And as the base price of email is "zero" (even though we're now finding out that "free" isn't so "free) it's hard to make money on it.

We should wonder if people are quitting because they're missing a bit of information that would help them understand why they find it hard. Like the fact that SPF/DKIM are mandatory, like the fact that you should have a valid rDNS + FCrDNS and in a ideal world a matching HELO name.

How comes a lot of us, postmasters, manage to handle their mails for decades with minimum maintenance (the last time I had to deal with a block for my own server was over two years ago, otherwise I don't think I ever do mail stuff ... outside deploying new code for testing), while others seem to hit pretty much any blocklist, get blocked at every major host, etc...

MOST blocks and junking come from a mistake to start with, something that degraded reputation or that you were not allowed to do (like contacting a spam trap). The way mail works requires a bit of doing something bad over and over again to actually be punished.

Sometimes you are a collateral damage, like my block from two years ago, but this gets fixed easily and doesn't happen every two days.

Then came protonmail. They do everything better (and I mean everything) than I could ever and for not much $ at all. Do I regret all those years of running an email system? No, the experience of keeping up with the technology and the internet kept me on my technical toes, a constant learning experience. That experience helped my career.

If you feel like protonmail is the proper choice for you, then you made the good choice :-)

I don't advocate for everyone to self-host, I advocate for people to give it a try if they want to do it rather than give up because others told them it's hard, and I advocate for people to spread across multiple hosts and not concentrate in the three or four top hosts that are all known for their monopolies in other areas.

So yes, you can run an email system. If you don't know how, but are interested in it, then by all means set one up. (The cloud is cheap). I have no regrets.

I have a self-hosted address, I have addresses at various hosts, we need them all !

@gwlperl

This comment has been minimized.

Copy link

commented Sep 1, 2019

PoolpOrg writes: We should wonder if people are quitting because they're missing a bit of information that would help them understand why they find it hard. Like the fact that SPF/DKIM are mandatory, like the fact that you should have a valid rDNS + FCrDNS and in a ideal world a matching HELO name.

I remember over the years, learning some new things the "hard way". (SPF record? What's that?) For me the learning part was the reward, for others it's independence and freedom from the "big mailcorps". And I think we agree - for whatever reason you decide to run one, actually running your own email server helps keep us all independent and free, so I salute all of you. It's not for everyone, but everyone benefits (except big mail corps :-)

@dm17

This comment has been minimized.

Copy link

commented Sep 1, 2019

It would be great if a bunch of mail server experts got together and put together a docker-compose or swarm that is well-refined! It would also help pool optimizations & recommended documentation between mail server tooling. I would be happy to help test it :)

@GaryGapinski

This comment has been minimized.

Copy link

commented Sep 2, 2019

@poolpOrg : very nice article. I agree with your assessments.

With one minor difference: I'd rather deflect bad actors than see them continually show up in logs. This is one of the things I use (just updated to add explanatory comments).

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

commented Sep 2, 2019

@GaryGapinski

@poolpOrg : very nice article. I agree with your assessments.

Thanks !

With one minor difference: I'd rather deflect bad actors than see them continually show up in logs. This is one of the things I use (just updated to add explanatory comments).

I like seeing them in my logs myself because I test filters on them, they're my tamagotchi :-p

@christhomas

This comment has been minimized.

Copy link

commented Sep 2, 2019

ok, so maybe this is a stupid thing to ask, but where are the installation instructions? https://www.opensmtpd.org/

Having built a mail server which runs on kubernetes (https://github.com/kubernetes-mail-server) I can say the biggest problem that I had was that there are so many working parts and none of them are really explained very well.

The man-pages are either 90% of what you need and the 10% that's missing is what you really need, but nobody thought it was important to write down. Or that options are described in very technical terms, but that doesn't mean anything to you specifically, so you google around for weeks trying to find out, how this option affects me, what does it do which I can't glean from reading a highly technical explanation.

Then you have all the programs, and ports, and pipes and files everywhere, written in different formats, each multiplying the problem of bad documentation (even after 20 years) that explains only the bare minimum.

Then you have the problem of IP addresses, mail servers are quite sensitive to them and resolving to the correct one isn't necessarily so easy if you try to run behind a firewall or a proxy, then you have to take care that you accept email where the SOURCE IP and not the FIREWALL IP, that bit me a few times before I realised what was happening. But not because it was explained. But because I sat down and really drove into the problem of why spam was happening.

Then you have the problem of restrictions, in postfix, which is the correct set of restrictions. Is there a page on postfix.org which says "PUT THESE RESTRICTIONS AND YOU'RE GOLDEN". Nope! It doesn't. But it does have a man page going into several hundred words explaining each option and what it does. But do you and have you the confidence to put them together in the right order and get it right? This also bit me in the ass a few times before I realised there is actually a right way and a wrong way.

I think the problem comes that nobody wants to tell you what a good "policy" is because this is open source, here are a bunch of engine parts. Go make a sports car! Don't ask me the right way to build it. You do you and you'll be fine. Except this isn't true. There are sometimes right ways and wrong ways and sometimes making decisions which cover 90% of the situations is better than not doing this in the spirit of "not dictating to others what or how to do things".

Does anybody know how to host multiple websites, with multiple SSL certificates per domain? Postfix says to run postfix-multi, but did you know that dovecot supports submission now? But have you configured it before? It has very little docs on it, but when it works, it's great. Then you can add as many domains as you want with as many SSL certs as you want without all the complexity of running one MTA per SSL cert. But I might be out of date cause I'm not certain whether it's the only way to do it. It even works nicely with LetsEncrypt certs that you can reuse for the domain website if you configure it properly.

Then when I managed to finalise a working mail server from all of these engine parts. I encoded it and allowed you to change a few of the options, many others you can only change if you edit the code. I'm dictating policy because I know that other people can't and other people don't have 1000 hours to read every single page on postfix or dovecots website.

So I don't entirely agree that mail isn't hard. I think it gets easier when you spend time with it. But if you try with zero experience to set-up a mail server. You'll fail for weeks before you succeed. Either that or you use somebody else's preconfigured solution and that solves your problem and you never really built it yourself in the first place.

@christhomas

This comment has been minimized.

Copy link

commented Sep 2, 2019

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

commented Sep 2, 2019

@christhomas

ok, so maybe this is a stupid thing to ask, but where are the installation instructions? https://www.opensmtpd.org/

Having built a mail server which runs on kubernetes (https://github.com/kubernetes-mail-server) I can say the biggest problem that I had was that there are so many working parts and none of them are really explained very well.

OpenSMTPD is an OpenBSD software, it is distributed with the system.

On other systems, the portable archive should come with a README providing details on how to install:

https://github.com/OpenSMTPD/OpenSMTPD/blob/portable/README.md

Note however that OpenSMTPD depends on LibreSSL as of latest stable release, so if you want it to use OpenSSL, you'll need to get the development branch of wait for next stable release which is due in a few weeks.

The man-pages are either 90% of what you need and the 10% that's missing is what you really need, but nobody thought it was important to write down. Or that options are described in very technical terms, but that doesn't mean anything to you specifically, so you google around for weeks trying to find out, how this option affects me, what does it do which I can't glean from reading a highly technical explanation.

OpenBSD projects are fully documented in their man pages which are often reworked to make things clearer, provide examples, and such:

https://opensmtpd.org/manual.html

The smtpd.conf man page will provide multiple examples of common setups.

Then you have all the programs, and ports, and pipes and files everywhere, written in different formats, each multiplying the problem of bad documentation (even after 20 years) that explains only the bare minimum.

I don't understand this, sorry.

Myself, I have multiple simple setups with 10 lines configuration files and I have complex setups which involve multiple machines with segregated roles, relaying to each other, with configuration files that don't exceed 10 lines either.

They all use the same software, there's only one file to control the software, it's in a straightforward format.

Then you have the problem of IP addresses, mail servers are quite sensitive to them and resolving to the correct one isn't necessarily so easy if you try to run behind a firewall or a proxy, then you have to take care that you accept email where the SOURCE IP and not the FIREWALL IP, that bit me a few times before I realised what was happening. But not because it was explained. But because I sat down and really drove into the problem of why spam was happening.

Then you have the problem of restrictions, in postfix, which is the correct set of restrictions. Is there a page on postfix.org which says "PUT THESE RESTRICTIONS AND YOU'RE GOLDEN". Nope! It doesn't. But it does have a man page going into several hundred words explaining each option and what it does. But do you and have you the confidence to put them together in the right order and get it right? This also bit me in the ass a few times before I realised there is actually a right way and a wrong way.

I think we have a different terminology.

When I say it's not hard, I don't mean that it's a two click thing that doesn't require work. I mean that it's not hard in the sense that "you can get it running relatively fast and it won't need you to spend an hour a day on it".

You still need to learn whatever software you chose, some being harder than others, you still need to know basic networking and some of the key points behind the protocols you're going to deploy. The same is true for HTTP, the same is true for DNS, the same is true for anything you setup to face Internet.

Setting up a mail server requires work, it requires preparation, none of which is hard, but all of which is mandatory to get things going. I have seen people that have gone from zero to running in a few hours and that can now do it in a few minutes.

I think the problem comes that nobody wants to tell you what a good "policy" is because this is open source, here are a bunch of engine parts. Go make a sports car! Don't ask me the right way to build it. You do you and you'll be fine. Except this isn't true. There are sometimes right ways and wrong ways and sometimes making decisions which cover 90% of the situations is better than not doing this in the spirit of "not dictating to others what or how to do things".

I don't get that, the rules are very widespread:

  • rDNS and forward-confirmed rDNS are mandatory
  • you need to have an SPF record
  • you need to DKIM sign your mails and publish your public key in DNS

the first two points are trivial, the third one requires a google search to know how to generate a DKIM key.

I can literally do that in less than 2 minutes and this is not because I'm particularly skilled.

Sure you'd take some time doing it the first time, but does it qualify as hard ?

Does anybody know how to host multiple websites, with multiple SSL certificates per domain? Postfix says to run postfix-multi, but did you know that dovecot supports submission now? But have you configured it before? It has very little docs on it, but when it works, it's great. Then you can add as many domains as you want with as many SSL certs as you want without all the complexity of running one MTA per SSL cert. But I might be out of date cause I'm not certain whether it's the only way to do it. It even works nicely with LetsEncrypt certs that you can reuse for the domain website if you configure it properly.

I'm not a Postfix user and generally you will always find cases harder than others, but:

  • because there are cases harder than others doesn't mean the whole idea is hard either
  • maybe there are alternatives to Postfix where this is much easier because what's hard here is the software's way of doing it, not the task at hand

Then when I managed to finalise a working mail server from all of these engine parts. I encoded it and allowed you to change a few of the options, many others you can only change if you edit the code. I'm dictating policy because I know that other people can't and other people don't have 1000 hours to read every single page on postfix or dovecots website.

So I don't entirely agree that mail isn't hard. I think it gets easier when you spend time with it. But if you try with zero experience to set-up a mail server. You'll fail for weeks before you succeed. Either that or you use somebody else's preconfigured solution and that solves your problem and you never really built it yourself in the first place.

I disagree with you:

I've seen people failing for hours before succeeding, they now run servers that don't require maintenance and that plain works.

work != hard

@christhomas

This comment has been minimized.

Copy link

commented Sep 2, 2019

@christhomas

ok, so maybe this is a stupid thing to ask, but where are the installation instructions? https://www.opensmtpd.org/
Having built a mail server which runs on kubernetes (https://github.com/kubernetes-mail-server) I can say the biggest problem that I had was that there are so many working parts and none of them are really explained very well.

OpenSMTPD is an OpenBSD software, it is distributed with the system.

On other systems, the portable archive should come with a README providing details on how to install:

https://github.com/OpenSMTPD/OpenSMTPD/blob/portable/README.md

I was referring to the website, where there is barely any information at all, not even a page saying much more than "here is a link to the man pages"

Note however that OpenSMTPD depends on LibreSSL as of latest stable release, so if you want it to use OpenSSL, you'll need to get the development branch of wait for next stable release which is due in a few weeks.

The man-pages are either 90% of what you need and the 10% that's missing is what you really need, but nobody thought it was important to write down. Or that options are described in very technical terms, but that doesn't mean anything to you specifically, so you google around for weeks trying to find out, how this option affects me, what does it do which I can't glean from reading a highly technical explanation.

OpenBSD projects are fully documented in their man pages which are often reworked to make things clearer, provide examples, and such:

https://opensmtpd.org/manual.html

The smtpd.conf man page will provide multiple examples of common setups.

I was referring to mail servers in general, like your article does.

Then you have all the programs, and ports, and pipes and files everywhere, written in different formats, each multiplying the problem of bad documentation (even after 20 years) that explains only the bare minimum.

I don't understand this, sorry.

The problem of configuring mail servers comes down to the ... then add what I said.

Myself, I have multiple simple setups with 10 lines configuration files and I have complex setups which involve multiple machines with segregated roles, relaying to each other, with configuration files that don't exceed 10 lines either.

You obviously don't use postfix or dovecot. Their base configuration is probably 10x longer than that. I think you're reading my words in a highly focused way on opensmtpd instead of against the articles context, which is mail servers in general

They all use the same software, there's only one file to control the software, it's in a straightforward format.

Then you have the problem of IP addresses, mail servers are quite sensitive to them and resolving to the correct one isn't necessarily so easy if you try to run behind a firewall or a proxy, then you have to take care that you accept email where the SOURCE IP and not the FIREWALL IP, that bit me a few times before I realised what was happening. But not because it was explained. But because I sat down and really drove into the problem of why spam was happening.
Then you have the problem of restrictions, in postfix, which is the correct set of restrictions. Is there a page on postfix.org which says "PUT THESE RESTRICTIONS AND YOU'RE GOLDEN". Nope! It doesn't. But it does have a man page going into several hundred words explaining each option and what it does. But do you and have you the confidence to put them together in the right order and get it right? This also bit me in the ass a few times before I realised there is actually a right way and a wrong way.

I think we have a different terminology.

When I say it's not hard, I don't mean that it's a two click thing that doesn't require work. I mean that it's not hard in the sense that "you can get it running relatively fast and it won't need you to spend an hour a day on it".

This isn't anywhere near true. In order to get postfix, dovecot, ssl, the various databases or configurations needed and setup for anti spam itself if are you on a typical system will take much more than that. Nobody on this entire planet with zero experience, can setup a mail server in one hour a day. It's just not possible.

You still need to learn whatever software you chose, some being harder than others, you still need to know basic networking and some of the key points behind the protocols you're going to deploy. The same is true for HTTP, the same is true for DNS, the same is true for anything you setup to face Internet.

Setting up a mail server requires work, it requires preparation, none of which is hard, but all of which is mandatory to get things going. I have seen people that have gone from zero to running in a few hours and that can now do it in a few minutes.

yes, which is what I am also saying, I'm also saying that it requires a lot of background knowledge of a lot of things which are not expliclty explained, or require you to just know things from experience. If you try to set these things up learning as you go. It's a very painful experience.

I think the problem comes that nobody wants to tell you what a good "policy" is because this is open source, here are a bunch of engine parts. Go make a sports car! Don't ask me the right way to build it. You do you and you'll be fine. Except this isn't true. There are sometimes right ways and wrong ways and sometimes making decisions which cover 90% of the situations is better than not doing this in the spirit of "not dictating to others what or how to do things".

I don't get that, the rules are very widespread:

  • rDNS and forward-confirmed rDNS are mandatory
  • you need to have an SPF record
  • you need to DKIM sign your mails and publish your public key in DNS

try installing postfix from scratch and see how many of those tickboxes it checks out of the box, without you needing to do much. Then you'll see that this, whilst is good advice, is something that you then have to spend hours configuring and reading about because some configuration option you didn't think was important, has the wrong value and nothing tells you apart from on some mailing list you'll find it after 3 hours of googling.

the first two points are trivial, the third one requires a google search to know how to generate a DKIM key.

Then you need to know how to install it, opendkim's installation is easy enough, but getting it to generate the right keys in a scalable way means you need to either build a file dynamically or run a database. Then you have to read how to setup opendkim with a database, then you have to generate the keys, etc, etc. Thats more hours of work.

I can literally do that in less than 2 minutes and this is not because I'm particularly skilled.

No you can't. To install and configure all the parts necessary, with my knowledge I have right now, would take around 1-2 hours. Without experience. It'll take a lot longer. I think you're exaggerating to say it'd take you 2 minutes

Remember, the point of this is that email isn't hard. You've never setup a server before, but it's not crazy difficult. You'll be able to do it. But we both know that the only reason you think it's easy is because of your vast experience. It's not because it is technically easy if you don't have that vast experience to rely on.

Sure you'd take some time doing it the first time, but does it qualify as hard ?

yes, it does, because doing it the first time requires building the knowledge of what all the options do, how the parts go together, how to get your MTA to talk with opendkim, how to configure opendkim to use a database, or how are you going to build the static file? use a script? okay cool, which script? there isn't one, now you have to write a script cause nobody else has a drop in script ready to go. etc. etc. etc

Does anybody know how to host multiple websites, with multiple SSL certificates per domain? Postfix says to run postfix-multi, but did you know that dovecot supports submission now? But have you configured it before? It has very little docs on it, but when it works, it's great. Then you can add as many domains as you want with as many SSL certs as you want without all the complexity of running one MTA per SSL cert. But I might be out of date cause I'm not certain whether it's the only way to do it. It even works nicely with LetsEncrypt certs that you can reuse for the domain website if you configure it properly.

I'm not a Postfix user and generally you will always find cases harder than others, but:

Right and we're talking about mail servers right? Linux people will opt for postfix. I've never heard of opensmtpd and by the look at the website it's a bit on the bare bones side in terms of docs and unless you're a unix expert. You're going to have a very difficult weekend.

  • because there are cases harder than others doesn't mean the whole idea is hard either
  • maybe there are alternatives to Postfix where this is much easier because what's hard here is the software's way of doing it, not the task at hand

But again, we're talking about setting up mail servers with all the parts necessary. Maybe opensmtpd is super easy. But most people run postfix/dovecot and perhaps that is colouring your and my judgement of the same coin in different ways?

Then when I managed to finalise a working mail server from all of these engine parts. I encoded it and allowed you to change a few of the options, many others you can only change if you edit the code. I'm dictating policy because I know that other people can't and other people don't have 1000 hours to read every single page on postfix or dovecots website.
So I don't entirely agree that mail isn't hard. I think it gets easier when you spend time with it. But if you try with zero experience to set-up a mail server. You'll fail for weeks before you succeed. Either that or you use somebody else's preconfigured solution and that solves your problem and you never really built it yourself in the first place.

I disagree with you:

I've seen people failing for hours before succeeding, they now run servers that don't require maintenance and that plain works.

work != hard

Reading 300 tabs of various websites, mailing lists, archives, stackoverflow for several days or over an extended period of time means that it becomes annoying, difficult, laborious, the payoff gets smaller as you go, the frustration increases as you can't find a decent explanation of various options, etc.

So you're right, work != hard, but we're talking about the EFFORT it takes and whether people are willing to do it. I think a lot of people don't do it because of the reasons I've stated above and I think there are quite a lot of extremely frustrated people out there who might agree, if they had the chance to also comment their experience here for us to read.

@myfirstnameispaul

This comment has been minimized.

Copy link

commented Sep 2, 2019

@poolpOrg When it comes to mail servers, the modern dev and tech crowd have difficulty seeing the forest through the trees. You may have more success trying to reach a different audience.

The Mail-in-a-Box project (MiaB) often has people posting to their forum that I can tell have little experience with server management and occasionally zero experience with command line. The community there is ready, willing, and able to provide the minimal assistance needed to get them off and running, and they rarely come back to ask more questions.

Somewhere, out there, is a group of people who wants to hear your message.

What if it is small law firms dealing with government corruption cases? What if it is cannabis dispensaries who keep getting their accounts switched off? What if it is remote communities that just want a reliable way to communicate with one another, even if their www connection is unreliable?

I feel if you expand outside the industry you've worked in, you may discover broader and better opportunities for finding people receptive to your message.

@dm17

This comment has been minimized.

Copy link

commented Sep 2, 2019

@dm17 checkout: https://github.com/kubernetes-mail-server

Though you just said previously that you had troubles with it? Will do though... Multiple SSL certs per site? I'd use letsencrypt nginx companion.

@christhomas

This comment has been minimized.

Copy link

commented Sep 2, 2019

@dm17 checkout: https://github.com/kubernetes-mail-server

Though you just said previously that you had troubles with it? Will do though... Multiple SSL certs per site? I'd use letsencrypt nginx companion.

Not multiple SSL certs "per site" but a single dovecot serving multiple domains for email with separate SSL certs.

Maybe we didn't understand each other, the setup doesn't have problems, I was talking about the issues of setting up postfix with multiple SSL certs and how I solved it using dovecot with submission and it's SSL cert configuration allows SSL certs per domain using SNI over port 587

@christhomas

This comment has been minimized.

Copy link

commented Sep 2, 2019

@poolpOrg When it comes to mail servers, the modern dev and tech crowd have difficulty seeing the forest through the trees. You may have more success trying to reach a different audience.

The Mail-in-a-Box project (MiaB) often has people posting to their forum that I can tell have little experience with server management and occasionally zero experience with command line. The community there is ready, willing, and able to provide the minimal assistance needed to get them off and running, and they rarely come back to ask more questions.

Somewhere, out there, is a group of people who wants to hear your message.

What if it is small law firms dealing with government corruption cases? What if it is cannabis dispensaries who keep getting their accounts switched off? What if it is remote communities that just want a reliable way to communicate with one another, even if their www connection is unreliable?

I feel if you expand outside the industry you've worked in, you may discover broader and better opportunities for finding people receptive to your message.

I agree with what you're saying, I just want to point out that I've been using linux since around 1996, so I'm not one of the "modern dev and tech crowd" you're referring to. I'm pretty knowledgable when it comes to linux and yet I still had to read for hours to get things working cause things didn't work exactly as described and the docs were lacking.

@myfirstnameispaul

This comment has been minimized.

Copy link

commented Sep 2, 2019

@christhomas "Modern" would encompass most of anyone in the tech crowd today, without consideration to where or when they entered.

@christhomas

This comment has been minimized.

Copy link

commented Sep 2, 2019

I agree, I just wanted to point out that this isn't a problem because I don't have experience with the command line or lack of experience. But this is just a problem in general with a lot of server-side software which badly lacks good and well-written documentation.

@GaryGapinski

This comment has been minimized.

Copy link

commented Sep 4, 2019

So you're right, work != hard, but we're talking about the EFFORT it takes and whether people are willing to do it.

I think @christhomas has aptly identified the difficulty. Not hard, but not effortless.

@binarykitchen

This comment has been minimized.

Copy link

commented Sep 4, 2019

@christhomas

This comment has been minimized.

Copy link

commented Sep 4, 2019

They literally did about four comments up @binarykitchen :/

@binarykitchen

This comment has been minimized.

Copy link

commented Sep 4, 2019

Ugh, I've mistakenly searched for mailinabox :)

@ashiq54689

This comment has been minimized.

Copy link

commented Sep 5, 2019

BUT it seems to me that an even bigger problem than hosting mail is the lack of clients with Gmail's capabilities. If someone developed a Gmail clone frontend for such a solution, I would gladly pay a hefty monthly fee for it.

This is very very true from the user perspective. Although Roundcube 1.4 is coming, and the new default UI is much better compared to the previous version, but still nowhere near Gmail or Outlook web interface.

@ngirard

This comment has been minimized.

Copy link

commented Sep 5, 2019

Hi there,

can't wait to read your next article !

I wish to suggest updating your CSS rules just a bit, because the line length of the text is unconstrained, making it difficult to read on a maximized / large window.

As the 58 bytes of css to look great nearly everywhere article suggests, it can be as simple as adding something like

main {
  max-width: 38rem;
  padding: 2rem;
  margin: auto;
}

I'd have been glad to submit this as a pull request to your poolpOrg.github.io repo, but the HTML skeleton of your site has nothing I can hook it up to.

Maybe consider using e.g. <div class="content">, <div class="main">, or even <article>, and while you're at it, replacing your header and footer divs with the standard <header> and <footer> tags ?

Cheers from Paris, and don't hesitate to drop me a mail, I'd be happy to buy you a coffee !

@plgruener

This comment has been minimized.

Copy link

commented Sep 5, 2019

You will never reach “absolute 0 spam”, it was proven mathematically in the 2000s

I would actually be very interested in this proof (or an outline). Any tip for further research?

@ngirard

This comment has been minimized.

Copy link

commented Sep 5, 2019

You will never reach “absolute 0 spam”, it was proven mathematically in the 2000s

I would actually be very interested in this proof (or an outline). Any tip for further research?

See e.g.

Banday, M. Tariq, and Tariq R. Jan. “Effectiveness and Limitations of Statistical Spam Filters.” ArXiv:0910.2540 [Cs], October 14, 2009. http://arxiv.org/abs/0910.2540.

@myfirstnameispaul

This comment has been minimized.

Copy link

commented Sep 5, 2019

Also see spamsolutions.txt[1]

[1] https://craphound.com/spamsolutions.txt

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

commented Sep 5, 2019

@ashiq54689

This is very very true from the user perspective. Although Roundcube 1.4 is coming, and the new default UI is much better compared to the previous version, but still nowhere near Gmail or Outlook web interface.

I use Rainloop which is quite fine but I agree with this, now a lot of people tell me that they don't use a webmail and read on their smartphones, but it would be nice to have a choice of good UI for webmail. Sadly, I have no UI skills :-)

@ngirard

I wish to suggest updating your CSS rules just a bit, because the line length of the text is unconstrained, making it difficult to read on a maximized / large window.

thanks, will apply, I suck at anything graphic so this is helpful :-)

I'd have been glad to submit this as a pull request to your poolpOrg.github.io repo, but the HTML skeleton of your site has nothing I can hook it up to.

Maybe consider using e.g. <div class="content">, <div class="main">, or even <article>, and while you're at it, replacing your header and footer divs with the standard <header> and <footer> tags ?

will take some time this week-end to rework the website a bit, I had planned to switch to a different static generator, so that might be the occasion :-)

Cheers from Paris, and don't hesitate to drop me a mail, I'd be happy to buy you a coffee !

sure thanks !

@plgruener @ngirard

You will never reach “absolute 0 spam”, it was proven mathematically in the 2000s

I would actually be very interested in this proof (or an outline). Any tip for further research?

See e.g.

Banday, M. Tariq, and Tariq R. Jan. “Effectiveness and Limitations of Statistical Spam Filters.” ArXiv:0910.2540 [Cs], October 14, 2009. http://arxiv.org/abs/0910.2540.

This was not the one I was refering to, but i'll go read it :-)

Basically, a paper demonstrated that virus detection could be highly effective but not reach 100% and another paper demonstrated that virus detection and spam detection share the same characteristics. I'll try to find it back but don't hold your breath because these days I'm under water and the papers I'm referring to date from 2002/2003 IIRC

@ashiq54689

This comment has been minimized.

Copy link

commented Sep 5, 2019

I use Rainloop which is quite fine but I agree with this, now a lot of people tell me that they don't use a webmail and read on their smartphones, but it would be nice to have a choice of good UI for webmail. Sadly, I have no UI skills :-)

I use Rainloop too. But I am waiting for Roundcube 1.4. Here are some screenshots of the new Roundcube.

Screen Shot 2019-09-06 at 1 19 16 AM

Screen Shot 2019-09-06 at 1 19 02 AM

@richardfive

This comment has been minimized.

Copy link

commented Sep 13, 2019

Just found your article which was referenced form another site. Agree that it is not hard to do but you do need to put in some elbow grease. I've done it all myself om a mac and documented the hell out of it on http://diymacserver.com and learned all the gritty details on DNS and SMPT, TLS and whatsoever for which I'm still grateful. In the end I got frustrated by keeping it up all the time with the ever changing software and config issues. Now a run a VPS with https://mailinabox.email/ which takes all of the configuration and administrative maintenance out of the equation and only lets you enjoy having your own server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.