High Availability Deployment

[iammattmartin]

Would love to hear if anyone has extended out postal to a HA setup. With the postal install looking for IPs bound locally, that causes some issue.

Extending out the database/queues shouldn't be too hard but comes down to how mail is sent.

Having single reliance on a VM/server running postal is always a worry. Whilst the underlying hardware (for us) has resilience, there is still the single point of the machine instance itself (crashing badly, software corruption etc etc).

Maybe atech have some secret sauce for when they had a commercial intent for the product to span more than one running/active server.

I originally posted this in #41, so more than a re-post, wondering if anyone has managed it. It has been asked a few times so the question is to the wider community.... or maybe adamcooke (not tagging to avoid pesky emails) has any secret docs for achieving it :)

[willpower232]

It would certainly be good to get some official feedback on the matter.

Postal will send from any IP it can see by default so I presume you're trying to use IP pools so you're worried about IP addresses from that perspective?

It is probably not recommended behaviour but I have had a little mess around with the Procfile to turn off some parts of Postal and fit everything I wanted to test in a low-resources test server and I'm interested in running high availability ingress at least so our systems can at least queue up the email, even if it isn't sent immediately.

[iammattmartin]

Pretty much (in reference to the IPs). We only use internally so would like a specific brand to go out via specific IP(s). i.e some sort of IP matrix where you assign an IP each node to use for each brand. That itself is probably fairly complex.

However, for anyone just using postal as an app based mailer, that probably isn't too much of a concern. Although it would still be nice to have pools of IPs on each node to be specific to marketing, transactional and X, Y, Z mails.

[adamcooke]

I've been playing with running Postal on Kubernetes which means that potential issues with an HA deployment will be raised quite quickly. I'll let you know how I get on.

[iammattmartin]

@adamcooke, thanks for the update. Sounds exciting! :)

[mike-dube]

@adamcooke Any progress on that awesome matters?

Kubernetes is the way to go :)

[outbackdingo]

wow... im hunting for a k8s emails sever solution and found this... does it work ? or still a work in progress?

[willpower232]

It should work for the most part however there are reports of problems with newer OpenSSL versions (#999 and linked issues) so you may encounter some problems but it depends which features you are using.

[outbackdingo]

i found the postal hippo/helm chart works at https://github.com/postalhq/k8s-hippo last night and it did deploy on kubernetes successfully

[outbackdingo]

welp thats a fail.... seesm hippo-k8s postal requires LoadBalancer which seems broken on kubespray with ipvs and layer 2 .... so how can i deploy this on k8s now?

[willpower232]

That repository also includes YAML files which look like they would work directly with kubectl and could be modified to suit your particular configuration style.

As you can see from those files, you essentially just need to run this repo with a config directory appropriately filled in and preferably a separate deployment for each command.

[outbackdingo]

one would thing ingress-nginx would be able to proxy smtp right ?? if it was properly configured

[willpower232]

I believe it is capable of proxying port 25 to the SMTP server deployment but I am not sure how it would work if you want to use port 25 for different applications in the same cluster, you may need a second load balancer or whatever to give you a separate route if that makes sense.

[mfilotto]

@adamcooke A standard helm chart with official image on docker hub would be awesome

[Wouter0100]

I was looking into the Hippo repository, but all links to Hippo seems to be offline. What is Hippo and is there some documentation regarding it? Otherwise a Helm chart would be awesome.

[batonac]

Apparently: https://docs.hippofactory.dev/

[adamcooke]

It was this but we don't use that any longer.

[uhlhosting]

@adamcooke any news on Postsl with K8s?

[smolinari]

@Wouter0100 - Chalk me up as also interested in a Helm chart.

Scott

[Pacerino]

Just pinging here to see if @Wouter0100 released his helm chart?

We're moving everything to Kubernetes.

[Wouter0100]

Unfortunately not yet. We're planning on redoing a lot at some point, after which we may be able to publish it.

[adamcooke]

We will have an official one in the not too distant future.

[Pacerino]

@Wouter0100 Thanks for your reply!
@adamcooke That sound great! I'm looking forward for it :)

[cten]

@willpower232 Does postal need any local permanent disk storage (not DB)? Trying to determine if I should even try to deploy to K8s.

[Wouter0100]

No, it does not.

[cten]

To clarify, only the "worker" sends out emails?

[Wouter0100]

Yes, we have various workers running on VM's started currently manually and do failover using keepalived. All other parts run in K8s.

See here for an earlier explanation.

[willpower232]

I don't think the rails error log is output to stdout but you shouldn't need persistent disk storage with the official image

[BorjaRafolsMartinez]

Anyone ever managed to get IP pools working with HA?

[masihfathi]

Did you find a way for config of haproxy?

[adamcooke]

We run one of our Postal clusters behind HAProxy without any issues. It basically works out of the box. Postal even supports "Proxy Protocol" to pass through IPs properly.