autoconfig support

[lupine]

Configuring MUAs is made much easier with support for automatic configuration. Mozilla's de-facto standard for this is quite widely used: https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration

I'm currently manually maintaining one of these at https://autoconfig.ur.gs/.well-known/autoconfig/mail/config-v1.1.xml

There are DNS-based standards too, but I don't think they're as widely used.

We'll need a HTTP(S) listener for automatic TLS, activesync, webmail, etc, support too, so I don't think it's a big deal to add one.

[foxcpp]

How to determine public hostname to expose in auto-config?
I think we should just let the user specify all details:

http http://0.0.0.0:8888 {
  autoconfig {
    display_name "ur.gs Email Service"
    incoming imap.ur.gs 143 starttls
    outgoing smtp.ur.gs 465 tls
  }
}

(need to think more about configuration structure)

Also, there are few different formats used by Outlook and Apple Mail: https://github.com/L11R/go-autoconfig
What should we do with them?

[lupine]

We can just serve the expected file for each of them, I guess. I'm not very familiar with the MS and Apple ecosystems, but I can't imagine they're publishing anything fundamentally different - just varying structures.

[emersion]

I think we should just let the user specify all details:

I don't really like incoming and outgoing. The configured servers could automatically be picked up.

Also, we might want to offer an option for people who don't want an embedded web server. For instance by writing files to a directory instead of serving them.

We might want to re-use the configured mail server HTTPS certificates (if no reverse proxy is used).

We also need to consider MTA-STS, which also requires a webserver.

(As an alternative to go-autoconfig, which doesn't seem to expose a real library (yet?))

• https://github.com/ProtonMail/go-apple-mobileconfig
• https://github.com/ProtonMail/go-outlook-autodiscover

[foxcpp]

Something like that could work as a generic base for HTTP-related static stuff:

http https://0.0.0.0 { # can use TLS in the same way as other endpoint modules
  tls ... # including per-endpoint override option

  <generator module reference>
  <generator module reference>
  <generator module reference>
}

# Write out generators results into directory.
http_dir directory_path {
  <generator module reference>
  <generator module reference>
  <generator module reference>
}

type HTTPGenerator interface {
  // Returns map Path -> Blob. Usually called once during initialization.
  Generate(params *config.Node) (map[string][]byte, error)
}

For dynamic contents we might allow "generator" module to implement http.Hander (???).

...TLS, activesync, webmail, etc, support...

I really doubt the idea of including webmail in maddy, tho.

[foxcpp]

Regarding auto-auto-config:

This is kinda tricky IMO and on the edge of being "too clever to be useful".

Unless the user explicitly defines endpoint information to use - pick first IMAP and Submission endpoint instances. Prefer Implicit TLS over STARTTLS. Use hostname directive (either local to autoconfig or global) for actual domains.

[emersion]

Something like that could work as a generic base for HTTP-related static stuff

Hmm. I guess the alternative would be to have a very simple filesystem interface (Open(path string) (io.Reader, error) and Create(path string) (io.WriteCloser, error)) backed by a "real" filesystem in case of http_dir and backed by an in-memory store implementing http.Handler in case of http. Gah, while this allows for dynamic updates, scratch that, it's getting too complicated.

It's probably better to use your design, and extend it if we need.

This is kinda tricky IMO and on the edge of being "too clever to be useful".

Yeah, maybe... But it's really annoying to specify again and manually all connection parameters for such a minor thing.

I really doubt the idea of including webmail in maddy, tho.

Yeah, webmail is out-of-scope IMHO. Maybe JMAP (or another API standard, /shrug) can help users to wire up a third-party webmail…

[foxcpp]

Side note: JMAP requires an HTTP endpoint too. So we need dynamic content support.

[foxcpp]

Different approach to HTTP endpoint module interfaces.
It doesn't require keeping a separate "registry" for HTTP-related modules. Instead of HTTP endpoint module calling "attached" modules, modules that need HTTP call HTTP endpoint module interfaces.

type HTTPEndpoint interface {
  AddHandler(pattern string, h http.Handler) error
  AddStaticResource(path string, data io.Reader) error
}

I'm not sure about special handling for http_dir and static directory, probably we should drop it, it is not that hard to setup reverse proxy.

For example, automatic TLS configuration using CertMagic (#3):

certmagic {
  ...
  challenge https https://0.0.0.0:443
}

imap ... {
  tls auto
}

http https://0.0.0.0:443 {
  tls auto  
}

Other thing, JMAP.

jmap {
  session_endpoint https://0.0.0.0:443/.well-known/jmap
  blob_endpoint https://0.0.0.0:443/.well-known/jmap/data
}
http https://0.0.0.0:443 {
  tls auto  
}

...

[emersion]

AddStaticResource(path string, data io.Reader) error

This won't do. An endpoint needs to be able to serve the same contents multiple times.

However everything can be done with an http.Handler, so it's no big deal.

I'm not sure about special handling for http_dir and static directory, probably we should drop it, it is not that hard to setup reverse proxy.

Yeah, it's not like there's a big overhead in including an HTTP server.

For example, automatic TLS configuration using CertMagic

The general idea LGTM. Maybe the exact syntax could be improved (e.g. jmap https://0.0.0.0:443).

[foxcpp]

What I think about automatic configuration: it helps only in exotic setups because in more simple cases MUA can just probe standard ports. So guessing of parameters for auto-config is essentially useless. Also, this generally limits usefulness since server admins usually avoid non-standard configurations to improve interoperability.

[lupine]

For mobile clients, autoconfig is super-valuable - it saves a lot of awkward data entry, even when the values are obvious and easy to guess / memorized.

[emersion]

I'd just say it's not high priority at all. As @foxcpp said people can always add various autoconfig file themselves right now.

What annoys me is that there's no standard. The world would be a better place if people used SRV records.

[onny]

Currently I'm using the project go-autoconfig as an additional web service to support autoconfig as described here https://nixos.wiki/wiki/Maddy#Autoconfig