Handle TCP/UDP endpoints

[YuukanOO]

Configuring traefik to proxy TCP/UDP requests (https://doc.traefik.io/traefik/routing/routers/#configuring-tcp-routers) instead of just HTTP ones will enable postgres to be correctly exposed for example.

Overview

Determine which kind of router (HTTP, TCP or UDP) should be used to expose a specific service.

For this, we can rely on the port definition in the compose file. For example:

services:
  app:
    image: something
    ports:
      - "8080:80" # Default = HTTP
      - "8081:81/tcp" # TCP
      - "8082:82/udp" # UDP

If not specifically set, seelf will assume it should use the HTTP router. From the docker perspective, when no protocol is defined, it fallbacks to TCP. From our side, we need to distinguish (for traefik purposes) between raw TCP and HTTP specific ones.

I think the most straightforward way, from a user stand point, is to force it to specify the protocol if needed. For example, for exposing a postgres container, one may use the "5432:5432/tcp" port definition.

When loading the compose file project, we can rely on a specific interpolation function to catch the port raw value and check if the protocol has been explicitly defined by the user hence knowing the distinction between HTTP/TCP before docker has fallback to tcp. Something like this:

opts, _ := cli.NewProjectOptions([]string{"compose.yml"},
  cli.WithName("testouille"),
  cli.WithLoadOptions(func(o *loader.Options) {
	o.Interpolate = &interpolation.Options{
		TypeCastMapping: map[tree.Path]interpolation.Cast{
			"services.*.ports.[]": func(value string) (interface{}, error) {
				// Parse the port value definition and check if the protocol is user-defined
				// and save it somewhere to be able to distinguish HTTP/TCP/UDP ones
				// and use the appropriate traefik router.
				return value, nil
			},
		},
	}
}),
  cli.WithNormalization(true),
)

Warning

Due to this, the host mapping part will be mandatory. Without it (ex. just specifying - "8080" and relying on ephemeral ports) we can't distinguish between services and things may break. Services are looped in a non determinist order and the interpolate function does not provide the service being processed.

Apply labels accordingly

The first HTTP exposed port will be handled by the actual proxy using the application subdomain generated. Other ports will define a specific entrypoint, router and service with a unique name to reach that port.

The Service struct will store application service entrypoints. Every non default entrypoints will be saved in the target because it needs to configure them.

When cleaning up an application, we must ensure the mapping on the target side is deleted.

Find a random available port (for TCP/UDP)

If non default entrypoints exists, when configuring a target, we can launch a one off container with ephemeral ports (for every port not mapped yet), retrieve those allocated. This will make sure they are available on the host and leverage Docker.

With those new ports found, we can configure the proxy with all entrypoints added and relaunch it. If the configuration has not changed, Docker should skip the restart.

Note

For now, we will use the same proxy, causing a tiny unavailable period. This will keep the resource usage low but in the future, maybe we can add a configuration option to expose those custom ports on a second proxy to prevent that unavailability.

So when a new deployment expose new TCP or UDP entrypoints, the target will save them and trigger a re-configuration to handle them appropriately.

Note

With this solution, only the proxy know the final url / port of everything. It makes easy to change the URL (as this is the case right now) or the port mapping without having to redeploy everything.

Update the Service struct and the UI

A Service exposed will now have an array of entrypoints with a protocol and subdomain or port and will make them available in the UI so the user can know how to reach those entrypoints (based on the target url).

[YuukanOO]

Almost ready, just need to be tested to make sure every case has been handled.

[sardaukar]

Very cool progress, can't wait for the next release!

[YuukanOO]

The branch feat/tcp-udp-ports is fully functional and can be tested by cloning it and running a docker compose up -d --build at the project root.

I had initially planned a release for today for this feature but since there is a lot of modification going on, I prefer to test it a little bit more. Expect a release at the start of next week.

I should also update the documentation.

[YuukanOO]

🎉 This issue has been resolved in version 2.2.0 🎉

The release is available on:

• v2.2.0
• GitHub release

Your semantic-release bot 📦🚀

[sardaukar]

@YuukanOO congrats on the release! With this and private registries, I'll be able to replace dockge