Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support DNS based addresses for nodes #74

Open
matdehaast opened this issue Jan 28, 2022 · 2 comments
Open

Support DNS based addresses for nodes #74

matdehaast opened this issue Jan 28, 2022 · 2 comments

Comments

@matdehaast
Copy link
Contributor

matdehaast commented Jan 28, 2022

Currently Tigerbeetle only supports IP based addresses when starting nodes. This make it very difficult to work with in cloud native enviroments that use DNS based routing.

In order to work better in these environments, Tigerbeetle should allow DNS based strings in the addresses flag

@lewisdaly
Copy link
Contributor

Yep - I've just learned this problem myself. I was trying to specify a docker-compose file for local development, and the only way I could get each replica to find each other while also respecting the addresses field was to use the host network instead of the local docker network.

Here's an example docker-compose file in case anyone else runs into this problem:

version: "3.7"

##
# Local example of a 3-node tigerbeetle instance
# 
# Note: this example might only work with linux + using `network_mode:host` because of 2 reasons:
# 
# 1. When specifying an internal docker network, other containers are only available using dns based routing:
#    e.g. from tigerbeetle_0, the other replicas are available at `tigerbeetle_1:3002` and
#    `tigerbeetle_2:3003` respectively.
#
# 2. Tigerbeetle performs some validation of the ip address provided in the `--addresses` parameter
#    and won't let us specify a custom domain name.
#
# The workaround for now is to use `network_mode:host` in the containers instead of specifying our
# own internal docker network
##

services:
  tigerbeetle_0:
    container_name: tigerbeetle_0
    image: tigerbeetle:local
    command: "sh -c './tigerbeetle start --addresses=0.0.0.0:3001,0.0.0.0:3002,0.0.0.0:3003 0_0.tigerbeetle'"
    ports:
      - "3001"
    network_mode: host
    volumes:
      - /opt/beta-beetle/

  tigerbeetle_1:
    container_name: tigerbeetle_1
    image: tigerbeetle:local
    command: "sh -c './tigerbeetle start --addresses=0.0.0.0:3001,0.0.0.0:3002,0.0.0.0:3003 0_1.tigerbeetle'"
    ports:
      - "3002"
    network_mode: host
    volumes:
      - /opt/beta-beetle/

  tigerbeetle_2:
    container_name: tigerbeetle_2
    image: tigerbeetle:local
    command: "sh -c './tigerbeetle start --addresses=0.0.0.0:3001,0.0.0.0:3002,0.0.0.0:3003 0_2.tigerbeetle'"
    ports:
      - "3003"
    network_mode: host
    volumes:
      - /opt/beta-beetle/

@matdehaast
Copy link
Contributor Author

matdehaast commented Sep 7, 2022

@lewisdaly how we are currently getting around this is using a nginx proxy sidecar

Its crude but you can see an example of it here

@sentientwaffle sentientwaffle mentioned this issue Nov 18, 2022
92 tasks
@cb22 cb22 mentioned this issue Oct 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants