Skip to content

discovery cluster connection with multiple DNS entry does not work if first node is down #414

Open
@innocentiv

Description

@innocentiv

🐛 Current behavior

I am using the latest version of kurrentDB node client (1.0.1) to connect to a cluster of 3 eventstoredb machines (v23.10).

I resolve the cluster using this connection string:

kurrentdb+discover://user:pass@01.com:2113,02.com:2113,03.com:2113

If 02 or 03 are down the client connection works perfectly fine. If 01 is down I cannot connect with error:

[Error [UnavailableError]: UnavailableError] { metadata: {} }

This is caused by the nodejs client using only the first host for discovery:

https://github.com/kurrent-io/KurrentDB-Client-NodeJS/blob/master/packages/db-client/src/Client/index.ts

if (options.dnsDiscover) {
      const [discover] = options.hosts;

      if (options.hosts.length > 1) {
        debug.connection(
          `More than one address provided for discovery. Using first: ${discover.address}:${discover.port}.`
        );
      }

      return new Client(
        rustClient,
        {
          discover,
          nodePreference: options.nodePreference,
          discoveryInterval: options.discoveryInterval,
          gossipTimeout: options.gossipTimeout,
          maxDiscoverAttempts: options.maxDiscoverAttempts,
          throwOnAppendFailure: options.throwOnAppendFailure,
          keepAliveInterval: options.keepAliveInterval,
          keepAliveTimeout: options.keepAliveTimeout,
          defaultDeadline: options.defaultDeadline,
          connectionName: options.connectionName,
        },
        channelCredentials,
        options.defaultCredentials
      );
    }

According to a discussion on kurrent discord general channel this is a bug on the client. I would like to better understand what is the correct approach to maintain availability on a 2n+1 cluster when the first node is down. This is also necessary to enable the incremental upgrade of a running cluster.

🔍 Steps to reproduce

  • setup a cluster of 3 nodes
  • add dns entries for each of the nodes
  • setup a client to connect to the 3 node via discovery
  • bring down the first node
  • the client unable to connect to the cluster

Reproducible link

https://codesandbox.io/p/devbox/w5v9md

💭 Expected behavior

When the first node in the connection string is down, and the other 2 nodes are up, I can connect to the cluster anyway.

Package version

KurrentDB-NodeJS-Client 1.0.1

KurrentDB Version

EventstoreDB 23.10

Connection string

kurrentdb+discover://user:pass@01.com:2113,02.com:2113,03.com:2113

☁️ Deployment Environment

Multi-node cluster (Cloud)

Other Deployment Details

No response

Operating system

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions