ConnectionTimeoutError: Connection timeout

[younes-io]

I get this error and I don't understand why:

How do I get more details about this:

/mnt/c/dev/client-adapter/node_modules/redis/dist/lib/socket.js:157
            socket.setTimeout(__classPrivateFieldGet(this, _RedisSocket_options, "f").connectTimeout, () => socket.destroy(new errors_1.ConnectionTimeoutError()));
                                                                                                                           ^
ConnectionTimeoutError: Connection timeout
    at Socket.<anonymous> (/mnt/c/dev/client-adapter/node_modules/redis/dist/lib/socket.js:157:124)
    at Object.onceWrapper (node:events:513:28)
    at Socket.emit (node:events:394:28)
    at Socket.emit (node:domain:475:12)
    at Socket._onTimeout (node:net:486:8)
    at listOnTimeout (node:internal/timers:557:17)
    at processTimers (node:internal/timers:500:7)
[nodemon] app crashed - waiting for file changes before starting...

---

Environment

─❯ node -v 
v16.6.2

─❯ redis-server -v 
Redis server v=5.0.7 sha=00000000:0 malloc=jemalloc-5.2.1 bits=64 build=636cde3b5c7a3923

─❯ lsb_release -a   
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal

[leibale]

Can you please share the configuration you send to createClient (without the server, username, and password)?

[leibale]

@younes-io?

[younes-io]

Sorry @leibale for the late response.

Actually, I did just as mentioned in the docs:

createClient({
  url: 'redis://alice:foobared@awesome.redis.server:6380',
});

I realized that the error is thrown in these cases:

• Using redis without TLS (my Redis connection uses TLS)
• Using the user default or when omitting the user:

createClient({
  url: 'rediss://:foobared@awesome.redis.server:6380',
});

[eladitzhakian]

I'm having the same problem.

Using node-redis 4.0.0
Redis server image v4-alpine
Connects just fine with redis-cli but getting ConnectionTimeout using node-redis
This is how I connect:

createClient({ url: 'redis://localhost:6379' })

[leibale]

@eladitzhakian

docker run -it -p 6379:6379 redis:4-alpine

import { createClient } from 'redis';

const client = createClient({ url: 'redis://localhost:6379' });

await client.connect();
await client.ping();
await client.disconnect();

works for me (the classic answer :P)... wanna debug it together?

[romulovalez]

I reproduced this issue in two different projects, but I find it difficult to reproduce it on a basic server, If you want we can debug it together 👍

[Chocobozzz]

Hello,

We had this issue on our project. The root cause is that we try to .connect(), and then require many node modules blocking the node loop with sync calls. The socket is blocked and the timeout is thrown. I suggest you to profile your applications to find if it's the same kind of error.

[a1003584293]

Hello,

We had this issue on our project. The root cause is that we try to .connect(), and then require many node modules blocking the node loop with sync calls. The socket is blocked and the timeout is thrown. I suggest you to profile your applications to find if it's the same kind of error.

you are right, Should not be used .connect when using createcluster api

[Raraku]

Hi so, what's the fix?

[aarhusgregersen]

I am facing the same issue as a result of upgrading from v3 to v4. Everything is working just fine, so I am little confused as to why I'm seeing the error.

@Chocobozzz how did you find out about this? And I am also a bit perplexed by the node loop being blocked with sync calls. I mean, how do you know which requires are sync, and which aren't?
This does not necessarily seem to correlate directly to sync/async code, in the classical async/await sense.

[Chocobozzz]

@aarhusgregersen All requires are sync. You can profile your application using chrome about:inspect and nodejs --inspect-brk option

[menocomp]

The fix here is to increase the timout of the socket as follow:

const redisCluster = createCluster({
  rootNodes: [
    {
      url: '', // your host and port here
    },
  ],
  defaults: {
    socket: {
      connectTimeout: 50000,
    },
  },
});

[aarhusgregersen]

@menocomp great suggestion!

However, to my knowledge, I am not using a cluster and I therefore don't think the solution applies to me :-(

[Raraku]

Hi,
I got this issue because of race conditions; it was an older project, one without top level await so I couldn't follow the recommended setup. I fixed it by using return new Promise to ensure that the client connection is established before any calls are made

[aarhusgregersen]

That's great @Raraku ! Thank you. I have the same restrictions, so perhaps this will work for me too.

I will confirm and get back.

[aarhusgregersen]

Running the command node --inspect dist/app.js with no issues indicates to me that there is a typescript compiler issue (perhaps from ts-node) going on in my case, moreso than it being an actual issue with redis.

[typhoon11]

What is the fix for this I am using it in TS and sometimes it work and sometimes this error comes

[kabapy]

same Problem here ... connection timeout on my windows 10 dev machine

[kabapy]

i think it may be related to redis configuration in the protected Mode
in redis conf try disabling protected mode

protected-mode no

[kabapy]

on further investigation

i tried binding to all interfaces instead of loopback, leaving protected mode on ... this is working so far

bind * -::*
protected-mode yes

[depak379mandal]

Got the may Be,

According to me this is issue because of race condition of async tasks. So to overcome that we can just go with await or we can connect to socket or server after

 redisClient.connect().then(() => {
 // other tasks
})

May be this solves issues then I am glad.

[kabapy]

Unfortunately i couldn't resolve this issue
I ended up using "ioredis" package instead!!

[maximilize]

I also had to switch to ioredis since I could not find the source of the timeout error. Connections via redis-cli or netcat always worked, but via redis.connect() it had a failure rate of around 25%.

Switching to ioredis was very quick and easy.

[leibale]

@kabapy @maximilize The ConnectionTimeout error is thrown by the underlying node.js socket API, (both packages call net.connect under the hood, which throws this error). I'm pretty sure that you don't see the error in ioredis not because it's not happening, but because it's hiding the error from you and just retiring in the background (see #2058 (comment) for example)

[kabapy]

@kabapy @maximilize The ConnectionTimeout error is thrown by the underlying node.js socket API, (both packages call net.connect under the hood, which throws this error). I'm pretty sure that you don't see the error in ioredis not because it's not happening, but because it's hiding the error from you and just retry in the background (see #2058 (comment) for example)

I think this behavior should be implemented in this lib. Not in my code!

[leibale]

@kabapy it is implemented, it's emitting an 'error' to let you know something is not working as expected, it does not means the client is not reconnecting..

What do you prefer:

1. Not knowing an error has happened + the package takes care of it
2. Knowing an error has happened + the package takes care of it

BTW, one difference between the packages that I've found now is the default connectTimeout:
5s in this package
10s in ioredis
So if your event loop is blocked for 5-10sec you'll see an error here but not with ioredis.

[kabapy]

@leibale that is correct, but I couldn't catch this error in my try...catch block. I end up with a hanging process or a crashed one if my app is dependant on redis and can't run without it. Even with pm2 I couldn't remedy this issue with automatic restart.

I used the lib in a project 2 years ago, and I never faced this issue on the same system. I don't know if it is an issue related to specific node version or a lib implementation problem.

I agree with you that the developer should know that a problem has occured. But this problem should be catched and corrected either by the library or the user. With the unpredictable behavior of the problem and the absence of a working solution, we had no course of action except switching to another lib.

[leibale]

@kabapy in order to catch this error you gave to handle error events. This is not specific to this package, this is how node.js event emitter works. Just make sure to add client.on('error', err => ...);

Before calling client.connect and the app won't crash on timeout.

BTW, there is already an issue + PR to add a note to the docs (#2302).

[kabapy]

@leibale I did try that and even tried wrapping it in async function and use try catch block. it do sometime catch the error and I have to terminate the process because the app can't run without redis. Sometime it just hangs.

I am sorry, the problem is indeed tricky. I wish I could be of more help.

I remembered now that my old project used the redis library indirectly through socket.io.redis 5.2.0 lib. And it never failed to connect

[leibale]

@kabapy you had an error listener and the app still exists with "Connection Timeout" error? This should be impossible.. you have a code to reproduce the error maybe?

"Try catch" won't catch event based errors

[kabapy]

@leibale I am sorry that project is not active right now. I didn't use any fancy setup. Just followed documentation.

It just connected to mongo and redis before starting the http server. I used redis mainly as a caching layer.

As I described.. The problem is unpredictable some days it didn't happen during development and some days I spent half an hour restarting and tinkering with redis server.

I managed to reproduce the issue during beta testing, On windows 2016 server environment. I hoped that it was my dev machine that caused the issue, But I was wrong.

In this thread many decribed having similar issue with the library. I hope some of them can shed some light on this issue.

Again, I am sorry, I wish i can be of more help

[leibale]

@kabapy no worries, thanks for trying!
I'm not too sure what happened to you, but a lot of users are missing the fact that the error listener is mandatory, and we definitely need to make this more obvious. I hope that the docs update in #2373 will solve it.

[maximilize]

I'm using the error listener also for the ioredis package, no errors so far. I had already a connection timeout of 10 seconds with node-redis, but it didn't help. Via redis-cli or netcat the connection time is <10ms since redis is running in the same cluster. What I also noticed is that the app startup is way faster now, and not a single error.

I suspect there is a weird error somewhere. If it may help, we use URL's as connection string like redis://redis:6379/0.

[leibale]

@maximilize wanna have a quick meeting and show me the error? I tried to reproduce it so many times but never manage to do it, if you'll be able to help it'll be amazing! I'll wait in this meeting for the next 30 minutes.

edit:
on top of that, can you please share:

1. The error (with the stack trace)?
2. What changed in the startup time? The app connects to redis faster?

[ManarArabi]

@leibale Did you managed to solved it?

[leibale]

@ManarArabi I didn't manage to reproduce it even.. do you get this error? Can you help me reproduce it?

[ManarArabi]

@leibale for me after several hours of debugging, it worked after I connect with the IP directly and increase the timeout to 50000
this is node-redis v4.6.1

RedisClient = createClient({
  url: `redis://${process.env.REDIS_CLIENT_HOST}:${process.env.REDIS_CLIENT_PORT}`,
  username: process.env.REDIS_USERNAME,
  password: process.env.REDIS_PASSWORD,
  socket: {
    connectTimeout: 50000,
  },
});

[leibale]

@ManarArabi 50 seconds to connect?! I think that your event loop is blocked for too long on startup, that's why you get these errors. See this comment.
I would love to debug it together, if you can please join this call, I'll wait there for the next 15 minutes, hope you'll join :)

BTW, I'm pretty sure that connecting to the IP/hostname will give the same result, can you try with connectTimeout 50, but with the hostname (not IP)?

[denizdogan]

Has anyone here tried using the pingInterval option?

createClient({
  url: "redis://localhost:6379",
  pingInterval: 1000,
})

I am using node-redis (via redis-om) with Deno+Fresh, and a clean Redis Stack container running in a local Docker. Here's what I did, it seems to have solved the issue for now. Hope this helps someone.

import { Client } from "npm:redis-om"
import { createClient } from "npm:redis"

const nodeClient = createClient({
  url: "redis://localhost:6379",
  pingInterval: 1000,
})
await nodeClient.connect()
const omClient = new Client()
const redis = await omClient.use(nodeClient)

export { redis }

[SZharkov]

Adding pingInterval fixed read ETIMEDOUT error. Thanks

[younes-io]

@leibale this issue is still labelled "Pending Author Input"; is that still relevant?

[matt-halliday]

I don't know how, but pingInterval also fixed it here...

[leibale]

@younes-io We need a way to reproduce the issue...
@matt-halliday some Redis providers (Azure for example) will close an idle connection without respecting TCP keep alive, so in some providers, you'll have to issue a command in intervals to make sure that the socket won't close.

[matt-halliday]

It's definitely odd - we have had a redis cache implemented in our production environment for many months and no issue there... we added a cache to our non-prod environments a week ago and no issues with it until today... this is running in AWS on Kubernetes (EKS) with Redis (Elasticache). Some pods were starting up fine first try, others after one or two attempts, some went into CrashLoopBackoff. The application hadn't changed... and neither had the environment as far as I know.

Certainly doesn't seem trivial to recreate reliably in our usage at least. The pingInterval fix is unusual but I'm not too close to the implementation bar a cursory look at the source.

[charlie-harvey]

I am running into the same problem but only in my unit tests. I believe I have found the issue. Just have no idea how to work around it.

import { createClient } from "redis";

jest.useFakeTimers();   // <<-- THIS MAKES REDIS NEVER CONNECT. Remove it and the test passes.

describe("redis", () => {

  it("connect to redis", async () => {
    const redis = createClient({
      url: 'redis://localhost:6379/2'
    });
    await redis.connect();
    const thing = await redis.set('myKey', 'myValue');
    expect(thing).not.toBeNull();
    redis.quit();
  });

});

  redis
    ✕ connect to redis (50002 ms)

  ● redis › connect to redis

    thrown: "Exceeded timeout of 50000 ms for a test.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      11 |   });
      12 |
    > 13 |   it("connect to redis", async () => {
         |   ^
      14 |     const redis = createClient({
      15 |       url: 'redis://localhost:6379/2'
      16 |     });

      at services/cache/redis.spec.ts:13:3
      at Object.<anonymous> (services/cache/redis.spec.ts:5:1)

Jest has detected the following 1 open handle potentially keeping Jest from exiting:

  ●  TCPWRAP

      15 |       url: 'redis://localhost:6379/2'
      16 |     });
    > 17 |     await redis.connect();
         |                 ^
      18 |     const thing = await redis.set('myKey', 'myValue');
      19 |     expect(thing).not.toBeNull();
      20 |     redis.quit();

      at RedisSocket._RedisSocket_createNetSocket (../node_modules/@redis/client/dist/lib/client/socket.js:209:21)
      at ../node_modules/@redis/client/dist/lib/client/socket.js:176:101
      at RedisSocket._RedisSocket_createSocket (../node_modules/@redis/client/dist/lib/client/socket.js:173:12)
      at RedisSocket._RedisSocket_connect (../node_modules/@redis/client/dist/lib/client/socket.js:148:154)
      at RedisSocket.connect (../node_modules/@redis/client/dist/lib/client/socket.js:51:96)
      at Commander.connect (../node_modules/@redis/client/dist/lib/client/index.js:184:71)
      at Object.<anonymous> (services/cache/redis.spec.ts:17:17)

[charlie-harvey]

If it makes you feel any better I just tried the same test with ioredis and got the same result. The jest.useFakeTimers() somehow destroys it.

[charlie-harvey]

So its a Jest problem. Specifically with the fake timers in any version > 27.

jest.useFakeTimers({ legacyFakeTimers: true });

Have to use "legacy" now or everything locks up.

[samwaterbury]

In my case, I was getting the Connection timeout error when trying to connect to a local Redis instance with the URL redis://localhost:6379.

Changing it to reference the explicit IPv4 address redis://127.0.0.1:6379 fixed the issue for me!

[ddNils]

I feel like this is either a bug or rather a documentation issue.

For me as I want to migrate from v3 to v4 - I did get the timeout error as well, when I tried to use the official documentation like this:

  const clientTimeout = redis.createClient({
    password: '*****',
    socket: {
        port: 6379,
        host: '127.0.0.1',
        tls: true
    }
  });

But when I use the following code (with non-documented properties), the client does connect.

  const clientWORKING = redis.createClient({
    host: '127.0.0.1',
    port: 6379,
    password: '*****',
    tls : true
  });

So I would argue, that the documentation is not listing tls, host and port as root properties correctly.

The above code was run in a node test project using redis version 4.6.7

[ivandotv]

I have also stumbled on to this only when using Jest (29)
I have solved this in my unit tests by NOT mocking certain things

    jest.useFakeTimers({
      doNotFake: ['nextTick', 'performance', 'queueMicrotask']
    })

[Thomaxius]

This also happened for us when mocking time. For some reason the error only occurred in ci/cd server which has different timezone. This thread helped to figure out the following workaround:

this.clock = sinon.useFakeTimers({ now: DateTime.now().setZone('UTC').toMillis(), toFake: ['Date'], });