Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SocketIO deployment enters crashloopbackoff when using external redis deployment #60

Closed
bruxillensis opened this issue Jan 26, 2021 · 2 comments
Labels
bug Something isn't working

Comments

@bruxillensis
Copy link

Description of the issue

Helm chart socketio deployment crashes when trying to use an existing redis deployment

Context information (for bug reports)

Steps to reproduce the issue

  1. Install MariaDB and Redis externally from erpnext
  2. Install ERPnext using the helm chart and the following values for Redis:
redisQueueHost: "redis-master.default.svc.cluster.local"
redisCacheHost: "redis-master.default.svc.cluster.local"
redisSocketIOHost: "redis-master.default.svc.cluster.local"

Observed result

SocketIO deployment enters crash loop backoff because it still expects the following host: erpnext-redis-socketio not the one specified in the helm chart. This is also the case if the chart is named anything other than "erpnext".

Expected result

Chart deploys successfully

Stacktrace / full error message if available

listening on *: 9000
node:events:356
      throw er; // Unhandled 'error' event
      ^

Error: getaddrinfo ENOTFOUND erpnext-redis-socketio
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:69:26)
Emitted 'error' event on RedisClient instance at:
    at RedisClient.on_error (/home/frappe/frappe-bench/apps/frappe/node_modules/redis/index.js:406:14)
    at Socket.<anonymous> (/home/frappe/frappe-bench/apps/frappe/node_modules/redis/index.js:279:14)
    at Socket.emit (node:events:379:20)
    at emitErrorNT (node:internal/streams/destroy:188:8)
    at emitErrorCloseNT (node:internal/streams/destroy:153:3)
    at processTicksAndRejections (node:internal/process/task_queues:81:21) {
  errno: -3008,
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'erpnext-redis-socketio'
}
@bruxillensis bruxillensis added the bug Something isn't working label Jan 26, 2021
@revant
Copy link
Collaborator

revant commented Jan 27, 2021

I am able to use separate Elasticache in one of the clusters.
On another cluster I've setup separate redis server manually and I can access it by private ip.

Check networking of the external helm chart, does it need extra config to make it available for access?

@bruxillensis
Copy link
Author

I believe I have solved it. I did not realize that the env variables are saved on the first deployment and not used afterward (documented in the docker images). This would be helpful to document in this repo. I was using an AWS EFS storageclass (to achieve ReadWriteMany on sites_dir) that persisted after a helm uninstall. Originally I installed the helm chart as default (with the redis deployments) and only afterward switched to an external Redis deployment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants