Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 [Bug] The Network Gives A 504 Gate-Away Time-Out From Time To Time #3648

Open
Tirokk opened this issue Jun 17, 2020 · 0 comments
Open

Comments

@Tirokk
Copy link
Member

Tirokk commented Jun 17, 2020

🐛 Bugreport

Sometimes the network gives a 504 gate-way time-out.
(Heavily happened on the 13th and 14th of June 2020 and in the week before as well.)

There was nor explicit error I’ve seen. Except an event I got displayed by:
kubectl -n human-connection get events --sort-by=.metadata.creationTimestamp
It said:
Warning – FailedScheduling – pod/nitro-neo4j-86596f49cd-snjn4 – 0/7 nodes are available: 6 node(s) didn't match node selector, 7 Insufficient memory.

At the end I had to restart everything several time by deleting all the network pods on after the other.

Matt and I solved a memory problem of the Neo4j database by applying memory requests and limits to the containers of the pods. We labeled a new node with enough memory and assigned the Neo4j pod to the label of this node, as to say to the node.
See issue #2628 and PR #2629 .

I have the impression that this could help us to solve the network problem, because.
It is recommended anyways:

We have to define the following settings to all deployments (Webapp, Backend, Neo4j):

    spec:
      containers:
        resources:
          requests:
            memory: XXX
            cpu: XXX
          limits:
            memory: XXX
            cpu: XXX

Steps to reproduce the behavior

Happens from time to time …

Expected behavior

Network should not give an error …

Additional context

None.

@Tirokk Tirokk added the bug label Jun 17, 2020
@Tirokk Tirokk changed the title 🐛 [Bug] The Network Gives A 504 Gate-Away Time-Out From Time To Time [WIP] 🐛 [Bug] The Network Gives A 504 Gate-Away Time-Out From Time To Time Jun 17, 2020
@Tirokk Tirokk self-assigned this Jun 17, 2020
@Tirokk Tirokk added the DevOps label Jun 17, 2020
@Tirokk Tirokk added this to New in Roadmap via automation Jun 17, 2020
@Tirokk Tirokk added this to To Do in Human-Connection via automation Jun 17, 2020
@Tirokk Tirokk changed the title [WIP] 🐛 [Bug] The Network Gives A 504 Gate-Away Time-Out From Time To Time 🐛 [Bug] The Network Gives A 504 Gate-Away Time-Out From Time To Time Jun 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Roadmap
  
New
Development

No branches or pull requests

1 participant