WBD FROST Server (sta.wbd-rd.nl) extremely slow/ timing out

[mathisvfr]

We're experiencing timeouts when connecting to the WBD FROST server

• Unauthenticated requests take ~12s to return a 401
• Authenticated requests time out entirely (>30s, no response)
• This affects both entity registration and observation pushes from our connector
• The other two testbed targets (Collaborall v1.1 and FROST v2.0 demo) respond in <300ms

Is there a known issue with the WBD server, or is maintenance planned? Any ETA on recovery would be helpful so we can plan around it.

[hylkevds]

Odd, I see no delays or errors:
https://sta.wbd-rd.nl/FROST-Server/v1.1/Things?$orderby=id%20asc&$select=id,name
Also not when creating a new Thing

Which queries are giving issues?

@znetsixe, anything visible in the server logs?

[mathisvfr]

Thanks for checking! It might have been a transient issue. Our connector was hitting read timeouts (>15s, sometimes >30s) on authenticated requests between roughly 14:00-16:30 UTC today. Simple unauthenticated requests returned 401 in ~12s, but authenticated queries (GETs with $filter and POSTs for entity creation) hung until our timeout kicked in.

It's working fine now, all requests complete in <150ms. Might have been a temporary thing. We'll keep monitoring and report back if it recurs.

The specific queries that failed were OData filter lookups ($filter=name eq '...') and entity POST/PATCH used by our registration and observation push flows, all with Basic auth (write user).

[znetsixe]

Yes. We managed to break the entire vps yesterday for half a day 🥇 . One of the few times it wasn't me who broke it. We fixed everything around 17.00 so the timing you're describing is sound.
We "only" have about 24 GB RAM on the VPS, and it turns out our Docker containers weren't capped in usage so they were actively competing for RAM at a certain point. My colleague was executing a compute-heavy script which eventually tipped the VPS in the wrong direction, and we had an OOM problem which was even worse, seeing as we couldn't SSH in to reboot and the login (because of the billing) isn't our group (yet), so we had to wait for another colleague to reboot it from the dashboard.

Fun times :-P

I implemented Docker container memory safety after identifying the problem, so this hopefully can't happen again. At least not in the way it happened before.