JDODataStoreExceptions after startup #2141
Comments
|
Taking a wild guess here, this really looks like an issue in DataNucleus' query compilation and / or caching. The fact that the issue persists after it occurred for the first time, but is fixable by a restart strongly suggests that the query caching is playing a role. |
|
Initialization of those caches maybe. It happened now every time we installed Debian updates on the host and restarted the host. Possibly since the upgrade to 4.6.x. I wonder what could cause the initialization of something to fail. The database is remote in RDS and available all the time. Will try to enable DEBUG logging for datanucleus. Looking at the code they are logging parameters etc, so that might give a clue and/or more information to log an issue at datanucleus. |
|
Had another occurrence, this time with another exception: Which seems to have been reported before in #409 and #422. One of them refers to increasing / decreasing timeouts. Since my issue occurs right after a reboot it's probably not related to session timeouts. |
valentijnscholten
changed the title
|
Got this same behaviour on DependencyTrack 4.9.1 The lookup api Simultenously the logs in DT api-server has following error: |
|
I looks I'm also encountering this issue in 4.10.1 Jenkins using the dependency-Track plugin tries to push a new BOM in the post build action, which fails with a 500. docker log for the api container reports this: Docker was recently updated from After running |
|
Solved on our side by removing the No idea of the root cause tho. The server just broke by itself during the weekend |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Current Behavior
I have now seen 3 occasions of the below problem, maybe someone recognizes this.
We're running DT 4.6.2 in docker in ECS with postgres RDS instances as datastore and a volume for the DT /data directory.
Every week updates are installed on the Docker host OS (EC2, Debian 11). This usually triggers an EC2 restart, and thus a docker restart.
The containers come up succesfully and users can browse DT from the UI. Also most API calls work fine, but not all of them.
Example 1: /api/v1/project/lookup
Everything seems to be working, except /api/v1/lookup?name=bla&version=xyz:
Other calls, using the same API Key value, are working fine.
This code is in the Alpine framework and about retrieving an APIKey. However the query contains only 1 parameter:
https://github.com/stevespringett/Alpine/blob/157958bfe7fa5ad95e653667f7ce3282aed23af1/alpine-infra/src/main/java/alpine/persistence/AlpineQueryManager.java#L100-L104
permalink doesn't work it seems, here's the code:
Which should be translated by datanucleus into 2 parameters in the SQL query seen above.
So it's a mystery how parameter 1 can be set, but parameter 2 can be empty.
I spent some time digging around, but couldn't explain this.
So decided to take a professional approach: restart docker (so both containers).
This solved the problem.
Example 2: /api/v1/project/
Everything seems to be working, except /api/v1/project/:
Exception
Again this doesn't make sense as there is only 1 paramater uuid, and again parameter 2 is considered empty?
Restarted docker, problem solved.
Example 3: /api/v1/project/lookup (again)
Everything seems to be working, except /api/v1/project/lookup?name=valentijn&version=master:
Exception
Parameter 3 is the version. Which is set and should be present in the query as it's mandatory. If I remove the version from the url, it works.
DT says project cannot be found. But I haven't checked if that is the actual query returning nothing, or an early exit because the version is mandatory.
Anyway, I already knew a restart would probably fix it, so I gathered some logs and restarted docker again.
Solution
Anyone has any ideas / suggestions?
I realize this might not be a bug in DT nor in Alpine, but maybe others are seeing the same or have some suggestions.
It is not a resourcing issue as this instance is mostly idle with only 1 nightly job to upload 100 BOMs. And a nightly job that retrieves all projects to load them into backstage.
It seems that once the instance is running OK, the problem stays away and it only occurs after a restart.
I have 11 instances of DT, only 1 instances has the above problem. But it's also the only one under heave read load from backstage indexing.
Valentijn
Dependency-Track Version
4.6.2 / docker / ECS / RDS / PostgreSQL 13.7
Full stacktraces
The text was updated successfully, but these errors were encountered: