Tiles

[kmcallorum]

We have had a lot of issues lately, one has been OOM issue with this plugin. I want to make sure that there is something on record here. The tiles that are new, we had a user add some and the query was made to the entire repository. This threw an hprof in the log directory and thread starvation in the logs. Effectively making XLR not response to URL's. I left it run for 30 minutes afterwards but it never recovered so I was forced to restart the service.

Per Jeroene
We can reproduce such a large request by using the API end-point with same url: "/deployit/tasks/v2/query". This call potentially return hundreds of megabytes of data and also puts a significant load on the database for a long time. This call seems to be made by a Jython plugin as shown in the sceenshot of the memory threads of XLR. This Jython plugin may then cause big heap usage causing an OOM Exception. Our search for such a plugin has revealed that the "xlr-xldeploy-plugin" has a dashboard tile that does such a data call retrieving a lot of data. This corresponds to the tile request that was shown in the logs and this particular plugin indeed has a tile that cause such large requests to the API.

According to the logs in shown in the mail there is a user (cshaw4) that has this tile in his dashboard. To avoid this problem from re-occuring we suggest that you remove the tile from the template. You can navigate to any release and then modify the URL to point to "Release991664026" to find the corresponding template to modify.

Note that even when this tile is removed, any user can still re-add this tile and cause the problem to re-appear. We can release a version of the community plugin with this specific tile functionality disabled to avoid this. When the next version of the plugin is released you can then upgrade without the risk of running OOM caused by this plugin as we are working with the community on a new release of this plugin to avoid this problem.

[tjrandall]

@kmcallorum - did the updated xlr-xldeploy plugin get installed on Sunday? And if yes, can we close this issue? Thank you!

[kmcallorum]

I haven't see any patches to the code to deploy yet. So it's still open.
Kevin

[tjrandall]

I just checked with Sunil. Per Sunil, It's on an Optum branch and Optum is going to check out the changes. Will get a status from Clinton today

[kmcallorum]

The optum branch does not have any fixes or even this project. This repo has no optum branch and no new code in about 29 days from what I see. Since there is no other repo at tnd I have pulled from this repo, I don't see any code changes to this fix.

[kmcallorum]

There is a branch 2.9.x but we are 7.0.2 in XLR. We need the version 3 fixed not 2.9.x version.

[wasabibob]

@kmcallorum, can you confirm what version of the xlr-xldeploy-plugin you are using now, with 7.0.2? Please and thank you.

[kmcallorum]

xlr-xldeploy-plugin-3.3.1.jar I will be going to the latest version if I can keeping this thing up to date.
Kevin

[wasabibob]

Okay, I'm working on getting my updates to v3.x.

[wasabibob]

Branch v3.x is updated, code review and dev testing complete. PR has been submitted. @kmcallorum, I believe this branch ready for testing.