NSF from OpenNTF: Agent did not complete within configured time limit

[zeromancer1972]

I used the pre-built NSF from the OpenNTF download page. When I open the start page it runs to death because the getdatabases agent runs infinite. Interesting: this message occurs already after 10 seconds though the agent execution time on my server is set to 10 minutes.

[zeromancer1972]

Seems to be a bigger problem: after loggin in (not Anonymous anymore) I started the Agent and it crashed the server completely. Maybe there is a problem when using DbDirectory with a huge amount of databases so it exceeds a certain memory limit or something like that. Why not using Java to code the agent that produces a JSON output? Not sure if using Java instead would solve this issue though.

[zeromancer1972]

I did some further investigation: the database I used in my first example was the fakenames.nsf which is basically a Domino directory. It seems that the agent can't handle that huge amount of forms and fields that are in the design. Switching to a much simpler database worked. I am not sure if this is a limitation with Lotusscript, so again I would suggest using Java - or limit the amount of information that is crawled from each database.

[dprosper]

I believe we hit an issue in LotusScript, the db.GetForm('formname') returns a non-form item if another item as same or similar name, in the case of names.nsf database, there is a form called location which when the db.GetForm("location") runs returns a handle to an item called location.gif which is not a form.

So the LS was not able to handle that. It also crashed the server which is unexpected.

We added a work around to handle the situation where the item returned is not really a form and/or contains no fields.

That should take care of the crash.

[scott-good]

A potential issue that's related: In one of the lookup reference docs we had specified to pull the data from Names.nsf. It turns out (not surprisingly) that Names.nsf has a huge number of both forms and fields and even after the fix Dimitri mentions we were not getting a value back from the getdocuments agent. I suspect we were overrunning the string length limitations. We've pulled Names.nsf out of the reference doc for now, but this is an issue that needs to be addressed because surely there will be a need for some folks to use DQL against the address book.

[zeromancer1972]

I bet it's a memory problem with LS so maybe consider using a custom service bean and the REST component from the ExtLib on an XPage. Once the NSF part comes as ODP I am happy to contribute that solution ;-)

[dprosper]

Scott and I had a conversation around this issue this morning, i.e. getdatabases just does too many things. So we need to break it out into smaller chunks of work, i.e. getdatabases (only database names, titles), getformsfordatabase, getviewsfordatabase and/or getfoldersfordatabase, then getfieldsforselectedform, etc... doing it this way will likely reduce the work done on the server.

I think this model is needed and would work regardless if the agent is LS or something else.

This change requires a few updates to the Front End app which we will do this weekend (a few hours) and then we can push this update along with the ODP version. And yes would love to see your contributions to the code.

[dprosper]

Thank your for reporting. We've address this in a new release https://github.com/icstechsales/dql-explorer/releases