-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] With 8k+ tiddlers, in a NodeJS wiki everything is Slow #7151
Comments
Did you try to create a new server and import your data there. Is it still slow. If not you may have to look at your plugins. |
Hi @linonetwo performance at that scale is definitely a problem, and we welcome investigations to identify and optimise performance hotspots. One thing I'd love to experiment with is to use the new SQLLite JavaScript release as the basis for the wiki store object. It has a synchronous API which is needed for the TW store. |
It is certainly possible that there is a more performant way of making a case-insensitive sort, and again would welcome investigations. |
@pmario You are right, some of friends in QQ group report lagging is caused by locator plugin and kin filter plugin. I will try this in a clean environment. |
I should have added that there are three usages of wiki.getTiddlers() in the core, and it looks like the usage in |
One thing I like about nodejs wiki is its file based storage, which is nice for git diff, so I don't really want to use sqlite to solve this problem. |
We wouldn't necessarily need to use SQLLite as our persistence format; we could still save to the same file formats that we do today. |
I don't know locator, but IMO kin filter should only be slow, if it is shown. Similar to the Recent tab. So if Open tab is shown by default it may be faster. |
This comment was marked as outdated.
This comment was marked as outdated.
It is kin-filter that being slow, but why will it be called during starting? I know, it is |
My script to generate the flame graph prifiling: 0x -o ./tiddlywiki.js "+plugins/tiddlywiki/filesystem" "+plugins/tiddlywiki/tiddlyweb" /Users/linonetwo/Desktop/wiki/wiki --listen anon-username= port=5212 host=0.0.0.0 root-tiddler=$:/core/save/lazy-images
🔥 Profilingsyncer-server-filesystem: Dispatching 'save' task: $:/StoryList
Serving on http://0.0.0.0:5212
(press ctrl-C to exit) |
Are there any startup actions in the wiki that use the kin filter? Edit: Nevermind, just saw that it is used in the FileSystemPaths. |
After removed the kin-filter, I added up the filter consumed times, its 1004ms in my wiki, so other code takes 1.5S, I need to add more perf timers. There are still something to optimize on server side. But it seems not very urgent now. I'm going to fix client side first. |
try #6550 |
@linonetwo ... Did you find the problem? It's not clear to me. Do we need to keep this issue open -- or do you create a new one when you have more data |
On server side, it is mostly because of FileSystemPaths. On client side it is not clear, I will find time for this, so keep this open for a while. |
@linonetwo: Depending on what you need |
This issue is workarounded by using yaisog/taggingtree-filter and linonetwo/in-tagtree-of plugins. And boot speed can be further increased by #7329 |
Describe the bug
I have imported 8k+ google calendar events into my nodejs wiki, they are mostly metadata, no long texts. Then each click in my wiki becomes slow, saving is slow, and starting the wiki is slow too.
Expected behavior
Be as fast as a empty wiki when we have 10 - 100k tiddlers, in nodejs wiki.
To Reproduce
I will prepare a github repo later, but now I'm debugging in my own wiki...
Screenshots
TiddlyWiki Configuration
Desktop (please complete the following information):
TidGi (darwin)
Version v0.7.13.
Additional context
I'll do some metrics tomorrow, and come up with a plan. I must fix this, because I want to use tw as my diary tool to replace the google calendar.
The text was updated successfully, but these errors were encountered: