Pagination on /#Schedule

[vikingtoby]

We have quite a large install and on the /#Schedule page, we have 859 schedules/jobs as of right now.

Would it be possible to add some pagination support, like X per page or something, it's getting slower and slower to load all these schedules/jobs and their respective status.

It's already done in the Completed tab: /#History?sub=history

PS: Absolutely amazing tool you've built, scales well on very few ressources, it just works. :)

[dheise-rheinenergie-trading-com]

😂

[jhuckaby]

Are you seeing slowdown as well? It surprises me that the schedule page would be slow with 1K items in the table.

[dheise-rheinenergie-trading-com]

not really. Cronicle and the Scheduled page loads in about 1 second.

[mikeTWC1984]

@vikingtoby Just wondering, are you using Filesytem plugin for storage? If so did you ever encounter issues with inodes (too many files)?

[jhuckaby]

Setting job_data_expire_days to a reasonably small number of days should greatly reduce the INODE usage with the Filesystem Plugin. The default is 180 days, but setting this to 30 or 15 days is probably better for a schedule of @vikingtoby's size.

This of course depends on the filesystem and mount (ext4, zfs, etc.). Some filesystems have a huge amount of available INODEs, and you can increase them in some cases, or configure them when formatting.

[vikingtoby]

Yes, we're using Filesystem for storage.
I ended up switching the /opt/cronicle path to ZFS (dynamically assigned inodes) for this very reason, as the inodes usage was going up rapidly, we have on average +30k jobs running a day.

[dheise-rheinenergie-trading-com]

I set my expire_days to 10 days. Works without any problems. My data directory is 14GB with 3.556.000 files.

[jhuckaby]

@vikingtoby How slow are we talking here? I ask because it surprises me that the page would be slow with 859 items in the table. Modern browsers should eat this for lunch.

The schedule data is all preloaded when the app UI first loads, so when you click on the "Schedule" tab, it isn't loading anything from the server. All it is doing is rendering the table from memory. It should be very fast 🤔

[vikingtoby]

Well, slowness is perhaps an overstatement.

Opening for example the "Completed" tab which has pagination, is absolutely instant (100ms or so), whereas the "Schedule" tab is noticably slower, but still fast. (about 1100ms).
I was more worried about the future scaling, where pagination could definitely help.

Perhaps do it dynamically? When +500 schedules, start using pagination or something like that.

[jhuckaby]

Got it -- thank you for the clarification. Cronicle v2 (Orchestra) will progressively render chunks of the schedule as you scroll down (100 items at a time, configurable).

The schedule doesn't "paginate" like other pages like the Completed tab, for two reasons. First, the data is already loaded into memory, so we have all the items ready to render. And second, it's a particularly complicated table with "section headers", depending on the mode you are in:

e.g. sort by title, or split up and group events by category, plugin or target. The latter 3 modes insert section headers inside the table. So paginating it the traditional sense would be rather difficult.

So in Orchestra (which has an even more advanced schedule table) I opted to paginate it by progressively rendering it, based on the scroll position. I'll try to make this as seamless as possible.

This should eliminate any browser rendering delays for large schedules.