Posting Journal Feature: Batched Data Download

[jniles]

The Journal currently downloads all data at once, which, even on a local network, takes several seconds for massive amounts of rows. Even though we really should never encounter a condition with >10,000 rows in the posting journal, we should still anticipate that this might arise. The initial large data download results in UI freezing, and a lack of usability of the application.

In this proposal, the grid should download an initial payload of a certain size, then download data batches either (1) progressively or (2) automatically. In the former case, the grid waits until the user demands more data, such as hitting the next arrow in a pagination toolbar or scrolling towards the bottom of the infinite scrolling list. In the latter case, the grid immediately schedules another batch for download after the initial payload is loaded.

This proposal calls for three configuration options:

1. Initial Payload Size - the number of rows (e.g. 50, 100 rows) that will be downloaded by the grid upon first connection.
2. Batch Size - the number of rows (e.g. 25, 350 rows) that will be requested when a new batch is requested by the grid
3. Automatic Batch Loading - a Boolean (True/False). A "False" value indicates that the data will be progressively loaded. A "True" value indicates that the batches will be automatically scheduled after the first successful batch is loaded.

On the UI side, we will need the following improvements:

1. A loading indicator covering the entire grid during the initial batch loading.
2. A small loading indicator on the edge of the grid (such as top or bottom corners) that indicates that further batches are being retrieved. It should indicate the batch numbers loaded via a label (e.g. 15/65 Batches Downloaded).
3. Somewhere else on the grid (such as the opposite corner), the grid should indicate the number of rows in the current view. (e.g. 135 rows).

I believe this feature is flexible enough to do either pagination or virtual scrolling, although the implementation details may be so different that we favor one or the other.