Limit number of entries to prefill from cache #1792

Merged
merged 1 commit into from Jan 10, 2017

Projects

None yet

2 participants

@erikjohnston
Member

Some tables, like device_inbox, take a long time to query at startup for
the stream change cache prefills. This is likely because they are slower
growing streams and so are more fragmented on disk. For now, lets pull
fewer entries out to make startup quicker.

In future, we should add a better index to make it even faster.

@erikjohnston erikjohnston Limit number of entries to prefill from cache
Some tables, like device_inbox, take a long time to query at startup for
the stream change cache prefills. This is likely because they are slower
growing streams and so are more fragmented on disk. For now, lets pull
fewer entries out to make startup quicker.

In future, we should add a better index to make it even faster.
dd52d4d
@NegativeMjark
Contributor

LGTM codewise but I don't think it'll actually fix the perf for device_inbox prefill since the query is doing an index scan rather than a backwards index scan. See query plans #1768.

@erikjohnston
Member

... but I don't think it'll actually fix the perf for device_inbox prefill...

It does, I tested it.

... since the query is doing an index scan rather than a backwards index scan

Going forwards or backwards only matters if you're counting. When using the stream_id > ? predicate going forwards or backwards is equivalent.

@NegativeMjark
Contributor
NegativeMjark commented Jan 10, 2017 edited

K, makes sense. (would be really awesome if we could tune this on a worker by worker basis, but I digress).

@erikjohnston erikjohnston merged commit d524bc9 into develop Jan 10, 2017

6 of 8 checks passed

Sytest Dendron (Commit) Build #1339 origin/erikj/limit_cache_prefill_device failed in 56 sec
Details
Sytest Dendron (Merged PR) Build finished.
Details
Sytest Postgres (Commit) Build #2155 origin/erikj/limit_cache_prefill_device succeeded in 8 min 20 sec
Details
Sytest Postgres (Merged PR) Build finished.
Details
Sytest SQLite (Commit) Build #2223 origin/erikj/limit_cache_prefill_device succeeded in 6 min 57 sec
Details
Sytest SQLite (Merged PR) Build finished.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment