/synapse

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Add an index to event_search #2218

Merged
merged 4 commits into from May 16, 2017

Conversation

Reviewers

@erikjohnston erikjohnston

Assignees

@erikjohnston erikjohnston

Labels
maintenance
Projects
None yet
Milestone
No milestone
2 participants
@richvdh @erikjohnston
@richvdh
Member

richvdh commented May 11, 2017

also, add some more logging

richvdh added some commits May 11, 2017

@richvdh
Add an index to event_search 
- to make the purge API quicker
baafb85
@richvdh
Add more logging for purging 
Log the number of events we will be deleting at info.

Some checks were not successful

2 failing and 6 successful checks
continuous-integration/travis-ci/pr — The Travis CI build failed
Details
continuous-integration/travis-ci/push — The Travis CI build failed
Details
@matrixbot
Sytest Dendron (Commit) — Build #2121 origin/rav/event_search_index succeeded in 9 min 12 sec
Details
@matrixbot
Sytest Dendron (Merged PR) — Build finished.
Details
@matrixbot
Sytest Postgres (Commit) — Build #2956 origin/rav/event_search_index succeeded in 5 min 34 sec
Details
@matrixbot
Sytest Postgres (Merged PR) — Build finished.
Details
@matrixbot
Sytest SQLite (Commit) — Build #3026 origin/rav/event_search_index succeeded in 6 min 10 sec
Details
@matrixbot
Sytest SQLite (Merged PR) — Build finished.
Details
114f290

@richvdh richvdh assigned erikjohnston May 11, 2017

richvdh added some commits May 11, 2017

@richvdh
Don't create event_search index on sqlite 
... because the table is virtual
34194aa
@richvdh
Add a comment to old delta

All checks have passed

8 successful checks
@matrixbot
Sytest Dendron (Commit) — Build #2122 origin/rav/event_search_index succeeded in 8 min 47 sec
Details
@matrixbot
Sytest Dendron (Merged PR) — Build finished.
Details
@matrixbot
Sytest Postgres (Commit) — Build #2957 origin/rav/event_search_index succeeded in 7 min 51 sec
Details
@matrixbot
Sytest Postgres (Merged PR) — Build finished.
Details
@matrixbot
Sytest SQLite (Commit) — Build #3027 origin/rav/event_search_index succeeded in 6 min 42 sec
Details
@matrixbot
Sytest SQLite (Merged PR) — Build finished.
Details
continuous-integration/travis-ci/pr — The Travis CI build passed
Details
continuous-integration/travis-ci/push — The Travis CI build passed
Details
ff3d810
@erikjohnston
Owner

erikjohnston commented May 15, 2017

Is there a reason we make this more complicated and make it UNIQUE?
@richvdh
Member

richvdh commented May 15, 2017

Is there a reason we make this more complicated and make it UNIQUE?

does that actually make it materially more complicated? I'd argue register_background_index_update ought to support UNIQUE indexes anyway, at which point the difference is a single param.
@erikjohnston
Owner

erikjohnston commented May 16, 2017

does that actually make it materially more complicated? I'd argue register_background_index_update ought to support UNIQUE indexes anyway, at which point the difference is a single param.

It's probably be fine, I don't think UNIQUE is that much slower than non unique indices
@richvdh
Member

richvdh commented May 16, 2017

It's probably be fine, I don't think UNIQUE is that much slower than non unique indices

I'm struggling to imagine an implementation in which a unique index is not at least as fast as a non-unique one, for any given operation.

@richvdh richvdh merged commit d648f65 into develop May 16, 2017
8 checks passed

8 checks passed

Sytest Dendron (Commit) Build #2122 origin/rav/event_search_index succeeded in 8 min 47 sec
Details
Sytest Dendron (Merged PR) Build finished.
Details
Sytest Postgres (Commit) Build #2957 origin/rav/event_search_index succeeded in 7 min 51 sec
Details
Sytest Postgres (Merged PR) Build finished.
Details
Sytest SQLite (Commit) Build #3027 origin/rav/event_search_index succeeded in 6 min 42 sec
Details
Sytest SQLite (Merged PR) Build finished.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@richvdh richvdh deleted the rav/event_search_index branch May 16, 2017

@erikjohnston
Owner

erikjohnston commented May 16, 2017

I'm struggling to imagine an implementation in which a unique index is not at least as fast as a non-unique one, for any given operation.

Specifically an INSERT as the the DB would then need to check whether the unique constraint was violated or not. OTOH, that's probably not any worse than just updating the index anyway.

@erikjohnston erikjohnston added the maintenance label May 17, 2017

psaavedra added a commit to psaavedra/synapse that referenced this pull request May 19, 2017

@psaavedra
Merge tag 'v0.21.0' into v0.21.0_no_federate_by_default 
Changes in synapse v0.21.0 (2017-05-18)
=======================================

No changes since v0.21.0-rc3

Changes in synapse v0.21.0-rc3 (2017-05-17)
===========================================

Features:

* Add per user rate-limiting overrides (PR #2208)
* Add config option to limit maximum number of events requested by ``/sync``
  and ``/messages`` (PR #2221) Thanks to @psaavedra!

Changes:

* Various small performance fixes (PR #2201, #2202, #2224, #2226, #2227, #2228,
  #2229)
* Update username availability checker API (PR #2209, #2213)
* When purging, don't de-delta state groups we're about to delete (PR #2214)
* Documentation to check synapse version (PR #2215) Thanks to @hamber-dick!
* Add an index to event_search to speed up purge history API (PR #2218)

Bug fixes:

* Fix API to allow clients to upload one-time-keys with new sigs (PR #2206)

Changes in synapse v0.21.0-rc2 (2017-05-08)
===========================================

Changes:

* Always mark remotes as up if we receive a signed request from them (PR #2190)

Bug fixes:

* Fix bug where users got pushed for rooms they had muted (PR #2200)

Changes in synapse v0.21.0-rc1 (2017-05-08)
===========================================

Features:

* Add username availability checker API (PR #2183)
* Add read marker API (PR #2120)

Changes:

* Enable guest access for the 3pl/3pid APIs (PR #1986)
* Add setting to support TURN for guests (PR #2011)
* Various performance improvements (PR #2075, #2076, #2080, #2083, #2108,
  #2158, #2176, #2185)
* Make synctl a bit more user friendly (PR #2078, #2127) Thanks @APwhitehat!
* Replace HTTP replication with TCP replication (PR #2082, #2097, #2098,
  #2099, #2103, #2014, #2016, #2115, #2116, #2117)
* Support authenticated SMTP (PR #2102) Thanks @DanielDent!
* Add a counter metric for successfully-sent transactions (PR #2121)
* Propagate errors sensibly from proxied IS requests (PR #2147)
* Add more granular event send metrics (PR #2178)

Bug fixes:

* Fix nuke-room script to work with current schema (PR #1927) Thanks
  @zuckschwerdt!
* Fix db port script to not assume postgres tables are in the public schema
  (PR #2024) Thanks @jerrykan!
* Fix getting latest device IP for user with no devices (PR #2118)
* Fix rejection of invites to unreachable servers (PR #2145)
* Fix code for reporting old verify keys in synapse (PR #2156)
* Fix invite state to always include all events (PR #2163)
* Fix bug where synapse would always fetch state for any missing event (PR #2170)
* Fix a leak with timed out HTTP connections (PR #2180)
* Fix bug where we didn't time out HTTP requests to ASes  (PR #2192)

Docs:

* Clarify doc for SQLite to PostgreSQL port (PR #1961) Thanks @benhylau!
* Fix typo in synctl help (PR #2107) Thanks @HarHarLinks!
* ``web_client_location`` documentation fix (PR #2131) Thanks @matthewjwolff!
* Update README.rst with FreeBSD changes (PR #2132) Thanks @feld!
* Clarify setting up metrics (PR #2149) Thanks @encks!
badd30b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment