New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use native UPSERTs where possible #4306

Merged
merged 49 commits into from Jan 24, 2019

Conversation

Projects
None yet
4 participants
@hawkowl
Copy link
Contributor

hawkowl commented Dec 18, 2018

No description provided.

@hawkowl hawkowl changed the title Use native UPSERTs Use native UPSERTs where possible Dec 18, 2018

@hawkowl

This comment has been minimized.

Copy link
Contributor Author

hawkowl commented Dec 18, 2018

Fixes #4015 (on recent postgreses)

hawkowl added some commits Dec 18, 2018

@codecov-io

This comment has been minimized.

Copy link

codecov-io commented Dec 18, 2018

Codecov Report

Merging #4306 into develop will increase coverage by 1.13%.
The diff coverage is 81.39%.

@@            Coverage Diff             @@
##           develop   #4306      +/-   ##
==========================================
+ Coverage    73.66%   74.8%   +1.13%     
==========================================
  Files          302     336      +34     
  Lines        29822   33997    +4175     
  Branches      4896    5527     +631     
==========================================
+ Hits         21969   25432    +3463     
- Misses        6421    7001     +580     
- Partials      1432    1564     +132

hawkowl added some commits Dec 18, 2018

@hawkowl hawkowl requested a review from matrix-org/synapse-core Dec 18, 2018

hawkowl added some commits Dec 21, 2018

fix
@richvdh
Copy link
Member

richvdh left a comment

looks plausible but since there are conflicts and test failures I assume you're still working on it...

hawkowl added some commits Jan 3, 2019

@richvdh richvdh requested a review from matrix-org/synapse-core Jan 18, 2019

Show resolved Hide resolved synapse/storage/engines/__init__.py
"""
Can we use native UPSERTs? This requires PostgreSQL 9.5+.
"""
return self._version >= 90500

This comment has been minimized.

@erikjohnston

erikjohnston Jan 21, 2019

Member

This is going to blow up if called before on_new_connection. This is probably unlikely, but it may be worth defaulting it to something like 0. Or have a _can_do_native_upsert parameter set to False by default instead, and have on_new_connection update it when its called

This comment has been minimized.

@hawkowl

hawkowl Jan 24, 2019

Author Contributor

This won't ever be called before we're actually connected, since it's in transaction code?

Show resolved Hide resolved synapse/storage/engines/sqlite.py Outdated
Show resolved Hide resolved tox.ini
Show resolved Hide resolved .coveragerc
Show resolved Hide resolved MANIFEST.in Outdated
Show resolved Hide resolved synapse/storage/_base.py
Show resolved Hide resolved synapse/storage/_base.py Outdated

hawkowl added some commits Jan 23, 2019

@hawkowl hawkowl requested a review from matrix-org/synapse-core Jan 23, 2019

@erikjohnston
Copy link
Member

erikjohnston left a comment

Woo, this looks like it'll work! Just some small nits.

It's a shame that we can't get rid of the return value of _simple_upsert. The only way I can think of is to implement a version of _simple_upsert that lets you specify generic expressions, rather than simply key/value pairs, but there's probably not much point if we're going to drop support for 9.4 soon.

Show resolved Hide resolved synapse/storage/_base.py
Show resolved Hide resolved synapse/storage/_base.py Outdated
Show resolved Hide resolved synapse/storage/user_directory.py Outdated

@hawkowl hawkowl merged commit 58f6c48 into develop Jan 24, 2019

5 checks passed

ci/circleci: sytestpy2merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy2postgresmerged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3postgresmerged Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@hawkowl hawkowl deleted the hawkowl/upsert-e2e branch Jan 24, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment