Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving upserts #79

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ Getting Started
- Write access to an empty `PostgreSQL <http://www.postgresql.org>`_ database.
- A Python installation with `Jupyter Notebook <https://github.com/jupyter/notebook>`_ >= 5.0.

PGContents will put its table in the `pgcontents` namespace. When you log onto the PostgreSQL database server, make
sure the `pgcontents` schema is in the search path (e.g., `set search_path to 'pgcontents'`; see `PostgreSQL documentation`_).

**Installation:**

0. Install ``pgcontents`` from PyPI via ``pip install pgcontents``.
Expand All @@ -23,3 +26,4 @@ Demo Video
You can see a demo of PGContents in action in `this presentation from JupyterCon 2017`_.

.. _`this presentation from JupyterCon 2017` : https://youtu.be/TtsbspKHJGo?t=917
.. _`PostgreSQL documentation` : https://www.postgresql.org/docs/14/ddl-schemas.html#DDL-SCHEMAS-PATH
41 changes: 14 additions & 27 deletions pgcontents/query.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
select,
Unicode,
)
from sqlalchemy.dialects.postgresql import insert

from sqlalchemy.exc import IntegrityError

Expand Down Expand Up @@ -500,41 +501,27 @@ def rename_directory(db, user_id, old_api_path, new_api_path):
def save_file(db, user_id, path, content, encrypt_func, max_size_bytes):
"""
Save a file.

TODO: Update-then-insert is probably cheaper than insert-then-update.
"""
content = preprocess_incoming_content(
content,
encrypt_func,
max_size_bytes,
)
directory, name = split_api_filepath(path)
with db.begin_nested() as savepoint:
try:
res = db.execute(
files.insert().values(
name=name,
user_id=user_id,
parent_name=directory,
content=content,
)
with db.begin_nested():
res = db.execute(
insert(files)
.values(
name=name,
user_id=user_id,
parent_name=directory,
content=content,
)
except IntegrityError as error:
# The file already exists, so overwrite its content with the newer
# version.
if is_unique_violation(error):
savepoint.rollback()
res = db.execute(
files.update().where(
_file_where(user_id, path),
).values(
content=content,
created_at=func.now(),
)
)
else:
# Unknown error. Reraise
raise
.on_conflict_do_update(constraint="uix_filepath_username", set_={
"content": content,
"created_at": func.now()
})
)

return res

Expand Down