Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mechanism for adding multiple foreign key constraints at once #31

Closed
simonw opened this issue Jun 27, 2019 · 0 comments
Closed

Mechanism for adding multiple foreign key constraints at once #31

simonw opened this issue Jun 27, 2019 · 0 comments

Comments

@simonw
Copy link
Owner

simonw commented Jun 27, 2019

Needed by db-to-sqlite. It currently works by collecting all of the foreign key relationships it can find and then applying them at the end of the process.

The problem is, the add_foreign_key() method looks like this:

with self.db.conn:
cursor = self.db.conn.cursor()
schema_version = cursor.execute("PRAGMA schema_version").fetchone()[0]
cursor.execute("PRAGMA writable_schema = 1")
old_sql = self.schema
extra_sql = ",\n FOREIGN KEY({column}) REFERENCES {other_table}({other_column})\n".format(
column=column, other_table=other_table, other_column=other_column
)
# Stick that bit in at the very end just before the closing ')'
last_paren = old_sql.rindex(")")
new_sql = old_sql[:last_paren].strip() + extra_sql + old_sql[last_paren:]
cursor.execute(
"UPDATE sqlite_master SET sql = ? WHERE name = ?", (new_sql, self.name)
)
cursor.execute("PRAGMA schema_version = %d" % (schema_version + 1))
cursor.execute("PRAGMA writable_schema = 0")
# Have to VACUUM outside the transaction to ensure .foreign_keys property
# can see the newly created foreign key.
cursor.execute("VACUUM")

That means it's doing a full VACUUM for every single relationship it sets up - and if you have hundreds of foreign key relationships in your database this can take hours.

I think the right solution is to have a .add_foreign_keys(list_of_args) method which does the bulk operation and then a single VACUUM. .add_foreign_key(...) can then call the bulk action with a single list item.

simonw added a commit that referenced this issue Jun 28, 2019
Still needs unit tests and documentation.
@simonw simonw mentioned this issue Jun 28, 2019
2 tasks
@simonw simonw closed this as completed in 997d875 Jun 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant