Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for multiple schemas in a database #7760

Merged
merged 63 commits into from
Apr 29, 2024
Merged

Support for multiple schemas in a database #7760

merged 63 commits into from
Apr 29, 2024

Conversation

zachmu
Copy link
Member

@zachmu zachmu commented Apr 18, 2024

Companion PRs:

dolthub/vitess#338
dolthub/go-mysql-server#2466
dolthub/doltgresql#181

This PR is a proof of concept for addressing multiple table namespaces (schemas) in a database. This is to support schemas in Doltgres initially, but could also be used to implement storing multiple databases in the same Dolt commit graph if we wanted to do that. All tables are stored in the root's table map like always, but tables with non-empty schemas have that schema name prepended in its map key, surrounded by null bytes.

There are lots of things that don't work yet. This is what does work:

create schema mySchema;
create schema otherSchema;
CREATE TABLE mySchema.test (pk BIGINT PRIMARY KEY, v1 BIGINT);
insert into mySchema.test values (1,1), (2,2);
CREATE TABLE otherSchema.test (pk BIGINT PRIMARY KEY, v1 BIGINT);
insert into otherSchema.test values (3,3), (4,4);
SELECT * FROM mySchema.test;
SELECT * FROM otherSchema.test;

Things I specifically want feedback on:

  • I am not crazy about the sweeping changes to table-access interfaces in root_val.go. In particular it seems likely to cause errors over time as people forget to include schema names in code changes where necessary, silently breaking doltgres. Another option I considered was to introduce some sort of RootReader interface on top of root value, which you create with a particular schema name, and thereafter all table accesses reference the given schema. There may be places where we have to juggle multiple RootReader instances at once, rather than just asking for qualified table names from a single root. But this is the pattern I used for the boundary between GMS and Dolt and it worked well, so maybe it could work for root_val.go as well.

@zachmu zachmu changed the title Support for schemas Support for multiple schemas in a database Apr 18, 2024
@zachmu
Copy link
Member Author

zachmu commented Apr 24, 2024

Regarding a marker approach, we already disallow all tables from the "dolt_" namespace right? Perhaps we could make use of that in some way to form a marker.

This is the route I went with, but instead of dolt_ I used null bytes to offset the schema name.

Copy link
Contributor

@Hydrocharged Hydrocharged left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me!

@coffeegoddd
Copy link
Contributor

@zachmu DOLT

comparing_percentages
100.000000 to 100.000000
version result total
2ecfd80 ok 5937457
version total_tests
2ecfd80 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@zachmu DOLT

comparing_percentages
100.000000 to 100.000000
version result total
053d1f2 ok 5937457
version total_tests
053d1f2 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@zachmu DOLT

comparing_percentages
100.000000 to 100.000000
version result total
027054e ok 5937457
version total_tests
027054e 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000
version result total
1da1f41 ok 5937457
version total_tests
1da1f41 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@zachmu DOLT

comparing_percentages
100.000000 to 100.000000
version result total
4ed8a62 ok 5937457
version total_tests
4ed8a62 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000
version result total
fe845d8 ok 5937457
version total_tests
fe845d8 5937457
correctness_percentage
100.0

@zachmu zachmu merged commit bf82c48 into main Apr 29, 2024
20 checks passed
@coffeegoddd
Copy link
Contributor

@zachmu DOLT

comparing_percentages
100.000000 to 100.000000
version result total
cf8c356 ok 5937457
version total_tests
cf8c356 5937457
correctness_percentage
100.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants