Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move legacy DD structs to Tarantool #2217

Closed
mejedi opened this issue Mar 17, 2017 · 2 comments
Closed

Move legacy DD structs to Tarantool #2217

mejedi opened this issue Mar 17, 2017 · 2 comments
Assignees
Labels
Milestone

Comments

@mejedi
Copy link
Contributor

mejedi commented Mar 17, 2017

Tables and indices in SQLite are identified by a single integer representing the root page number of the corresponding BTree.

Due to SQLite legacy we encode space and index ids in a page number. Store and pass spaceid, indexid explicitly.

Rationale: the current approach restricts id range; makes comprehending the code harder.

@mejedi mejedi added the sql label Mar 17, 2017
@kostja kostja added this to the 1.8.1 milestone Mar 28, 2017
@kostja kostja added the prio2 label Jul 4, 2017
@kyukhin
Copy link
Contributor

kyukhin commented Nov 10, 2017

There's no proposal here. One of main goals of integration of SQL w/ Trantool is to merge data dictionaries of SQL w/ Tarantool's one. So, we're going to merge struct Table and struct Index of SQL w/ corresponding Tnt entities (space/index) and refactoring SQL sources to adopt the change.

I think that this is a big deal and should be moved out of alpha-2 and scheduled to be implemented during alpha-2, beta-1 development slot.

@kostja kostja modified the milestones: 1.8.3, 1.8.4 Nov 21, 2017
@kyukhin kyukhin modified the milestones: 1.8.4, 1.8.6 Dec 19, 2017
@kyukhin
Copy link
Contributor

kyukhin commented Dec 21, 2017

At the end of the day, we'd like to use single data dictionary. Current vision is to remove most of legacy SQLite data structures, which represent data dictionary and extend backend's data structures w/ necessary fields.

Obvious mapping is (legacy -> TNT):
struct Table -> struct space;
struct Index -> struct index;

For struct Table, there're also field that have no sense to put inside struct space, unless corresponding feature is implemented inside of the server. Example:
FKey *pFKey; /* Linked list of all foreign keys in this table */, Tarantool have no FK so far.

We need to understand, how to implement fill of in-memory meta data as well.

kyukhin added a commit that referenced this issue Dec 28, 2017
SQL engine should use Tarantool core's meta information while
constructing queries. This is first step: use nullability flag
from tuple_format/tuple_field.
Since in SQL, nullability is not a flag, but one of possible actions,
including NONE (nullability is true) and abort (nullability is false)
new tuple_field's action field type was introduced with new enum type
enum, representing possible action.

Part of #2217
kyukhin added a commit that referenced this issue Dec 28, 2017
Remove macro definition OE_None, _Abort, _Replace, _Ignore, _Rollback,
_FAIL and use enum on_conflict_action throughout SQL engine source code.
Also, check column for nullability while constructing query IR if:
it is not ephemeral or it is a view.
Ephemeral tables implemented by Tarantool is not picked up so far. This
condition will be removed in nearest future.
Views have no corresponding space (and thus space_format) and this
condition will be unlikely removed in nearest perspective.

Part of #2217
kyukhin added a commit that referenced this issue Jan 9, 2018
SQL engine should use Tarantool core's meta information while
constructing queries. This is first step: use nullability flag
from tuple_format/tuple_field.
Since in SQL, nullability is not a flag, but one of possible actions,
including NONE (nullability is true) and abort (nullability is false)
new tuple_field's action field type was introduced with new enum type
enum, representing possible action.

Part of #2217
kyukhin added a commit that referenced this issue Jan 9, 2018
Remove macro definition OE_None, _Abort, _Replace, _Ignore, _Rollback,
_FAIL and use enum on_conflict_action throughout SQL engine source code.
Also, check column for nullability while constructing query IR if:
it is not ephemeral or it is a view.
Ephemeral tables implemented by Tarantool is not picked up so far. This
condition will be removed in nearest future.
Views have no corresponding space (and thus space_format) and this
condition will be unlikely removed in nearest perspective.

Part of #2217
kyukhin added a commit that referenced this issue Jan 10, 2018
Since in SQL, nullability is not a flag, but one of possible actions,
including NONE (nullability is true) and abort (nullability is false)
new tuple_field's action field type was introduced with new enum type,
representing possible action.
This new attribute is used while traslating SQL query ad set during
SQL DDL execution.

Part of #2217
kyukhin added a commit that referenced this issue Jan 15, 2018
Incapsulate is_nullable attribute of tuple_field and key_part structs:
introduce new read accessors to corresponding attribute for both
structures.
This is need for following patch, which will replace is_nullable
attribute of mentioned structures w/ on_conflict_action attribute.

Part of #2217
kyukhin added a commit that referenced this issue Jan 15, 2018
Since in SQL, nullability is not a flag, but one of possible actions,
including NONE (nullability is true) and abort (nullability is false)
new tuple_field's action field type was introduced with new enum type,
representing possible action.
This new attribute is used while traslating SQL query ad set during
SQL DDL execution.

Part of #2217
kyukhin added a commit that referenced this issue Jan 17, 2018
Remove legacy set of macro defining actions on constraint violation,
replacing it with new enum. Replace all usages throughout the SQL code.

Part of #2217
kyukhin added a commit that referenced this issue Feb 1, 2018
Incapsulate is_nullable attribute of tuple_field and key_part structs:
introduce new read accessors to corresponding attribute for both
structures.
This is need for following patch, which will replace is_nullable
attribute of mentioned structures w/ on_conflict_action attribute.

Part of #2217
kyukhin added a commit that referenced this issue Feb 1, 2018
Remove legacy set of macro defining actions on constraint violation,
replacing it with new enum. Replace all usages throughout the SQL code.

Part of #2217
kyukhin added a commit that referenced this issue Feb 1, 2018
Replace is_nullable attribute in tuple_field and key_part w/ action
enum. This was needed since SQL supports different actions on constraint
violation.
New action field was added to tuple_field_def and key_part_def as well,
making is_nullable attribute and action working consistenly.
Extend nullability test.

Part of #2217
@kyukhin kyukhin changed the title Don't combine spaceid, indexid into pageno Move legacy DD structs to Tarantool Feb 5, 2018
kyukhin added a commit that referenced this issue Feb 7, 2018
Incapsulate is_nullable attribute of tuple_field and key_part structs:
introduce new read accessors to corresponding attribute for both
structures.
This is need for following patch, which will replace is_nullable
attribute of mentioned structures w/ on_conflict_action attribute.

Part of #2217
kyukhin added a commit that referenced this issue Feb 7, 2018
Remove legacy set of macro defining actions on constraint violation,
replacing it with new enum. Replace all usages throughout the SQL code.

Part of #2217
@kostja kostja removed this from the 2.1.1 milestone Feb 11, 2018
@kostja kostja added this to the 2.0.5 milestone Feb 11, 2018
kyukhin added a commit that referenced this issue Feb 12, 2018
Remove legacy set of macro defining actions on constraint violation,
replacing it with new enum. Replace all usages throughout the SQL code.

Part of #2217
kyukhin added a commit that referenced this issue Feb 13, 2018
Incapsulate is_nullable attribute of tuple_field and key_part structs:
introduce new read accessors to corresponding attribute for both
structures.
This is need for following patch, which will replace is_nullable
attribute of mentioned structures w/ on_conflict_action attribute.

Part of #2217
kyukhin added a commit that referenced this issue Feb 13, 2018
Remove legacy set of macro defining actions on constraint violation,
replacing it with new enum. Replace all usages throughout the SQL code.

Part of #2217
@kyukhin kyukhin assigned Korablev77 and unassigned kostja Mar 13, 2018
Korablev77 added a commit that referenced this issue Aug 23, 2018
This patch finishes DD integration with Tarantool core. Within it table
cache has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema.

Closes #3561
Closes #2217
Korablev77 added a commit that referenced this issue Aug 23, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema.

Closes #3561
Closes #2217
Korablev77 added a commit that referenced this issue Aug 23, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema.

It is worth mentioning that some allocations during table and index
creations now can be moved to region, since those things are not used
after parsing and anyway destroyed.

Closes #3561
Closes #2217
Korablev77 added a commit that referenced this issue Aug 23, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema.

It is worth mentioning that some allocations during table and index
creations now can be moved to region, since those things are not used
after parsing and anyway destroyed.

Closes #3561
Closes #2217
Gerold103 pushed a commit that referenced this issue Aug 29, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema.

It is worth mentioning that some allocations during table and index
creations now can be moved to region, since those things are not used
after parsing and anyway destroyed.

Closes #3561
Closes #2217
Korablev77 added a commit that referenced this issue Sep 1, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema.

It is worth mentioning that some allocations during table and index
creations now can be moved to region, since those things are not used
after parsing and anyway destroyed.

Closes #3561
Closes #2217
Korablev77 added a commit that referenced this issue Sep 2, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema.

It is worth mentioning that some allocations during table and index
creations now can be moved to region, since those things are not used
after parsing and anyway destroyed.

Closes #3561
Closes #2217
Gerold103 pushed a commit that referenced this issue Sep 6, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema.

It is worth mentioning that some allocations during table and index
creations now can be moved to region, since those things are not used
after parsing and anyway destroyed.

Closes #3561
Closes #2217
Korablev77 added a commit that referenced this issue Sep 15, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema. Moreover, there
is no need anymore to add system spaces to separate table hash.

It is worth mentioning that some allocations during table and index
creations now can be moved to region, since those things are not used
after parsing and anyway destroyed.

Closes #3561
Closes #2217
Korablev77 added a commit that referenced this issue Sep 15, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema. Moreover, there
is no need anymore to add system spaces to separate table hash.

It is worth mentioning that some allocations during table and index
creations now can be moved to region, since those things are not used
after parsing and anyway destroyed.

Closes #3561
Closes #2217
Korablev77 added a commit that referenced this issue Sep 16, 2018
This patch finishes SQL DD integration with Tarantool core. Within it
Table hash has been removed alongside with callback routine and
OP_ParseSchema legacy opcode: we don't need anymore to update separate
SQL DD. Moreover, now we can get rid of struct Schema. Moreover, there
is no need anymore to add system spaces to separate table hash.

It is worth mentioning that some allocations during table and index
creations now can be moved to region, since those things are not used
after parsing and anyway destroyed.

Closes #3561
Closes #2217
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants