Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for online table alters #50

Closed
5 tasks done
Shelnutt2 opened this issue Jan 7, 2018 · 1 comment
Closed
5 tasks done

Add support for online table alters #50

Shelnutt2 opened this issue Jan 7, 2018 · 1 comment

Comments

@Shelnutt2
Copy link
Owner

Shelnutt2 commented Jan 7, 2018

The idea is to add support for online table alters by creating new version of the schema. We can then produce a migration of data on read. This would put the work on reads instead of causing a table rebuild when a column is changed. During file consolidation older data files can be upgraded to the latest version of the schema.

If column order is what is being changed nothing has to be done. Even if ondisk storage and capnp proto schema is in different order, the fields are set using the field interface, so no changes are needed.

If column name changes, then we can just create a new capn proto schema with the new names, but as long as order and data types are the same, then the data on disk doesn't need to change.

Changes are needed for:

  • ADD_COLUMN
  • ALTER_COLUMN_DEFAULT
  • ALTER_COLUMN_NULLABLE
  • ALTER_COLUMN_FORMAT
  • ALTER_COLUMN_STORAGE_TYPE

Implement:

  • check_if_supported_inplace_alter (TABLE *altered_table, Alter_inplace_info *ha_alter_info)
  • prepare_inplace_alter_table(TABLE *altered_table, Alter_inplace_info *ha_alter_info);
  • inplace_alter_table(TABLE *altered_table, Alter_inplace_info *ha_alter_info)
  • commit_inplace_alter_table(TABLE *altered_table, Alter_inplace_info *ha_alter_info, bool commit);
  • notify_table_changed();
Shelnutt2 added a commit that referenced this issue Jan 20, 2018
Shelnutt2 added a commit that referenced this issue Jan 21, 2018
When an alter table that is not inline takes place
mariadb will create a new tmp table whose name
typically includes special characters.
Shelnutt2 added a commit that referenced this issue Jan 21, 2018
Shelnutt2 added a commit that referenced this issue Jan 21, 2018
Shelnutt2 added a commit that referenced this issue Jan 21, 2018
Shelnutt2 added a commit that referenced this issue Jan 21, 2018
This works by renaming the folder, and updating the
cap'n proto schema files with the new table name.
Shelnutt2 added a commit that referenced this issue Jan 24, 2018
Currently only renaming columns is supported online
Shelnutt2 added a commit that referenced this issue Jan 28, 2018
@Shelnutt2
Copy link
Owner Author

Right now the only miss component is default values need to be added to the schema so when messages are updated, and new columns have no values, we don't have to take up space by adding it, we should just use capnp default value support.

Shelnutt2 added a commit that referenced this issue Feb 8, 2018
Ideally this would have been done by setting constants if a default is
set but its become difficult to get constants if we don't have valid
**Field to read from.
Shelnutt2 added a commit that referenced this issue Feb 18, 2018


Use vector of fields, instead of reading from table obj. This allows to
alter the fields, in order to keep the records compatible without
altering the mysql table object directly.
Shelnutt2 added a commit that referenced this issue Feb 18, 2018
Use vector of fields, instead of reading from table obj. This allows to
alter the fields, in order to keep the records compatible without
altering the mysql table object directly.
Shelnutt2 added a commit that referenced this issue Feb 18, 2018
If only column name changes take place, then the schemas are considered
compatible and the new schema will simply read the old message. We only
need to update the message to a new schema version if there are
incompatible changes (dropped/added columns, changing of data types).
Shelnutt2 added a commit that referenced this issue Feb 18, 2018
If only column name changes take place, then the schemas are considered
compatible and the new schema will simply read the old message. We only
need to update the message to a new schema version if there are
incompatible changes (dropped/added columns, changing of data types).
Shelnutt2 added a commit that referenced this issue Feb 21, 2018
This test fails on travis ci, but the failure can not be reproduced on
any other platform. It does not happen on test system with gcc-5, gcc-6
or gcc-7. It also does not happen on arm in testing. Since it can not be
preproduced it is being disabled.
Shelnutt2 added a commit that referenced this issue Feb 21, 2018
This test fails on travis ci, but the failure can not be reproduced on
any other platform. It does not happen on test system with gcc-5, gcc-6
or gcc-7. It also does not happen on arm in testing. Since it can not be
preproduced it is being disabled.
Shelnutt2 added a commit that referenced this issue Feb 21, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant