Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: migration of dataprep entities #1046

Closed
joohokim1 opened this issue Dec 7, 2018 · 0 comments
Closed

bugfix: migration of dataprep entities #1046

joohokim1 opened this issue Dec 7, 2018 · 0 comments
Assignees
Labels
bug Something isn't working @dataprep Component Name : Data preparation
Milestone

Comments

@joohokim1
Copy link
Contributor

Describe the bug
The following problems occur when migrating entities related to dataprep.

  1. When migrating the dataflow-dataset relation table, if the dataset or dataflow entity is missing, you will get a foreign key constraint exception.
  2. When there's CLOB columns in the original entity, you'll get a cannot cast exception. (For MySQL it works fine, but for H2 it's not)

Dataprep 관련 entity들을 migrate하는 데에 다음의 문제들이 발생합니다.

  1. 존재하지 않는 dataset 또는 dataflow간의 관계 table migration이 foreign key constraint exception 발생
  2. CLOB column을 옮길 때에 String으로 casting하다가 발생하는 문제 (MySQL은 제대로 동작. H2는 에러 발생)

To Reproduce
Steps to reproduce the behavior:

  1. Startup with setting polaris.dataprep.pmonInterval: 30
  2. Bootup fails if you use H2 as the meta DB.
  3. If there are some rows in prep_dataflow_dataset that corresponding dataflow or dataset is missing, you'll see error log, every pmonInterval seconds.

Expected behavior

  1. Startup should succeed with H2 meta DB.
  2. Only relations with existing foreign keys should be migrated.

Screenshots
2018-12-07 4 38 22

Desktop (please complete the following information):

  • OS: MacOS
  • Browser: Chrome
  • Version: 68.0.3440.106
@joohokim1 joohokim1 added bug Something isn't working @dataprep Component Name : Data preparation labels Dec 7, 2018
@joohokim1 joohokim1 added this to the 3.1.0 milestone Dec 7, 2018
@joohokim1 joohokim1 self-assigned this Dec 7, 2018
@joohokim1 joohokim1 changed the title bugfix: migration of dataprep entites bugfix: migration of dataprep entities Dec 7, 2018
joohokim1 added a commit that referenced this issue Dec 7, 2018
joohokim1 added a commit that referenced this issue Dec 7, 2018
* #1046 bugfix: ignore missing dataflow-dataest relation, resolved CLOB casting issue.

* #1046 enabled dataprep meta db migration by default
alchan-lee pushed a commit that referenced this issue Dec 7, 2018
* #1046 bugfix: ignore missing dataflow-dataest relation, resolved CLOB casting issue.

* #1046 enabled dataprep meta db migration by default
joohokim1 added a commit that referenced this issue Dec 11, 2018
* #1046 bugfix: skip when old schema does not exist

* #1046 suppress printing call stack trace in normal case.
alchan-lee pushed a commit that referenced this issue Dec 11, 2018
* #1046 bugfix: skip when old schema does not exist

* #1046 suppress printing call stack trace in normal case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working @dataprep Component Name : Data preparation
Projects
None yet
Development

No branches or pull requests

2 participants