Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#1092 add validation interval in ingestion #1263

Merged
merged 2 commits into from
Jan 17, 2019
Merged

#1092 add validation interval in ingestion #1263

merged 2 commits into from
Jan 17, 2019

Conversation

brandon-wonjune
Copy link
Contributor

Description

데이터소스 생성시 ingestion granularity에 따라 범위를 사용자가 지정할 수 있는 기능 추가

데이터소스 생성 > ingestion 단계
2019-01-11 6 31 52

데이터소스 상세화면 (current_time)
2019-01-11 6 31 21

데이터소스 상세홤화면
2019-01-11 6 31 10

Related Issue :

#1092

How Has This Been Tested?

  1. 데이터소스 > 데이터소스 생성 > schema설정 단계에서 TIMESTAMP를 column으로 선택
  2. ingestion 설정 단계에서 데이터 범위가 나타나는지 확인
  3. 2에서 segment granularity마다 데이터 범위 validation이 작동하는지 확인
  4. 2에서 segment granularity 변경시 데이터 범위의 granularity unit이 수정되어 나타나는지 확인
  5. current_time을 TIMESTAMP로 사용할때 ingestion 설정 단계에서 데이터 범위가 나타나지 않는 것 확인

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project. it will be added soon
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document. it will be added soon
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Additional Context

@brandon-wonjune brandon-wonjune added the @datasource Component Name : Datasource/Connection Management label Jan 16, 2019
@brandon-wonjune
Copy link
Contributor Author

@alchan-lee 브랜치를 새로 생성하였습니다

@alchan-lee
Copy link
Contributor

run build
deploy to 2

@alchan-lee alchan-lee added the testbed2 Review Test Server 2 label Jan 16, 2019
@ufoscw
Copy link
Contributor

ufoscw commented Jan 17, 2019

@brandon-wonjune
Segement Granularity를 변경할때 Data Range Unit만이 아니라 값이 바뀝니다.

  1. Hour

metatron_discovery

  1. Month

metatron_discovery

@deidera08
Copy link

Segement Granularity를 변경할때 Data Range Unit만이 아니라 값이 바뀝니다.

의도한대로 동작하고 있습니다.
타임스탬프로 지정한 컬럼의 전체 데이터값을 가져오면 좋겠지만, 일부값만 알 수 있기 때문에
이전단계에서 읽어온 최소 50개의 값을 가지고 계산을 하게 됩니다.
(50개의 값 중 제일 과거값-최신값에 앞/뒤로 Segement unit가 100개가 되는 시간지점 계산)
때문에 Segement Granularity에 따라 값이 바뀝니다.

@ufoscw ufoscw self-requested a review January 17, 2019 01:00
@brandon-wonjune
Merge branch 'master' of https://github.com/metatron-app/metatron-dis…
a17af07 
…covery into f#1092-fix_add_interval_in_ingestion

# Conflicts:
#	discovery-frontend/src/app/data-storage/data-source-list/component/advanced-setting.component.html
#	discovery-frontend/src/assets/i18n/en.json
#	discovery-frontend/src/assets/i18n/ko.json
@alchan-lee alchan-lee merged commit a2f5cb2 into master Jan 17, 2019
@alchan-lee alchan-lee deleted the f#1092-fix_add_interval_in_ingestion branch January 18, 2019 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ufoscw ufoscw ufoscw approved these changes

@deidera08 deidera08 deidera08 approved these changes

@kyungtaak kyungtaak Awaiting requested review from kyungtaak

@eltriny eltriny Awaiting requested review from eltriny

@alchan-lee alchan-lee Awaiting requested review from alchan-lee

@choonglee170924 choonglee170924 Awaiting requested review from choonglee170924

@sting009 sting009 Awaiting requested review from sting009

Assignees
No one assigned
Labels
@datasource Component Name : Datasource/Connection Management testbed2 Review Test Server 2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@brandon-wonjune @alchan-lee @ufoscw @deidera08