Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using URI to Create a File Type Snapshot #1306

Closed
sang-hee-lee opened this issue Jan 23, 2019 · 1 comment
Closed

Using URI to Create a File Type Snapshot #1306

sang-hee-lee opened this issue Jan 23, 2019 · 1 comment
Assignees
Labels
@dataprep Component Name : Data preparation enhancement Request Change and Feature Enhancement
Milestone

Comments

@sang-hee-lee
Copy link
Contributor

sang-hee-lee commented Jan 23, 2019

Is your feature request related to a problem? Please describe.
Using URI to Create a File Type Snapshot

Determining the URI in the UI, and thus determining the attributes of the snapshot on the server, has the following implications:
In the meantime, some part of the snapshot URI came from UI, and the rest part came from server's settings or choices from UI pop-ups then we decided the URI as the final result of complex conditional expressions. Using the URI determined by UI, I expect to be able to avoid many confusing cases.

UI에서 URI를 확정하고, 그에 따라 서버에서 snapshot의 속성을 정하는 것은 다음의 의의가 있습니다.
그동안 UI에서 일부 이름을 주고, 서버에서는 설정 및 사용자가 지정한 파일 포맷에 따라, 복잡한 조건식으로 snapshot의 경로를 결정했었습니다. 이제 UI에서 확정한 URI를 그대로 사용함으로써 혼동을 피할 수 있을 것으로 기대합니다.

Describe the solution you'd like
Using URI to Create a File Type Snapshot
URI can be modified by the user

In the UI, just set the URI scheme according to the storage type, and suffix it according to the file format. The server will determine the following accordingly.

  1. storedUri of Snapshot
  2. File format (CSV, JSON)
  3. Location type (LOCAL, HDFS, URI scheme)

UI에서는 단지 storage type에 따라 URI의 scheme을 정하고, file format에 따라 suffix를 붙여주면 됩니다.
서버에서는 그에 따라 다음을 결정할 예정입니다.

1. Snapshot의 storedUri
2. File format (CSV, JSON)
3. Location 종류 (LOCAL, HDFS 여부. URI의 scheme)

Additional context
Spaces, special characters, etc. that are difficult to use as a file path should be avoided from the UI as much as possible, but it is better to set the policies correctly to other issues.

공백, 특수문자 등, 파일 경로로 쓰이기 어려운 것은 최대한 UI에서부터 거르면 좋을 것 같습니다만, 이것은 다른 이슈로 정확하게 정책을 정해서 처리하는 것이 좋겠습니다.

@sang-hee-lee sang-hee-lee added the @dataprep Component Name : Data preparation label Jan 23, 2019
@sang-hee-lee sang-hee-lee added this to the 3.2.0 milestone Jan 23, 2019
@joohokim1 joohokim1 added the enhancement Request Change and Feature Enhancement label Jan 24, 2019
joohokim1 added a commit that referenced this issue Jan 28, 2019
joohokim1 added a commit that referenced this issue Jan 28, 2019
@joohokim1
Copy link
Contributor

2019-01-28 11 37 29

@sang-hee-lee @paigechoi

Information such as storageType and uriFileFormat are now ignored. (Or server just uses them to assert in comparison to storedUri). Please modify UI code to not to send the above information.
And you can see the old field that is not used at all. (format in the screenshot above)
Also, if possible, do not send fields that are not used depending on the snapshot type.
In this case dbName, hiveFileCompression are not used in URI snapshot.

storageType, uriFileFormat 등의 정보는 이제 무시됩니다. (아니면 storedUri와 비교해서 assert를 하는 용도밖엔 없습니다) UI에서 위의 정보들을 보내지 않도록 수정해주시면 좋겠습니다.
그리고 전혀 안쓰이는 옛날의 필드가 보입니다. (위의 스크린샷에서 format)
또 가능하면, 스냅샷 타입에 따라서 쓰이지 않는 필드들은 보내지 않도록 해주세요.
위의 경우 dbName, hiveFileCompression은 URI snapshot에선 쓰이지 않는 값입니다.

joohokim1 added a commit that referenced this issue Jan 30, 2019
* #1306 modify storedUri in a File Type Snapshot

* #1306 use storedUri from UI directly

* #1306 removed storageType, uriFileFormat from snapshot request (derived by storedUri)

* #1306 Remove unused fields from URI snapshots
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@dataprep Component Name : Data preparation enhancement Request Change and Feature Enhancement
Projects
None yet
Development

No branches or pull requests

2 participants