Skip to content
This repository has been archived by the owner on Feb 21, 2024. It is now read-only.

DSL redesign #13

Closed
walterddr opened this issue Jun 10, 2019 · 1 comment
Closed

DSL redesign #13

walterddr opened this issue Jun 10, 2019 · 1 comment

Comments

@walterddr
Copy link
Owner

walterddr commented Jun 10, 2019 •
edited
Loading

See:
https://docs.google.com/document/d/1L_HMYS1MSkN7CYpPjNjI1QJ9QaWMMTbH53V1J7gs3Y4/edit#

Several components needed:

  1. restructure of the Flux API module (DSL Syntax in Flux API module #11)
  2. moved current runner to the new module (adding in flink-flux-runner #12)
  3. create a ordered edge system needed by Flink (Support ordering in stream #15)
  4. create a flux-level compilation graph generator in flux-core and remove it from the flink-flux compilation core level (Under discussion whether it should fit in DSL Compiler Redesign #14 or here)

Additional components can be adjusted but will not be included in this DSL redesign.
@walterddr walterddr mentioned this issue Jun 10, 2019
Closed
walterddr added a commit that referenced this issue Jun 10, 2019
@walterddr
adding in flink-flux-runner (#12)
a9487b2 
Initial commit for the change that only reflect the copying from previous implementation

Several components needs to be fixed:
1. compiler factory discovery mechanism - how to find correct compiler (e.g. compiling into a "vertex object")
2. linker factory discovery mechanism - how to find linkage method based on
    a. incoming stream type
    b. outgoing stream type
    c. vertex object type
3. the fundamental base-class mechanism - how to make sure this overlaps with Flink's API

More detail in #12 and #13
@walterddr walterddr mentioned this issue Jun 12, 2019
Closed
@walterddr
Copy link
Owner Author

Moving point(4) to #14 and closing this issue.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@walterddr