Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add map filter dataset code generation #15

Closed
harbby opened this issue Feb 10, 2021 · 0 comments · Fixed by #18
Closed

Add map filter dataset code generation #15

harbby opened this issue Feb 10, 2021 · 0 comments · Fixed by #18

Comments

@harbby
Copy link
Owner

harbby commented Feb 10, 2021

该patch将引入code generation。旨在提高业务代码密度。
以这个例子为例:
dataSource.map(x->x+1).map(x->x * 3).filter(x % 2 ==0).count()
将生成如下物理执行计划:

...

通过code generation后将优化为如下伪代码:

while(dataSource.hashNext()) {
     input = dataSource.nextRow()
     input = input + 1
     input = input * 3
     if (input % 2 ==0)  {
            collect.collect(input)
      }
}

现代框架普遍存在大框架小业务的尴尬(俗称太重)。在提高密度的同时也面向过程的开始,将所有OO抽象一层层剥离。
理想的情况下。所有处理数据的业务代码将全部压缩在一个循环体中。这也是人在面向过程手写代码的形式。

高密度代码在性能上具有非常多的优势,这里暂略。

@harbby harbby changed the title Add DataSet code generation Add map filter dataset code generation Feb 23, 2021
@harbby harbby linked a pull request Feb 25, 2021 that will close this issue
@harbby harbby closed this as completed in #18 Mar 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant