Skip to content

Conversation

skunert
Copy link
Contributor

@skunert skunert commented Jun 10, 2014

This PR enables that forwarded fields are considered in the optimizer when they are not forwarded to the same output field.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a syntax error.

@skunert
Copy link
Contributor Author

skunert commented Jun 23, 2014

Updated to be automatically mergeable.

@StephanEwen
Copy link
Contributor

This is a good feature addition. I think we should address a few points before merging it:

  • The old code had the isFieldConstant() method on the optimizer nodes. You are adding more methods for forwarded fields. I think better design is to remove all of those methods and directly supply the SemanticProperties to the (Requested)Global/LocalProperties. The Semantic properties would need a method like getForwardedFields(int input, int field).
  • The methods in the (Requested)Global/LocalProperties can then be renamed to filterBySemanticProperties(), rather than filterByNodesConstantSet().
  • The LocalProperties have recently been changed to be an immutable type. That is much less error-prone in applications like the optimizer. The getters (for the set of unique fields) should also return immutable types (like Collections.unmodifiableSet(set)). Similar things would be good for the GlobalProperties, RequestedGlobalProperties, and RequestedLocalProperties.
  • Given that this is really logic is sensitive and crucial to the correctness of the optimizer, I think we need to have a few dedicated tests for the filterBySemanticProperties() method in the classes GlobalProperties, LocalProperties, RequestedGlobalProperties, and RequestedLocalProperties. There is currently only one test that test end-to-end some properties, but leaves out many cases.

qmlmoon pushed a commit to qmlmoon/incubator-flink that referenced this pull request Jul 21, 2014
Fix CrossOperator does not work
@skunert skunert closed this Jul 29, 2014
@skunert skunert deleted the constantFields_Optimizer branch July 29, 2014 13:47
vasia added a commit to vasia/flink that referenced this pull request Jan 16, 2015
Changes functions to static inner classes and adds initial tests
tillrohrmann referenced this pull request in tillrohrmann/flink Jan 16, 2015
vasia added a commit to vasia/flink that referenced this pull request Jan 22, 2015
Changes functions to static inner classes and adds initial tests
mxm added a commit to mxm/flink that referenced this pull request Feb 2, 2015
mxm added a commit to mxm/flink that referenced this pull request Feb 3, 2015
kerberos apache#2

kerberos apache#3

kerberos apache#3

kerberos apache#4

kerberos apache#5

bla

last test

blub

bla

blub

jo

ll

afaf

haha

null

jo

just testing

final?

blub

uiui

haha

jojojo
mxm added a commit to mxm/flink that referenced this pull request Mar 12, 2015
mxm added a commit to mxm/flink that referenced this pull request Apr 10, 2015
mxm added a commit to mxm/flink that referenced this pull request Apr 12, 2015
mxm added a commit to mxm/flink that referenced this pull request Apr 13, 2015
milliondreams pushed a commit to milliondreams/flink that referenced this pull request Nov 6, 2015
asfgit pushed a commit that referenced this pull request Nov 24, 2015
mxm added a commit to mxm/flink that referenced this pull request Feb 8, 2016
WIP: [standalone] start only when local jobmanager is leader

fix

React to "RegisterResourceManager"

TaskManagerAvailable & TaskManagerRemoved

fixes

WIP: LookupResourceManager & LookupResourceManagerReply

Send register to Resource Manager rather than to JobManager

disconnect

Tell TaskManagers to re-register with new Resource Manager after re-conciliation

cleanup Disconnect

fixes

shutdown

[tests] fix

fixes

test adaption

test adaption apache#2

add TestingResourceManager

fix

WIP

finish test adaption for JobManagerRegistrationTest

consistency: rename StandaloneResourceMaster to StandaloneResourceManager

TestingTaskManager: generate default resource id

formatting

WIP

blub

remove special code path for local resource manager

start resource manager separately

cleanup

resouce id passing and test adaptation

instance registratition fix

fix

[test-logging]
mxm added a commit to mxm/flink that referenced this pull request Feb 11, 2016
WIP: [standalone] start only when local jobmanager is leader

fix

React to "RegisterResourceManager"

TaskManagerAvailable & TaskManagerRemoved

fixes

WIP: LookupResourceManager & LookupResourceManagerReply

Send register to Resource Manager rather than to JobManager

disconnect

Tell TaskManagers to re-register with new Resource Manager after re-conciliation

cleanup Disconnect

fixes

shutdown

[tests] fix

fixes

test adaption

test adaption apache#2

add TestingResourceManager

fix

WIP

finish test adaption for JobManagerRegistrationTest

consistency: rename StandaloneResourceMaster to StandaloneResourceManager

TestingTaskManager: generate default resource id

formatting

WIP

blub

remove special code path for local resource manager

start resource manager separately

cleanup

resouce id passing and test adaptation

instance registratition fix

fix

[test-logging]
mxm added a commit to mxm/flink that referenced this pull request Feb 12, 2016
WIP: [standalone] start only when local jobmanager is leader

fix

React to "RegisterResourceManager"

TaskManagerAvailable & TaskManagerRemoved

fixes

WIP: LookupResourceManager & LookupResourceManagerReply

Send register to Resource Manager rather than to JobManager

disconnect

Tell TaskManagers to re-register with new Resource Manager after re-conciliation

cleanup Disconnect

fixes

shutdown

[tests] fix

fixes

test adaption

test adaption apache#2

add TestingResourceManager

fix

WIP

finish test adaption for JobManagerRegistrationTest

consistency: rename StandaloneResourceMaster to StandaloneResourceManager

TestingTaskManager: generate default resource id

formatting

WIP

blub

remove special code path for local resource manager

start resource manager separately

cleanup

resouce id passing and test adaptation

instance registratition fix

fix

[test-logging]
mxm added a commit to mxm/flink that referenced this pull request Feb 15, 2016
WIP: [standalone] start only when local jobmanager is leader

fix

React to "RegisterResourceManager"

TaskManagerAvailable & TaskManagerRemoved

fixes

WIP: LookupResourceManager & LookupResourceManagerReply

Send register to Resource Manager rather than to JobManager

disconnect

Tell TaskManagers to re-register with new Resource Manager after re-conciliation

cleanup Disconnect

fixes

shutdown

[tests] fix

fixes

test adaption

test adaption apache#2

add TestingResourceManager

fix

WIP

finish test adaption for JobManagerRegistrationTest

consistency: rename StandaloneResourceMaster to StandaloneResourceManager

TestingTaskManager: generate default resource id

formatting

WIP

blub

remove special code path for local resource manager

start resource manager separately

cleanup

resouce id passing and test adaptation

instance registratition fix

fix

[test-logging]
mxm added a commit to mxm/flink that referenced this pull request Feb 19, 2016
WIP: [standalone] start only when local jobmanager is leader

fix

React to "RegisterResourceManager"

TaskManagerAvailable & TaskManagerRemoved

fixes

WIP: LookupResourceManager & LookupResourceManagerReply

Send register to Resource Manager rather than to JobManager

disconnect

Tell TaskManagers to re-register with new Resource Manager after re-conciliation

cleanup Disconnect

fixes

shutdown

[tests] fix

fixes

test adaption

test adaption apache#2

add TestingResourceManager

fix

WIP

finish test adaption for JobManagerRegistrationTest

consistency: rename StandaloneResourceMaster to StandaloneResourceManager

TestingTaskManager: generate default resource id

formatting

WIP

blub

remove special code path for local resource manager

start resource manager separately

cleanup

resouce id passing and test adaptation

instance registratition fix

fix

[test-logging]
mxm added a commit to mxm/flink that referenced this pull request Feb 20, 2016
WIP: [standalone] start only when local jobmanager is leader

fix

React to "RegisterResourceManager"

TaskManagerAvailable & TaskManagerRemoved

fixes

WIP: LookupResourceManager & LookupResourceManagerReply

Send register to Resource Manager rather than to JobManager

disconnect

Tell TaskManagers to re-register with new Resource Manager after re-conciliation

cleanup Disconnect

fixes

shutdown

[tests] fix

fixes

test adaption

test adaption apache#2

add TestingResourceManager

fix

WIP

finish test adaption for JobManagerRegistrationTest

consistency: rename StandaloneResourceMaster to StandaloneResourceManager

TestingTaskManager: generate default resource id

formatting

WIP

blub

remove special code path for local resource manager

start resource manager separately

cleanup

resouce id passing and test adaptation

instance registratition fix

fix

[test-logging]
mxm added a commit to mxm/flink that referenced this pull request Feb 29, 2016
WIP: [standalone] start only when local jobmanager is leader

fix

React to "RegisterResourceManager"

TaskManagerAvailable & TaskManagerRemoved

fixes

WIP: LookupResourceManager & LookupResourceManagerReply

Send register to Resource Manager rather than to JobManager

disconnect

Tell TaskManagers to re-register with new Resource Manager after re-conciliation

cleanup Disconnect

fixes

shutdown

[tests] fix

fixes

test adaption

test adaption apache#2

add TestingResourceManager

fix

WIP

finish test adaption for JobManagerRegistrationTest

consistency: rename StandaloneResourceMaster to StandaloneResourceManager

TestingTaskManager: generate default resource id

formatting

WIP

blub

remove special code path for local resource manager

start resource manager separately

cleanup

resouce id passing and test adaptation

instance registratition fix

fix

[test-logging]
XComp referenced this pull request in XComp/flink Jul 1, 2021
XComp referenced this pull request in XComp/flink Jul 2, 2021
dmvk referenced this pull request in dmvk/flink Aug 13, 2021
rkhachatryan added a commit to rkhachatryan/flink that referenced this pull request Sep 8, 2021
rkhachatryan added a commit to rkhachatryan/flink that referenced this pull request Sep 8, 2021
rkhachatryan added a commit to rkhachatryan/flink that referenced this pull request Sep 16, 2021
CrynetLogistics added a commit to CrynetLogistics/flink that referenced this pull request Nov 8, 2021
XComp referenced this pull request in ververica/flink Nov 17, 2021
nicoweidner referenced this pull request in ververica/flink Nov 18, 2021
chengkaiyang2025 added a commit to chengkaiyang2025/flink-25705 that referenced this pull request May 2, 2022
parent 8d00622
author Chengkai Yang <yangchengkai2022@163.com> 1650023265 +0800
committer Chengkai Yang <yangchengkai2022@163.com> 1651508979 +0800

# This is a combination of 3 commits.
# This is the 1st commit message:

[FLINK-25705][docs]Translate "Metric Reporters" page of "Deployment" in to Chinese.

# This is the commit message apache#2:

[FLINK-25705][docs]Translate "Metric Reporters" page of "Deployment" in to Chinese.

# This is the commit message apache#3:

[FLINK-25705][docs]Translate "Metric Reporters" page of "Deployment" in to Chinese.

# This is the commit message apache#4:

[FLINK-25705][docs]Translate some description in the table of html file.
zhuzhurk added a commit to zhuzhurk/flink that referenced this pull request Sep 9, 2022
HuangZhenQiu pushed a commit to HuangZhenQiu/flink that referenced this pull request Sep 20, 2022
czy006 added a commit to czy006/flink that referenced this pull request Jan 9, 2023
yuta-cloud pushed a commit to yuta-cloud/flink-me that referenced this pull request Apr 27, 2023
czy006 added a commit to funny/flink that referenced this pull request Jun 6, 2023
czy006 added a commit to czy006/flink that referenced this pull request Jun 6, 2023
cbornet added a commit to cbornet/flink that referenced this pull request Jun 6, 2023
czy006 added a commit to czy006/flink that referenced this pull request Jun 16, 2023
czy006 added a commit to czy006/flink that referenced this pull request Jun 16, 2023
czy006 added a commit to czy006/flink that referenced this pull request Jun 21, 2023
zentol added a commit that referenced this pull request Nov 1, 2023
ConfluentSemaphore referenced this pull request in confluentinc/flink-public Aug 6, 2024
* Adds Compiled Plan annotations to BatchExecSort and BatchExecExchange.
* Tests the new annotations with the existing SortTestPrograms.
jnh5y added a commit to jnh5y/flink that referenced this pull request Aug 6, 2024
* Adds Compiled Plan annotations to BatchExecSort and BatchExecExchange.
* Tests the new annotations with the existing SortTestPrograms.
dinchamion pushed a commit to dinchamion/flink that referenced this pull request Feb 23, 2025
Also remove the deprecated Table catalog/source/sink because the latter is incompatible with the new sink.
seb-pereira pushed a commit to seb-pereira/flink that referenced this pull request Jun 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants