[Feature][Zeta] Improve local mode startup request ports

[liunaijie]

Purpose of this pull request

close #4607
tested with this command:

./bin/seatunnel.sh --config ./config/v2.batch.config.template -e local

the result is:

this is same pr with this. I had some missteps that caused the previous PR to be closed.

Check list

• Code changed are covered with tests, or it does not need tests for reason:
• If any new Jar binary package adding in your PR, please add License Notice according
  New License Guide
• If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
• If you are contributing the connector code, please check that the following files are updated:
  1. Update change log that in connector document. For more details you can refer to connector-v2
  2. Update plugin-mapping.properties and add new connector information in it
  3. Update the pom file of seatunnel-dist
• Update the release-note.

[liunaijie]

also tested with the SeaTunnelEngineExample i am not sure why there has some action failed.

[liunaijie]

@hailin0 PTAL

[EricJoy2048]

Can you add test case for this change? Like use e2e create two local mode job in one container at same time.

[hailin0]

Can you add test case for this change? Like use e2e create two local mode job in one container at same time.

+1

[liunaijie]

this looks a bug

engine-v2-it failed due to ClusterFaultToleranceTwoPipelineIT.testTwoPipelineStreamJobRestoreIn2NodeMasterDown:732 » ConditionTimeout

I checked this method, this method crate 2 hazelcast serve instance, and shutdown 1 instance when some tasks commit finished.
when the left node became master node, and start recover job, get NPE, zeta canceled the job, so the check won't be success. when wait time is over, failed.

Please see the screenshot i attached.

error log

the left node change to master

get NPE

task canceled

[liunaijie]

retest

[liunaijie]

@hailin0 please rerun the actions
in the previous failed actions, i find zeta engine has an bug. when restore task, there has somewhere past a NULL to SeaTunnelTask.restoreState method, in this method doesn't have parameter check, so get NPE and cause cancel the job. so i added a parameter check code.
Also i think we also need to find the root cause, where generated a NULL and past to this method.

[EricJoy2048]

Please fix the code style error.

[liunaijie]

Please fix the code style error.

done

[liunaijie]

@EricJoy2048 @hailin0 . now this config has error. the assert sink has a min value check but the fake source doesn't has the min value config.

For Now, this pr has 3 commit:

1. for issue [Feature][Zeta] Improve local mode startup request ports  #4607 , improve local mode startup request ports
2. fix zeta engine recover job get NPE. (but it's not the finially solution, just added parameter check, need find where past the NULL to this method)
3. fix Assert e2e config error

[hailin0]

@EricJoy2048 @hailin0 . now this config has error. the assert sink has a min value check but the fake source doesn't has the min value config.

For Now, this pr has 3 commit:

1. for issue [Feature][Zeta] Improve local mode startup request ports  #4607 , improve local mode startup request ports
2. fix zeta engine recover job get NPE. (but it's not the finially solution, just added parameter check, need find where past the NULL to this method)
3. fix Assert e2e config error

3. suggestion remove MIN check from assert sink

[liunaijie]

@EricJoy2048 @hailin0 . now this config has error. the assert sink has a min value check but the fake source doesn't has the min value config.
For Now, this pr has 3 commit:

1. for issue [Feature][Zeta] Improve local mode startup request ports  #4607 , improve local mode startup request ports
2. fix zeta engine recover job get NPE. (but it's not the finially solution, just added parameter check, need find where past the NULL to this method)
3. fix Assert e2e config error

3. suggestion remove MIN check from assert sink

I think in the assert e2e check, this is a good validaction rule, we just need update the source config.

[hailin0]

@EricJoy2048 @hailin0 . now this config has error. the assert sink has a min value check but the fake source doesn't has the min value config.
For Now, this pr has 3 commit:

1. for issue [Feature][Zeta] Improve local mode startup request ports  #4607 , improve local mode startup request ports
2. fix zeta engine recover job get NPE. (but it's not the finially solution, just added parameter check, need find where past the NULL to this method)
3. fix Assert e2e config error

3. suggestion remove MIN check from assert sink

I think in the assert e2e check, this is a good validaction rule, we just need update the source config.

agree

[hailin0]

@EricJoy2048 @hailin0 . now this config has error. the assert sink has a min value check but the fake source doesn't has the min value config.

For Now, this pr has 3 commit:

1. for issue [Feature][Zeta] Improve local mode startup request ports  #4607 , improve local mode startup request ports
2. fix zeta engine recover job get NPE. (but it's not the finially solution, just added parameter check, need find where past the NULL to this method)
3. fix Assert e2e config error

@EricJoy2048
What is your opinion on question 2

[EricJoy2048]

@EricJoy2048 @hailin0 . now this config has error. the assert sink has a min value check but the fake source doesn't has the min value config.
For Now, this pr has 3 commit:

1. for issue [Feature][Zeta] Improve local mode startup request ports  #4607 , improve local mode startup request ports
2. fix zeta engine recover job get NPE. (but it's not the finially solution, just added parameter check, need find where past the NULL to this method)
3. fix Assert e2e config error

@EricJoy2048 What is your opinion on question 2

It's ok.

[liunaijie]

@hailin0 please help review and merge this pr, thanks.