Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Pulsar Functions localrun to exit on error #12278

Merged
merged 3 commits into from
Oct 6, 2021

Conversation

jerrypeng
Copy link
Contributor

Motivation

There are situations where it is useful for localrun to exit if the instance it is running encounters an error. This is useful for situations where the localrun is being orchestrated by a scheduler like K8s. If there are errors in the function, it is easier to detect, observe, and alert on issues if the localrun is consisting exiting / crash loop.

Modifications

  1. Allow localrun to exist if the function encounters an error
  2. Fix inconsistencies in Pulsar IO Sink error handling. Currently, if a Pulsar IO sink throws exception during write. The error is only logged and not bubble up which is different than the behavior for functions and sources.

Verifying this change

Add tests.

@jerrypeng jerrypeng added this to the 2.9.0 milestone Oct 5, 2021
@jerrypeng jerrypeng self-assigned this Oct 5, 2021
@jerrypeng jerrypeng requested review from sijie, eolivelli and lhotari and removed request for sijie and eolivelli October 5, 2021 06:22
@jerrypeng jerrypeng merged commit cd9356a into apache:master Oct 6, 2021
dlg99 pushed a commit to dlg99/pulsar that referenced this pull request Mar 4, 2022
Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit cd9356a)
dlg99 pushed a commit to dlg99/pulsar that referenced this pull request Mar 4, 2022
Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit cd9356a)
eolivelli pushed a commit to datastax/pulsar that referenced this pull request Mar 7, 2022
Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit cd9356a)
eolivelli pushed a commit to datastax/pulsar that referenced this pull request Mar 7, 2022
Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit cd9356a)
dlg99 pushed a commit to dlg99/pulsar that referenced this pull request Mar 7, 2022
Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit cd9356a)
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
Co-authored-by: Jerry Peng <jerryp@splunk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants