-
Notifications
You must be signed in to change notification settings - Fork 1
added explanation about stateType (streaming algs priority) #72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
RonShvarz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 1 files reviewed, 4 unresolved discussions (waiting on @Adir111)
site/learn/Streaming.md line 170 at r1 (raw file):
### Streaming Algorithm Priority Streaming algorithms have a special priority setting. To use it, you need to set the `stateType` in your algorithm spec to either **stateless** or **stateful**.
If you are experiencing slower algorithm allocation in stateless pipelines, .. these algorithms have..
site/learn/Streaming.md line 173 at r1 (raw file):
When set: - **Stateful algorithms** - A Kubernetes Job is created and launched immediately. These jobs are given the highest priority and start execution right away.
Stateful , without the word algorithms
site/learn/Streaming.md line 174 at r1 (raw file):
When set: - **Stateful algorithms** - A Kubernetes Job is created and launched immediately. These jobs are given the highest priority and start execution right away. - **Stateless algorithms** - Each algorithm runs as a Kubernetes Job, but these jobs are launched after all stateful ones.
Stateless, without algorithms,
(Remove prefix) Jobs are launched ...
site/learn/Streaming.md line 176 at r1 (raw file):
- **Stateless algorithms** - Each algorithm runs as a Kubernetes Job, but these jobs are launched after all stateful ones. - **Algorithms without a `stateType` field** - These jobs are created after the stateless ones at controlled intervals to manage cluster load and avoid excessive pod launches at once. > Note - All jobs are re-requested as needed, following the interval defined in the config settings.
unneeded
Adir111
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 1 files reviewed, 4 unresolved discussions (waiting on @Adir111 and @RonShvarz)
site/learn/Streaming.md line 170 at r1 (raw file):
Previously, RonShvarz (Ron Shvarzburd) wrote…
If you are experiencing slower algorithm allocation in stateless pipelines, .. these algorithms have..
Done.
site/learn/Streaming.md line 173 at r1 (raw file):
Previously, RonShvarz (Ron Shvarzburd) wrote…
Stateful , without the word algorithms
Done.
site/learn/Streaming.md line 174 at r1 (raw file):
Previously, RonShvarz (Ron Shvarzburd) wrote…
Stateless, without algorithms,
(Remove prefix) Jobs are launched ...
Done.
site/learn/Streaming.md line 176 at r1 (raw file):
Previously, RonShvarz (Ron Shvarzburd) wrote…
unneeded
Done.
RonShvarz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@RonShvarz reviewed 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @Adir111)
added explanation about stateType (streaming algs priority) .... bump version [skip ci]
Added explanation for stateType field in algorithm spec under
Streaming Algorithm Priority(AtStreamingpage).This change is