Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add output data streams to python functions #1181

Merged
merged 12 commits into from
Feb 9, 2023

Conversation

SvenO3
Copy link
Contributor

@SvenO3 SvenO3 commented Jan 30, 2023

Purpose

This PR implements the support for output data streams for the python functions. This makes it possible to create a new data stream out of python and to write back live data to StreamPipes via that stream.
It also provides an implementation to use Online Machine Learning with the StreamPipesFunctions using the River library. You can now train an Online ML model on a data stream and sends the prediction back to StreamPipes with very little effort.

Remarks

PR introduces (a) breaking change(s): no

PR introduces (a) deprecation(s): no

@SvenO3 SvenO3 added enhancement New feature or request python Pull requests that update Python code labels Jan 30, 2023
@SvenO3 SvenO3 added this to the 1.0.0 milestone Jan 30, 2023
@SvenO3 SvenO3 requested a review from bossenti January 30, 2023 13:35
@SvenO3 SvenO3 linked an issue Jan 30, 2023 that may be closed by this pull request
4 tasks
Copy link
Contributor

@bossenti bossenti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work @SvenO3 🥇

@@ -0,0 +1,16 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets move this into a dedicated module called function_zoo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should function_zoo be a submodule of the functions module. Or should it be seperated to differentiate between implementation and examples?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would see it as separate module on the same level like functions.
Should also be beneficial in terms of imports

@bossenti
Copy link
Contributor

bossenti commented Feb 3, 2023

almost forgot: please rebase the PR before we merge it because we upgraded black

@bossenti
Copy link
Contributor

bossenti commented Feb 6, 2023

Needs to be merged in sync with #1228

@SvenO3
Copy link
Contributor Author

SvenO3 commented Feb 7, 2023

Needs to be merged in sync with #1228

What's the best way to do this?

@bossenti
Copy link
Contributor

bossenti commented Feb 7, 2023

Well, one of us has to do some work 😅
Either we merge #1228 first, then you need to update this PR accordingly or we do it the way around :)

@SvenO3 SvenO3 merged commit 6f15017 into apache:dev Feb 9, 2023
@SvenO3 SvenO3 deleted the improving-sp-functions branch February 13, 2023 08:16
@bossenti bossenti removed this from the 1.0.0 milestone Feb 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request python Pull requests that update Python code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Output data streams for python functions
2 participants