[Merged by Bors] - feat(smartmodule): added chaining support to smdk test #2784
Conversation
|
|
||
| # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html | ||
|
|
||
| [features] |
crates/fluvio/src/smartmodule.rs
Outdated
| /// Ok(smart_module) | ||
| /// } | ||
| /// ``` | ||
| pub struct FluvioSmartModule { |
There was a problem hiding this comment.
This is bit ambigious . Maybe SmartModuleAdmin ? or SmartModuleManager
There was a problem hiding this comment.
done. Renamed to SmartModuleAdmin
crates/fluvio/Cargo.toml
Outdated
| cfg-if = "1.0.0" | ||
| derive_builder = "0.11.0" | ||
| async-trait = "0.1.51" | ||
| flate2 = "1.0" |
There was a problem hiding this comment.
should be optional and depend on smartengine
There was a problem hiding this comment.
Those are independent. smartengine feature enables the engine on the client side (precisely, on the producer), whereas flate2 uses it to decompress SmartModule binary received from the Fluvio cluster. It can be used regardless of the producer.
sehz
left a comment
sehz
left a comment
There was a problem hiding this comment.
2nd pass with comments on SmartModule handling on Fluvio. Let's leverage existing API
crates/fluvio/src/smartmodule.rs
Outdated
|
|
||
| /// Represents a SmartModule loaded into Fluvio cluster | ||
| #[derive(Debug, Default)] | ||
| pub struct SmartModule { |
There was a problem hiding this comment.
Don't think we need to create another abstraction. Can we use SmartModuleMetadata or SmartModulePackage. It also has API to uncompress
There was a problem hiding this comment.
If you mean ones from fluvio-controlpanel-metadata, they don't have a binary wasm data there.
Also, they are a sort of contract between SC and the client or SPU. But here we are dealing with public API that will eventually leak to the user's code and other language clients as well, if we expose those abstractions, we will not be able to change them as frequently as we might want.
As an alternative, if we can just make fetch method return Vec<u8> of binary data only, and come up with new abstractions later, if we have doubts now.
There was a problem hiding this comment.
added fluvio-sc-schema::SmartModuleApiClient as API client for internal usage instead of exposing methods on Fluvio client
tjtelan
left a comment
tjtelan
left a comment
There was a problem hiding this comment.
Updates look good. It is nice to have the transformation config in a common place and to be able to test chaining with smdk
sehz
left a comment
sehz
left a comment
There was a problem hiding this comment.
LGTM. Excellent changes. Great consolidation of SmartModule chaining
|
bors r+ |
This PR changes: - ~~Introduced a new interface to Fluvio client to fetch SmartModules from a cluster (based on `FluvioAdmin`)~~ Introduced `fluvio-sc-schema::SmartModuleApiClient` that can be used from any components to interact with smartmodules in the cluster. - Added re-exports for the convenience of dealing with smartmodules to `fluvio` and `fluvio-smartengine` crates so it won't be needed to add other crates. - Moved `TransformationConfig` to a common place to be re-used from other parts (connectors, smdk, cli, etc.) - Added chaining support to `smdk test` tool.
|
Timed out. |
|
bors r+ |
This PR changes: - ~~Introduced a new interface to Fluvio client to fetch SmartModules from a cluster (based on `FluvioAdmin`)~~ Introduced `fluvio-sc-schema::SmartModuleApiClient` that can be used from any components to interact with smartmodules in the cluster. - Added re-exports for the convenience of dealing with smartmodules to `fluvio` and `fluvio-smartengine` crates so it won't be needed to add other crates. - Moved `TransformationConfig` to a common place to be re-used from other parts (connectors, smdk, cli, etc.) - Added chaining support to `smdk test` tool.
|
Pull request successfully merged into master. Build succeeded: |
bors
bot
changed the title
|
Very cool, is there documentation on how to use this functionality? |
Not yet, but it is planned. |
5 participants
This PR changes:
Introduced a new interface to Fluvio client to fetch SmartModules from a cluster (based onIntroducedFluvioAdmin)fluvio-sc-schema::SmartModuleApiClientthat can be used from any components to interact with smartmodules in the cluster.fluvioandfluvio-smartenginecrates so it won't be needed to add other crates.TransformationConfigto a common place to be re-used from other parts (connectors, smdk, cli, etc.)smdk testtool.