Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - refactor: simplified control plane api #3073

Closed
wants to merge 51 commits into from

Conversation

sehz
Copy link
Contributor

@sehz sehz commented Mar 12, 2023

This PR implements a more generalized admin API.

The existing Admin API uses a macro to map the admin type to a global object type. While this works, it requires maintaining a mapping relationship every time a new admin type is added. In addition, mapping for each action (create, delete, watch) has a slightly different kind of encoding, which adds to the complexity. The underlying complexity is that with the current Rust typing system, converting Traits with associated properties to trait objects is not allowed. Hence use of macro to map handle encoding and decoding.

With this PR, type mapping is lazily done. Borrowing from Arc and Anyhow approaches, downcasting/upcasting is delayed much as possible. This preserves type-specific Admin type calls, but extending to any arbitrary admin object types is possible.

The core of the new approach is this trait:

pub trait TryEncodableFrom<T>: Sized + Encoder + Decoder {
        fn try_encode_from(value: T, version: Version) -> Result<Self>;

        fn downcast(&self) -> Result<Option<T>>;
    }

Using fluvio's encoder/decoder, any arbitrary can be mapped into the binary blog and vice versa. Once received from the network, the encoded type can be downcast to a specific type as necessary.

The changes are fully backward compatible. The existing macro is used when talking to the previous version of Fluvio. The backward compatibility codes (classic) will be removed in the future.

@sehz sehz marked this pull request as draft March 12, 2023 01:58
@sehz sehz self-assigned this Mar 13, 2023
@sehz sehz force-pushed the controlplane_v2 branch 7 times, most recently from f6db42e to c05d69d Compare April 28, 2023 20:50
@sehz sehz marked this pull request as ready for review April 29, 2023 17:26
@sehz sehz added this to the 0.10.9 milestone Apr 29, 2023
@sehz
Copy link
Contributor Author

sehz commented May 4, 2023

bors r+

bors bot pushed a commit that referenced this pull request May 4, 2023
This PR implements a more generalized admin API.

The existing Admin API uses a macro to map the admin type to a global object type.  While this works, it requires maintaining a mapping relationship every time a new admin type is added.   In addition, mapping for each action (create, delete, watch) has a slightly different kind of encoding, which adds to the complexity.   The underlying complexity is that with the current Rust typing system, converting Traits with associated properties to trait objects is not allowed.  Hence use of macro to map handle encoding and decoding.

With this PR, type mapping is lazily done.  Borrowing from `Arc` and `Anyhow` approaches, downcasting/upcasting is delayed much as possible.  This preserves type-specific Admin type calls, but extending to any arbitrary admin object types is possible.

The core of the new approach is this trait:
```
pub trait TryEncodableFrom<T>: Sized + Encoder + Decoder {
        fn try_encode_from(value: T, version: Version) -> Result<Self>;

        fn downcast(&self) -> Result<Option<T>>;
    }
```

Using fluvio's encoder/decoder, any arbitrary can be mapped into the binary blog and vice versa.   Once received from the network, the encoded type can be downcast to a specific type as necessary.

The changes are fully backward compatible.  The existing macro is used when talking to the previous version of Fluvio.   The backward compatibility codes (classic) will be removed in the future.
@bors
Copy link

bors bot commented May 4, 2023

Canceled.

@sehz
Copy link
Contributor Author

sehz commented May 4, 2023

bors r+

bors bot pushed a commit that referenced this pull request May 4, 2023
This PR implements a more generalized admin API.

The existing Admin API uses a macro to map the admin type to a global object type.  While this works, it requires maintaining a mapping relationship every time a new admin type is added.   In addition, mapping for each action (create, delete, watch) has a slightly different kind of encoding, which adds to the complexity.   The underlying complexity is that with the current Rust typing system, converting Traits with associated properties to trait objects is not allowed.  Hence use of macro to map handle encoding and decoding.

With this PR, type mapping is lazily done.  Borrowing from `Arc` and `Anyhow` approaches, downcasting/upcasting is delayed much as possible.  This preserves type-specific Admin type calls, but extending to any arbitrary admin object types is possible.

The core of the new approach is this trait:
```
pub trait TryEncodableFrom<T>: Sized + Encoder + Decoder {
        fn try_encode_from(value: T, version: Version) -> Result<Self>;

        fn downcast(&self) -> Result<Option<T>>;
    }
```

Using fluvio's encoder/decoder, any arbitrary can be mapped into the binary blog and vice versa.   Once received from the network, the encoded type can be downcast to a specific type as necessary.

The changes are fully backward compatible.  The existing macro is used when talking to the previous version of Fluvio.   The backward compatibility codes (classic) will be removed in the future.
@bors
Copy link

bors bot commented May 4, 2023

Build failed:

@sehz
Copy link
Contributor Author

sehz commented May 4, 2023

bors r+

bors bot pushed a commit that referenced this pull request May 4, 2023
This PR implements a more generalized admin API.

The existing Admin API uses a macro to map the admin type to a global object type.  While this works, it requires maintaining a mapping relationship every time a new admin type is added.   In addition, mapping for each action (create, delete, watch) has a slightly different kind of encoding, which adds to the complexity.   The underlying complexity is that with the current Rust typing system, converting Traits with associated properties to trait objects is not allowed.  Hence use of macro to map handle encoding and decoding.

With this PR, type mapping is lazily done.  Borrowing from `Arc` and `Anyhow` approaches, downcasting/upcasting is delayed much as possible.  This preserves type-specific Admin type calls, but extending to any arbitrary admin object types is possible.

The core of the new approach is this trait:
```
pub trait TryEncodableFrom<T>: Sized + Encoder + Decoder {
        fn try_encode_from(value: T, version: Version) -> Result<Self>;

        fn downcast(&self) -> Result<Option<T>>;
    }
```

Using fluvio's encoder/decoder, any arbitrary can be mapped into the binary blog and vice versa.   Once received from the network, the encoded type can be downcast to a specific type as necessary.

The changes are fully backward compatible.  The existing macro is used when talking to the previous version of Fluvio.   The backward compatibility codes (classic) will be removed in the future.
@bors
Copy link

bors bot commented May 4, 2023

Pull request successfully merged into master.

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

@bors bors bot changed the title refactor: simplified control plane api [Merged by Bors] - refactor: simplified control plane api May 4, 2023
@bors bors bot closed this May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant