Repository navigation
Replies: 1 comment
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Feature description
I'd like to propose adding a compile-time kernel customization mechanism to
burn-flexthat allows external crates to replace selected built-in operations with optimized implementations, while retaining the existing Flex implementations as defaults.This is a follow-up to #5976, where the maintainers suggested that transparent kernel substitution would be more appropriate in Flex following the removal of
burn-ndarray.I've now implemented and tested a prototype against Burn 0.22.0.
Motivation
I'm working on optimizing Burn inference for CPUs with architecture-specific kernels, initially targeting the SpaceMiT X60 RISC-V processor.
My goal is to keep model code completely architecture-independent. Ideally, the same model should work with the standard Flex backend or a hardware-optimized variant, without changing the model implementation.
The existing
#[backend_extension(...)]mechanism supports adding custom operations, but my use case requires transparently replacing existing operations such as Conv1d.I'd also like to keep all hardware-specific implementations in external crates, without introducing architecture-specific dependencies into
burn-flex.Proposed approach
The prototype introduces a generic kernel-set parameter to the existing Flex backend:
pub struct Flex<K: FlexKernelSet = DefaultK> {
// ...
}
A public
FlexKernelSettrait provides the extension points. For example:pub trait FlexKernelSet:
Clone + Copy + Debug + Default + Send + Sync + 'static
{
fn conv1d(
x: FlexTensor,
weight: FlexTensor,
bias: Option,
options: ConvOptions<1>,
) -> FlexTensor {
default_conv1d::(x, weight, bias, options)
}
}
An external crate can implement this trait and define its own backend type:
pub type X60Flex = Flex;
The existing
ModuleOpsimplementation delegates toK::conv1d(...), allowing the selected kernel set to override the operation.The original Conv1d implementation remains available through
default_conv1d::<K>(), so external kernel sets can inherit the existing behavior for operations they don't override.The kernel set is selected at compile time, without runtime registries, trait objects, or function pointers.
Current status
I have a working prototype and a separate external test crate.
The tests verify:
The existing relevant tests also pass.
These tests use ordinary Rust implementations rather than hardware-specific X60 kernels, so the extension mechanism itself is architecture-independent.
Upstream considerations
The implementation currently requires genericizing parts of
burn-flexto propagate the kernel-set type through its backend trait implementations.I'm reviewing the patch to minimize unnecessary changes and preserve existing source compatibility wherever possible.
The intention is to keep the existing default Flex behavior unchanged and make kernel customization entirely opt-in.
Feedback requested
Before preparing a pull request, I'd appreciate feedback on whether this approach fits the intended direction of
burn-flex.In particular:
Flex<K>with a publicFlexKernelSettrait be an acceptable extension mechanism?I have a working patch and external integration tests, and I'm happy to refine the implementation to fit the project's conventions.
Thanks!
PS. I am still very new to both Rust and Burn but I've been having a lot of fun learning both. Fantastic library. I have been programming in various languages since the 1980's and working with this stuff is the most fun I've had in a long time
All reactions