Skip to content

Conversation

@kaixuanliu
Copy link

No description provided.

Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
@danieldk
Copy link
Member

Adding function support is really a great idea and nice work on the PR! Some changes I think would be useful:

  • Have the decorator on the function and not the layer that uses it, since we want to declare that the function can be extended and this also allows re-use.
  • Related to the previous point: avoid having to match the name of the function in the mapping to the member variable. The member variable name is an implementation detail, whereas kernel mapping name is a global. E.g., a layer might call the activation function self.activation, but doing @use_kernel_func_from_hub("activation") would pollute the global namespace with a non-descript activation (which might conflict if multiple modules use the same name), whereas @use_kernel_func_from_hub("silu_and_mul") is not ambiguous.
  • Functions handled by the same code path as modules (not requiring another iteration over the modules).
  • Being able to use Hub layers for functions (if the signatures match modulo self).
  • Being able to use Hub functions for layers (if the signatures match modulo self).

I have made a proposal that I think should work for your transformers PR, but also implements the properties above: #188

@kaixuanliu
Copy link
Author

@danieldk really creative idea for your proposal, great work! Will close this PR once 188 is merged. Thx again!

@danieldk
Copy link
Member

Merged! Let me know if you run into any issues using this in your transformers PR.

@kaixuanliu kaixuanliu closed this Nov 27, 2025
@kaixuanliu kaixuanliu deleted the tmp-kernels-rotary branch November 27, 2025 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants