improving type annotations, enhancing documentation, and refining activation function implementations.#4
Merged
DanielAvdar merged 13 commits intomainfrom Apr 17, 2025
Merged
Conversation
Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
…ntity activations Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
…nced reference formatting and clearer mathematical explanations Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
Replaced `min_val` and `max_val` with `x_min` and `x_max` for parameter naming consistency. Adjusted mathematical notation, description, and example code to reflect this change. This improves clarity and aligns with the updated API. Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
Updated the parameter names in the SoftClipping class and its usage across tests from `min_val` and `max_val` to `x_min` and `x_max` for improved clarity and consistency. Adjusted all references to reflect this change. Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
This commit introduces a randomized property-based test for the BentIdentity function using the Hypothesis library. The test ensures the function produces correct outputs by comparing its results to the expected formula. Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
Modified the SoftClipping activation to accept a user-defined clipping function, with the default set to sigmoid. Updated the documentation with an example using a custom function, such as hyperbolic tangent, enhancing flexibility for different use cases. Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
Clarify that the soft clipping activation can use various differentiable functions, such as sigmoid or tanh, instead of being limited to sigmoid. This improves flexibility and better describes the functionality. Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
…criptions Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
…ility functions Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
This commit adds type annotations to function signatures for improved readability and type safety. Modules affected include various activation functions such as Elish, Sparsemax, Entmax, Bent Identity, Soft Clipping, and Maxout. Additionally, minor imports and utility function type hints were adjusted for consistency. Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
Updated function signatures and added type hints across multiple files to enhance type safety. Removed unnecessary methods in `Sparsemax` and associated tests. Minor adjustments made to ensure consistency in type usage and MyPy compatibility. Signed-off-by: DanielAvdar <66269169+DanielAvdar@users.noreply.github.com>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #4 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 18 18
Lines 173 179 +6
=========================================
+ Hits 173 179 +6 ☔ View full report in Codecov by Sentry. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several updates to the
activations_pluslibrary, focusing on improving type annotations, enhancing documentation, and refining activation function implementations. Key changes include the addition of type hints across multiple activation functions, expanded mathematical explanations in the documentation, and increased flexibility for certain activation functions.Type Annotations and Code Refinements:
forwardand__init__methods of various activation functions, includingBentIdentity,ELiSH,Entmax,Maxout,SoftClipping,Sparsemax, andSReLU, ensuring better type safety and readability. ([[1]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-e3ddaa08f41695a0f82b7b6e4717f14a844105b2098c9ce03744308fbfd90a96L18-R18),[[2]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-bf433f930a110bdbbd5e11ef54aa31b12463aa966f21cc442a089f259db8c337L12-R12),[[3]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-0ea86e15dda43b38f95c1880fb75e45effebf8c1ecb502aa83885f857f10aa34L20-R21),[[4]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-facc4429febf77800b1412bb63db3a2e80657a4de30fc47a23df769a952979feL12-R16),[[5]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-860f228cb602167d39f8c201bb2c5c1c210e9d347478f4670701833f48bcb013L11-R51),[[6]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-7df0cf96685c313078cfd6aa2557c45ff5a855d5c8db0a451b6156ac6a73ae3dL24-R25),[[7]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-279bc7ae48f2f26be006d1c01f28bbefe592e2a92f5e78421345680f611cffa9L17-R25))Anytype forctxparameters in autograd functions and added type hints to theforwardandbackwardmethods inentmaxandsparsemaximplementations. ([[1]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-3596902ca767db84900b12a27d3739b415e9042f29722fa38db5732bbdb8e2f4L12-R14),[[2]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-3596902ca767db84900b12a27d3739b415e9042f29722fa38db5732bbdb8e2f4L25-R27),[[3]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-c450ca7293aad8c478cb1475dc7ece13c26f910dd969fa5a1b05b840d279042aL53-R55),[[4]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-da9b407e31772ef6a4699e47fb264e17dab93b17777ac4d5e73ea9b95035a2cbL24-R26))Documentation Enhancements:
Bent Identity,ELiSH,Entmax,Maxout,Soft Clipping, andSparsemax) with detailed mathematical explanations and example usages. This provides users with better insights into the behavior and applications of these functions. ([[1]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-68ecbc1ed851408c33cda22f03c7e51ed8757ed50f9a56fada6741132dc52913R10-R18),[[2]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-52ff047a648515e54f1aadb7d57a696bbaebbe90402bab074b04dc7a543c6facR12-R21),[[3]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-e40975234f03b2054ea9c958399b45a8b3f855d83a7c4e15e859b59c83c3b476R12-R20),[[4]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-b85f734fac5b06746b5dabc2671861f10eedf41c3acf5e2959d73f22087f1213R12-R20),[[5]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-42b47b3d1d9fb980bb01cd4ed3de5c648624a9a9e9df7ab10f5dff74c61715dbR7-R73),[[6]](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-879319da98667d9bceb2856f95294e52d53f4b5f1401947063befe11203b908eR10-R20))SoftClippingactivation function, showcasing how to use alternative clipping functions liketanh. ([docs/source/activations/soft_clipping.rstR7-R73](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-42b47b3d1d9fb980bb01cd4ed3de5c648624a9a9e9df7ab10f5dff74c61715dbR7-R73))Functional Updates:
SoftClippingactivation function by introducing a customizable clipping function (clip_func) and renamed parameters for clarity (x_minandx_maxinstead ofmin_valandmax_val). ([activations_plus/soft_clipping/soft_clipping_func.pyL11-R51](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-860f228cb602167d39f8c201bb2c5c1c210e9d347478f4670701833f48bcb013L11-R51))HardSwishactivation function from theREADME.mdto reflect its deprecation or removal. ([README.mdL23](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L23))Minor Fixes:
Makefilefor themypycommand. ([MakefileL28-R28](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52L28-R28))mypycheck in the GitHub Actions workflow. ([.github/workflows/code-checks.ymlL19-R19](https://github.com/DanielAvdar/activations-plus/pull/4/files#diff-ddf88e15b08104435ae66be9982938335f6c290a85de4cb9a09868e0e01dd4d4L19-R19))