Remaining .NET 8 work for TensorPrimitives

[stephentoub]

RC2

• Add naive implementations of Add, AddMultiply, Cosh, Divide, Exp, Log, Multiply, MultiplyAdd, Negate, Subtract, Sinh, Tanh - @tannergooding
• Add naive implementations of CosineSimilarity, Distance, Dot, Norm, Sigmoid, SoftMax - @michaelgsharp
• Add naive implementations of Max/Min{Magnitude}, IndexOfMax/Min{Magnitude}, ConvertToHalf/Single, Log2 - @stephentoub
• Add vectorized implementations of Abs, Sum, SumOfSquares, SumOfMagnitudes, Product, ProductOfSums, ProductOfDifferences - @stephentoub
• Vectorize Add, Subtract, Multiply, Divide, Negate, AddMultiply, MultiplyAdd - @stephentoub
• Backport merged functionality to release/8.0-rc2 - @michaelgsharp
• Rename L2Normalize to Norm? - @stephentoub

GA

• Consider source rename of netcore.cs and netstandard.cs since they don't only apply to those target framework identifiers? per @stephentoub it maps to our existing naming conventions
• Ensure packages set up to ship XML docs correctly.
• Review/update XML comments for API docs - @stephentoub
• Update the API proposal issue to reflect the new APIs we added since that issue was approved - @stephentoub
• Add content to the RC2 release notes with call to action to try out preview package - @luisquintanilla
  • Collect benchmarks to include in the post - @michaelgsharp
• Make it safe across all operations to use an input span as a destination, as well, test, and document - @stephentoub
• Update CosineSimilarity implementations to vectorize remainder processing - @stephentoub
• Update Aggregate helper to vectorize remainder processing (Distance, Dot, Norm, Sum, SumOfSquares, SumOfMagnitudes, Product, ProductOfSums, ProductOfDifferences) - @stephentoub
• Vectorize Exp - @tannergooding
• Vectorize Softmax - @stephentoub
• Vectorize Sigmoid - @stephentoub
• Vectorize Log - @tannergooding
• Vectorize Log2 - @tannergooding
• Vectorize Cosh - @stephentoub
• Vectorize Sinh - @stephentoub
• Vectorize Tanh - @stephentoub
• Vectorize Max - @stephentoub
• Vectorize Max element-wise - @stephentoub
• Vectorize Min - @stephentoub
• Vectorize Min element-wise - @stephentoub
• Vectorize MaxMagnitude - @stephentoub
• Vectorize MaxMagnitude element-wise - @stephentoub
• Vectorize MinMagnitude - @stephentoub
• Vectorize MinMagnitude element-wise - @stephentoub
• Vectorize ConvertToHalf - @stephentoub
• Vectorize ConvertToSingle - @stephentoub
• Determine if we want to revert or backport FMA commit - @tannergooding (partial revert by @stephentoub)
• Decide whether to throw for unsupported span overlaps (today it's just undefined behavior) - @stephentoub
• Fix functional correctness issues found during ML.NET adoption testing - @tannergooding (issues were due to in-place operations which weren't supported in RC2 but were since supported by @stephentoub)
• Assess CpuMath APIs for methods and update the backlog below - @michaelgsharp
• Validate perf of APIs (e.g. vectorized speedups are reasonable, comparison of scalar against naive implementations make sense) - @michaelgsharp
• Vectorize IndexOfMax @michaelgsharp
• Vectorize IndexOfMin @michaelgsharp
• Vectorize IndexOfMaxMagnitude - @michaelgsharp
• Vectorize IndexOfMinMagnitude - @michaelgsharp
• Alignment review/guidance/updates to existing methods - @tannergooding
  • Rerun perf comparisons to ensure this addresses performance penalities for small sets
  • Unary methods
  • Binary and ternary methods
  • Aggregate and methods not using the shared helpers
  • Moved to post-GA: ConvertToHalf, ConvertToSingle, CosineSimilarity, IndexOfMin, IndexOfMax, IndexOfMinMagnitude, IndexOfMaxMagnitude
• Confirm required ordering semantics in IndexOfXx methods - @tannergooding - e.g.

  runtime/src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

  Line 602 in d054157

  else if (IsNegative(max) && !IsNegative(current))

• Backport final functionality to release/8.0
• Adopt in ML.NET - @michaelgsharp
  • Methods that use indices will remain in ML.NET but could go into the TensorPrimitives backlog below
  • Methods that require composition will remain in ML.NET (for now), e.g. calculate max magnitude and subtract the mean
  • Add
  • AddScale
  • AddScaleCopy
  • Scale
  • ScaleAdd (via composition, but could be improved) -- Added to backlog
  • MulElementWise
  • Sum
  • SumSq
  • SumAbs (via composition)
  • MaxAbs
  • MaxAbsDiff (via composition)
  • DotProductDense
  • L2DistSquared
  • Add (with indices) -- Added to backlog
  • AddScale (with indices) -- Added to backlog
  • DotProductSparse -- Added to backlog
  • MatrixTimesSource -- Added to backlog
  • SdcaL1UpdateDense -- Added to backlog
  • SdcaL1UpdateSparse -- Added to backlog
  • ZeroMatrixItems -- Added to backlog
• Adopt in SemanticKernel once RC2 nuget package is available - @stephentoub

[ghost]

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors
See info in area-owners.md if you want to be subscribed.

Issue Details

---

• Backport merged functionality to release/8.0-rc2
• Backport final functionality to release/8.0
• Review/update XML comments for API docs
• Update implementations to better factor in alignment
• Update CosineSimilarity implementations to vectorize remainder processing
• Update Aggregate helper to vectorize remainder processing (Distance, Dot, L2Normalize, Sum, SumOfSquares, SumOfMagnitudes, Product, ProductOfSums, ProductOfDifferences)
• Validate L2Normalize naming and either keep it or rename
• Confirm required ordering semantics in IndexOfXx methods, e.g.

  runtime/src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

  Line 602 in d054157

  else if (IsNegative(max) && !IsNegative(current))

• Vectorize Exp
• Vectorize Log
• Vectorize Cosh
• Vectorize Sinh
• Vectorize Tanh
• Vectorize Softmax
• Vectorize Sigmoid
• Vectorize Max
• Vectorize Min
• Vectorize MaxMagnitude
• Vectorize MinMagnitude
• Vectorize IndexOfMax
• Vectorize IndexOfMin
• Vectorize IndexOfMaxMagnitude
• Vectorize IndexOfMinMagnitude
• Vectorize ConvertToHalf?
• Vectorize ConvertToSingle?

Author:	stephentoub
Assignees:	tannergooding, michaelgsharp
Labels:	area-System.Numerics.Tensors
Milestone:	8.0.0

[luisquintanilla]

cc: @rgesteve

[bencyoung-Fignum]

Hi, I think this work is fantastic but are there any plans for a double precision version of all this?

[stephentoub]

Hi, I think this work is fantastic but are there any plans for a double precision version of all this?

The surface area that's there is what will ship in .NET 8. We intend to create generic versions for post-. NET 8.

[jeffhandley]

All of the TensorPrimitives work for dotnet/runtime in .NET 8 GA is completed. We will have a pull request into dotnet/machinelearning to bring in the adoption of these APIs. We also have #93286 for tracking additional effort in this space.