[API Proposal]: Expand Complex Number support

[elipriaulx]

Background and motivation

The current implementation of Complex in System.Numerics feels like a second-class citizen; it cannot be easily used in similar ways to other numeric types, and as it is backed by doubles it doesn't lend itself well to interop with Cuda, Intel IPP/MKL, or other math libraries for vector operations and performant processing.

An expanded support of complex numbers could make .Net a more viable alternative to C++, Python, R, and Matlab code to users from science and engineer domains.

API Proposal

namespace System.Numerics;

/*
    My primary objective is to use a float32-backed complex number type. 
    It's implied the existing double-backed Complex type would be expanded uniformly, too. 
    
    This example is based on my use-case with Net Framework 4.7.2.
    I feel it's not in the best interest of dotNet 7+ to adopt this verbatim;
    instead, an implementation based on generic math would be more
    appropriate. See 'Alternative Designs' section.
*/

[Serializable]
[StructLayout(LayoutKind.Sequential)]
public readonly struct Complex32 : 
    IEquatable<Complex32>,
	IFormattable
{
    private readonly float real;
    private readonly float imaginary;

    public float Real { get; }
    public float Imaginary { get; }

    public float Argument { get; }
    public float Magnitude { get; }
    
    public static readonly Complex32 Zero { get; }
    public static readonly Complex32 One { get; }
    public static readonly Complex32 ImaginaryOne { get; }
    public static readonly Complex32 PositiveInfinity { get; }
    public static readonly Complex32 NaN { get; }

    public Complex32(float real, float imaginary);
    
    public static Complex32 FromPolar(float magnitude, float argument);
    
    public bool IsZero();
    public bool IsOne();
    public bool IsImaginaryOne();
    public bool IsNaN();
    public bool IsInfinity();
    public bool IsReal();
    
    public static bool operator ==(Complex32 left, Complex32 right);
    public static bool operator !=(Complex32 left, Complex32 right);
    public static Complex32 operator +(Complex32 current);
    public static Complex32 operator -(Complex32 current);
    public static Complex32 operator +(Complex32 left, Complex32 right);
    public static Complex32 operator -(Complex32 left, Complex32 right);
    public static Complex32 operator +(Complex32 left, float right);
    public static Complex32 operator -(Complex32 left, float right);
    public static Complex32 operator +(float left, Complex32 right);
    public static Complex32 operator -(float left, Complex32 right);
    public static Complex32 operator *(Complex32 left, Complex32 right);
    public static Complex32 operator *(float left, Complex32 right);
    public static Complex32 operator *(Complex32 left, float right);
    public static Complex32 operator /(Complex32 left, Complex32 right);
    public static Complex32 operator /(float left, Complex32 right);
    public static Complex32 operator /(Complex32 left, float right);

    public Complex ToComplex() => new Complex(real, imaginary);

    // The current complex implementation formats as ordinals by default (RE, IM)
    // This should instead format as 'RE+IMi', or some variant.
    public override string ToString();
	public string ToString(string format, IFormatProvider formatProvider);

    public override int GetHashCode()

    public bool Equals(Complex32 other);
    
    public override bool Equals(object obj);
    
    // The current Complex implementation does not have a Parse method.
    // Parsing this is a little trickier than existing numerics.
    public static Complex32 Parse(string value);
    public static bool TryParse(string value, out Complex32 result);

    public static implicit operator Complex32(byte value);
    public static implicit operator Complex32(short value);
    public static implicit operator Complex32(sbyte value);
    public static implicit operator Complex32(ushort value);
    public static implicit operator Complex32(int value);
    public static implicit operator Complex32(BigInteger value);
    public static implicit operator Complex32(long value);
    public static implicit operator Complex32(uint value);
    public static implicit operator Complex32(ulong value);
    public static implicit operator Complex32(float value);
    public static explicit operator Complex32(double value);
    public static explicit operator Complex32(decimal value);
    public static explicit operator Complex32(Complex value);
    
    // There is also no support in the Math.X set of operators...
    // ...it would make sense to add support for operations there, if not:

    public Complex32 Exponential();
    public Complex32 NaturalLogarithm();
    public Complex32 Logarithm(float logBase);
    public Complex32 SquareRoot();
    public Complex32 Power(Complex32 exponent);
    
    public Complex32 Conjugate();
    public Complex32 Reciprocal();
    
    public static Complex32 Negate(Complex32 value);
    public static Complex32 Conjugate(Complex32 value);
    public static Complex32 Add(Complex32 left, Complex32 right);
    public static Complex32 Subtract(Complex32 left, Complex32 right);
    public static Complex32 Multiply(Complex32 left, Complex32 right);
    public static Complex32 Divide(Complex32 dividend, Complex32 divisor);
    public static Complex32 Reciprocal(Complex32 value);
    public static Complex32 Sqrt(Complex32 value);
    public static float Abs(Complex32 value);
    public static Complex32 Exp(Complex32 value);
    public static Complex32 Pow(Complex32 value, Complex32 power);
    public static Complex32 Pow(Complex32 value, float power);
    public static Complex32 Ln(Complex32 value);
    public static Complex32 Log(Complex32 value, float baseValue);
}

API Usage

// Simplify complex math operations.
var a = new Complex32(2.0f, 3.0f); // 2+3i
var b = new Complex32(5.0f, 2.0f); // 5+2i

// Division is otherwise complicated to implement...
var c = a/b; // eg. (2+3i)/(5+2i) => ...use conjugate... => 16/29 + 11/29 i

// Parsing is otherwise difficult for a user to implement, especially with localisation:
var s1 = "2+3i"; 
var c1 = Complex32.Parse(s1); // ...instead of tokenising and parsing elements manually - especially when accounting for localisation!

// Ideally this wouldn't be in ordinal form.
// When in ordinal form it's not obvious that this even is a complex number...
var s2 = c1.Conjugate().ToString(); // 2-3i

// However I predominantly just want a native type I can debug/develop with, but also hand off to other libraries/tools without difficult conversions.
// Adherence to an 8 byte (2x float, conventionally Real then Imaginary) structure ensures compatibility with types across most math/computational libraries, without adopting any specific one as a first party dependency. 
var c2Array = new Complex32[100];
var c3Array = new Complex32[100];
...
IppLibraryImplementation.SomeFunction(c2Array, c3Array);
CudaLibraryImplementation.SomeFunction(c2Array, c3Array);

Alternative Designs

I am predominantly using Net Framework (🥲), and unfamiliar with the current state of Generic Math in dotNet; I feel a better solution would be to instead implement a generic Complex<T> that is treated with similar importance as float/integer types.

namespace System.Numerics;

[Serializable]
[StructLayout(LayoutKind.Sequential)]
public readonly struct Complex<T> : 
    ...
	where T : INumber<T> // <- How do we address this in Generic Math? 😭
{
    private readonly T real;
    private readonly T imaginary;

    public T Real { get; }
    public T Imaginary { get; }

    ...
}

Risks

• Existing code expecting an ordinal from unformatted Complex.ToString calls might need modification. Implementation via Complex avoids this.
• The type name Complex32 does not feel to align with the current conventions of .Net. I'm not sure what a more appropriate alternative might be in the context of .Net Framework, and dotnet < 7.0.
• Adding the type Complex32 feels to not align with the direction of dotnet 7+. A generic Complex<T> implementation seems more appropriate and less opinionated.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

The current implementation of Complex in System.Numerics feels like a second-class citizen; it cannot be easily used in similar ways to other numeric types, and as it is backed by doubles it doesn't lend itself well to interop with Cuda, Intel IPP/MKL, or other math libraries for vector operations and performant processing.

An expanded support of complex numbers could make .Net a more viable alternative to C++, Python, R, and Matlab code to users from science and engineer domains.

API Proposal

namespace System.Numerics;

/*
    My primary objective is to use a float32-backed complex number type. 
    It's implied the existing double-backed Complex type would be expanded uniformly, too. 
    
    This example is based on my use-case with Net Framework 4.7.2.
    I feel it's not in the best interest of dotNet 7+ to adopt this verbatim;
    instead, an implementation based on generic math would be more
    appropriate. See 'Alternative Designs' section.
*/

[Serializable]
[StructLayout(LayoutKind.Sequential)]
public readonly struct Complex32 : 
    IEquatable<Complex32>,
	IFormattable
{
    private readonly float real;
    private readonly float imaginary;

    public float Real { get; }
    public float Imaginary { get; }

    public float Argument { get; }
    public float Magnitude { get; }
    
    public static readonly Complex32 Zero { get; }
    public static readonly Complex32 One { get; }
    public static readonly Complex32 ImaginaryOne { get; }
    public static readonly Complex32 PositiveInfinity { get; }
    public static readonly Complex32 NaN { get; }

    public Complex32(float real, float imaginary);
    
    public static Complex32 FromPolar(float magnitude, float argument);
    
    public bool IsZero();
    public bool IsOne();
    public bool IsImaginaryOne();
    public bool IsNaN();
    public bool IsInfinity();
    public bool IsReal();
    public bool IsRealNonNegative();
    
    public static bool operator ==(Complex32 left, Complex32 right);
    public static bool operator !=(Complex32 left, Complex32 right);
    public static Complex32 operator +(Complex32 current);
    public static Complex32 operator -(Complex32 current);
    public static Complex32 operator +(Complex32 left, Complex32 right);
    public static Complex32 operator -(Complex32 left, Complex32 right);
    public static Complex32 operator +(Complex32 left, float right);
    public static Complex32 operator -(Complex32 left, float right);
    public static Complex32 operator +(float left, Complex32 right);
    public static Complex32 operator -(float left, Complex32 right);
    public static Complex32 operator *(Complex32 left, Complex32 right);
    public static Complex32 operator *(float left, Complex32 right);
    public static Complex32 operator *(Complex32 left, float right);
    public static Complex32 operator /(Complex32 left, Complex32 right);
    public static Complex32 operator /(float left, Complex32 right);
    public static Complex32 operator /(Complex32 left, float right);

    public Complex ToComplex() => new Complex(real, imaginary);

    // The current complex implementation formats as ordinals by default (RE, IM)
    // This should instead format as 'RE+IMi', or some variant.
    public override string ToString();
	public string ToString(string format, IFormatProvider formatProvider);

    public override int GetHashCode()

    public bool Equals(Complex32 other);
    
    public override bool Equals(object obj);
    
    // The current Complex implementation does not have a Parse method.
    // Parsing this is a little trickier than existing numerics.
    public static Complex32 Parse(string value);
    public static bool TryParse(string value, out Complex32 result);

    public static implicit operator Complex32(byte value);
    public static implicit operator Complex32(short value);
    public static implicit operator Complex32(sbyte value);
    public static implicit operator Complex32(ushort value);
    public static implicit operator Complex32(int value);
    public static implicit operator Complex32(BigInteger value);
    public static implicit operator Complex32(long value);
    public static implicit operator Complex32(uint value);
    public static implicit operator Complex32(ulong value);
    public static implicit operator Complex32(float value);
    public static explicit operator Complex32(double value);
    public static explicit operator Complex32(decimal value);
    public static explicit operator Complex32(Complex value);
    
    // There is also no support in the Math.X set of operators...
    // ...it would make sense to add support for operations there, if not:

    public Complex32 Exponential();
    public Complex32 NaturalLogarithm();
    public Complex32 Logarithm(float logBase);
    public Complex32 SquareRoot();
    public Complex32 Power(Complex32 exponent);
    
    public Complex32 Conjugate();
    public Complex32 Reciprocal();
    
    public static Complex32 Negate(Complex32 value);
    public static Complex32 Conjugate(Complex32 value);
    public static Complex32 Add(Complex32 left, Complex32 right);
    public static Complex32 Subtract(Complex32 left, Complex32 right);
    public static Complex32 Multiply(Complex32 left, Complex32 right);
    public static Complex32 Divide(Complex32 dividend, Complex32 divisor);
    public static Complex32 Reciprocal(Complex32 value);
    public static Complex32 Sqrt(Complex32 value);
    public static float Abs(Complex32 value);
    public static Complex32 Exp(Complex32 value);
    public static Complex32 Pow(Complex32 value, Complex32 power);
    public static Complex32 Pow(Complex32 value, float power);
    public static Complex32 Ln(Complex32 value);
    public static Complex32 Log(Complex32 value, float baseValue);
}

API Usage

// Simplify complex math operations.
var a = new Complex32(2.0f, 3.0f); // 2+3i
var b = new Complex32(5.0f, 2.0f); // 5+2i

// Division is otherwise complicated to implement...
var c = a/b; // eg. (2+3i)/(5+2i) => ...use conjugate... => 16/29 + 11/29 i

// Parsing is otherwise difficult for a user to implement, especially with localisation:
var s1 = "2+3i"; 
var c1 = Complex32.Parse(s1); // ...instead of tokenising and parsing elements manually - especially when accounting for localisation!

// Ideally this wouldn't be in ordinal form.
// When in ordinal form it's not obvious that this even is a complex number...
var s2 = c1.Conjugate().ToString(); // 2-3i

// However I predominantly just want a native type I can debug/develop with, but also hand off to other libraries/tools without difficult conversions.
// Adherence to an 8 byte (2x float, conventionally Real then Imaginary) structure ensures compatibility with types across most math/computational libraries, without adopting any specific one as a first party dependency. 
var c2Array = new Complex32[100];
var c3Array = new Complex32[100];
...
IppLibraryImplementation.SomeFunction(c2Array, c3Array);
CudaLibraryImplementation.SomeFunction(c2Array, c3Array);

Alternative Designs

I am predominantly using Net Framework (🥲), and unfamiliar with the current state of Generic Math in dotNet; I feel a better solution would be to instead implement a generic Complex<T> that is treated with similar importance as float/integer types.

namespace System.Numerics;

[Serializable]
[StructLayout(LayoutKind.Sequential)]
public readonly struct Complex<T> : 
    ...
	where T : INumber<T> // <- How do we address this in Generic Math? 😭
{
    private readonly T real;
    private readonly T imaginary;

    public T Real { get; }
    public T Imaginary { get; }

    ...
}

Risks

• Existing code expecting an ordinal from unformatted Complex.ToString calls might need modification. Implementation via Complex avoids this.
• The type name Complex32 does not feel to align with the current conventions of .Net. I'm not sure what a more appropriate alternative might be in the context of .Net Framework, and dotnet < 7.0.
• Adding the type Complex32 feels to not align with the direction of dotnet 7+. A generic Complex<T> implementation seems more appropriate and less opinionated.

Author:	elipriaulx
Assignees:	-
Labels:	api-suggestion, area-System.Numerics
Milestone:	-

[dakersnar]

and as it is backed by doubles it doesn't lend itself well to interop with Cuda, Intel IPP/MKL, or other math libraries for vector operations and performant processing.

A generic Complex<T> implementation seems more appropriate and less opinionated.

To clarify, would a float32-backed complex value be enough to interop with all of these other frameworks? If so, I think Complex32 would be reasonable to add on it's own. If more flexibility is needed, I could see the argument for Complex<T>.

That aside, I can briefly speak on what Generic Math integration might look like, and @tannergooding can weigh in on this as well. From a Generic Math perspective, it would be less important to create a generic complex type such as Complex<T>, and more important to allow the new type Complex32 (or whatever type we introduce) to work with generic algorithms by having it inherit the interfaces that Complex currently does. Given that, the type might look something like this:

public readonly struct Complex32
    : IEquatable<Complex32>,
      IFormattable,
      INumberBase<Complex32>,
      ISignedNumber<Complex32>
{
}

It could also be beneficial to "hoist" the parts of both Complex and Complex32 that are shared up to a new interface, and then have those types inherit from that interface.

public interface IComplex<TSelf>
    : INumberBase<TSelf>,
      ISignedNumber<TSelf>
    where TSelf : IComplex<TSelf>
{
  // This interface would include methods shared by all Complex numbers that aren't already exposed by the above interfaces

}
    

public readonly struct Complex
: IEquatable<Complex>,
  IFormattable,
  IComplex<Complex>
{

}

public readonly struct Complex32
: IEquatable<Complex32>,
  IFormattable,
  IComplex<Complex32>
{

}

This would allow you to write a single generic algorithm that could operate on both Complex and Complex32.

[tannergooding]

To clarify, would a float32-backed complex value be enough to interop with all of these other frameworks?

Yesn't. float/double are the primary, but not the only and with Half, BFloat16, and other "small types" becoming more prevalent in these spaces the others will become more prevalent.

From a Generic Math perspective, it would be less important to create a generic complex type such as Complex, and more important to allow the new type Complex32 (or whatever type we introduce) to work with generic algorithms by having it inherit the interfaces that Complex currently does.

I think there is importance on both sides. The type exposed needs to support the relevant interfaces so devs can use them generically, but having a singular Complex<T> can also avoid type explosion and ensure that we don't need to consider Complex16 or Complex128 or other types in the future.

[elipriaulx]

@dakersnar I think the significance of float32 is that GPUs typically don't have equal capacity for double precision operations. With an up-market consumer GPU, I can write a kernel in CUDA using floats and work with 38.71 TFLOPS, or doubles at only 1.21 TFLOPS. Float16s in this context might be appealing from a memory perspective, but typically performance is uniform to float32s on most hardware I have seen. This isn't going to vary 'library to library'.

My concerns aren't really about just any third party math library; libraries like Math.NET have some great ideas, but ultimately it's not going to improve the products I work on in the same way as being able to access vector intrinsics though Intel IPP/MKL or BLAS will, and all of these currently have their place, with large and opinionated APIs. I'd love to be able to access similar performance as Numpy in Python through C#/.Net natively, but the surface area and performance of these APIs in .Net doesn't seem to be there yet (and probably will never be there in .Net Framework at least).

---

'Type explosion' would be one of my primary concerns (it kind of feels like power creep in a game 😅).

This could be my ignorance from not having actually used any generic math features yet - I feel there is merit in not just being able to write an algorithm of MyAlgoirthm<TNumeric>(Complex<TNumeric> ...) , but where MyAlgorithm<TNumeric>(TNumeric ...) could have TNumeric specified AS Complex<T> (MyAlgorithm<Complex<xxx>>(Complex<xxx> ...)), OR any other Real Number type (float, or double, or single... MyAlgorithm<float>(float ...)) - I am not sure if this even makes sense though, yet, beyond simplistic contrived examples.

Realistically before I can contribute any value to this discussion I need to invest some time in becoming comfortable with what is already available in dotNet 7, and a sense of how complex number support could add value to what is already there.

[tannergooding]

This could be my ignorance from not having actually used any generic math features yet - I feel there is merit in not just being able to write an algorithm of MyAlgoirthm(Complex ...) , but where MyAlgorithm(TNumeric ...) could have TNumeric specified AS Complex (MyAlgorithm<Complex>(Complex ...)), OR any other Real Number type (float, or double, or single... MyAlgorithm(float ...)) - I am not sure if this even makes sense though, yet, beyond simplistic contrived examples.

Basically Complex<T> where T : IFloatingPointIeee754<T> would allow a few different types of algorithms:

1. Direct usage of a type: void MyAlgorithm(Complex<float> value)
2. Generic over complex: void MyAlgorithm<T>(Complex<T> value) where T : IFloatingPointIeee754<T>
3. Truly generic: void MyAlgorithm<T>(T value) where T : INumberBase<T>

The first means you can use the type the same as Complex32 or Complex, you just use Complex<float> and Complex<double> instead.

The second means you can write one algorithm that works with any Complex<T> (whether it is float or double or half or another T as the underlying type).

The third means you can write one algorithm that supports any number type, whether complex or real, but doesn't necessarily allow you to "specialize" based on what tyhe T is.

There is then a few variations on the third. For example if we defined some IComplexNumber<T, U> where T : IComplexNumber<T, U> and then had Complex<T> : IComplexNumber<Complex<T>, T> you could define void MyAlgorithm<T, U>(T value) where T : IComplexNumber<T, U>. Such an algorithm would then work with anything that implements IComplexNumber<T, U> so it would work with both Complex<double> and Complex for example.

We don't have anything like Higher Kinded Types which would allow you to say that T must itself be generic and take a generic parameter.

[rickbrew]

This would definitely have use in graphics and image processing. In Paint.NET 5.0, there's a new Bokeh blur effect I added with collaboration from @Sergio0694 and @mikepound. It uses System.Numerics.Complex on the CPU to compute some convolution kernel stuff, but there's an opportunity to more easily funnel data in and out of the GPU if we have a float version of the struct.

We'd need to add some logic to ComputeSharp so it can auto-marshal Complex<float> as a float2, but that's straightforward and has precedent (e.g. Vector4 is already marshaled as float4, IIRC).

[ruilvo]

This absolutes interests me. I'm writing some DSP code and a lack of a Complex<float> or a ComplexF or something like that would absolutely be important to me.

Also, if F# is being pushed as a data-science, maybe numerics, and a performance-oriented language, the type is a must-have...