Complex containers

[WardBrian]

Closes #1116 . This branch has gotten to a really good place. I believe with the exception of stan-dev/math#2671 that the basic tests I've written here should compile and pass.

Thanks to previous PRs like #1102, #1115, #1106, most of these changes are pretty "boring", mostly just doing the obvious things based on the existing container types and the promotion semantics.

Things this PR adds:

1. complex_matrix, complex_vector, complex_row_vector types. The behavior and code generation of these largely mirrors the existing Eigen types.
2. Promotion from matrix, etc. to complex_matrix as necessary (e.g. in assignment, in function calls)
3. Basic math signatures (add, subtract, multiply, minus, elt_multiply, elt_divide) and basic container functions (e.g rows and the like)

Things this PR does not add:

1. Many, many other function signatures which would be valid or interesting for complex types. There are things like pow, which might work already, and things like FFTs, which will need math support. I leave these all for later PRs.
2. Add "mixed" signatures for things like multiply: (complex_vector, row_vector)=> complex_matrix. NB: this does currently work, but it does so by promoting the row_vector to complex_row_vector. We can later change this for speedups.

Submission Checklist

• Run unit tests
• Documentation
  • If a user-facing facing change was made, the documentation issue is: add complex scalar and vector types throughout docs docs#433

Release notes

Added complex containers: complex_matrix, complex_vector, and complex_row_vector with basic arithmetic support. More special functions and specializations will follow. Note that real-valued containers will be promoted to their complex variants as needed

Copyright and Licensing

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the BSD 3-clause license (https://opensource.org/licenses/BSD-3-Clause)

[WardBrian]

@SteveBronder the above commit adds the test we looked at on Friday, if you still have the chance to look at stan-dev/math#2671

[SteveBronder]

@WardBrian just put up the PR! Are you able to make a local cmdstan and try out stan-dev/math#2680?

[WardBrian]

@SteveBronder I ran this branch with the pr here: https://jenkins.flatironinstitute.org/blue/organizations/jenkins/Stan%2FStanc3/detail/PR-1133/7/pipeline/ and everything is green!

I would still like to see stan-dev/math#2676 get addressed whenever @rok-cesnovar has the chance before this would get merged, just to automatically test all the declared complex signatures.

[SteveBronder]

Will look at this tmrw!

[WardBrian]

@SteveBronder did you get a chance to look at this?

[SteveBronder]

Sorry its on my plate today

[SteveBronder]

Everything looks good to me! Though I think we need to do one thing in mem_pattern to check that any matrix or vector that is assigned to a complex matrix or vector is demoted to AoS. Besides that I'd like to try to cleanup the requires a bit on the functions so we don't have so many as I worry that could hurt compile time a good bit. But I can't do that then this PR looks good

[SteveBronder]

This all looks good! I think it just needs the thing in mem_pattern then it's good to merge

[WardBrian]

I'm not sure I understand what needs to change in mem_pattern to get the necessary behavior

[SteveBronder]

I'll take a look today. It's just that on the first pass we want to make sure that if a matrix/vector is being assigned to a complex matrix/vector then it is tagged as not possible to be an SoA matrix. So in query_initial_demotable_stmt we want to look for assignments whose meta type is ComplexMatrix/Vector and if the rhs expression is an Autodiffable Matrix/Vector then we want to add any named matrix/vector objects in the rhs expression to the list of matrices that cannot be promoted to SoA matrices. So we want to make sure the below on O1 will not have A_p be an SoA matrix.

parameters {
 matrix[10, 10] A_p;
}
transformed parameters {
 complex_matrix[10, 10] A_complex_tp = A_p;
}

https://github.com/stan-dev/stanc3/blob/master/src/analysis_and_optimization/Mem_pattern.ml#L357

[WardBrian]

I'd appreciate you taking a look, that part of the code is tough for me

[SteveBronder]

See bb816a5 I think that's all it needs so that it knows that is an object that cannot be promoted. Do you still need to add any new error messages specifically for complex types?

[WardBrian]

I believe all the error messages should be good to go

[WardBrian]

Looks like it needs a dune promote

[SteveBronder]

lgtm!

[WardBrian]

Thanks @SteveBronder - I'll hit merge on this as soon as 2.29.2 is out and good to go

[WardBrian]

Well, also after @bob-carpenter and I update the relevant docs