P1673 A free function linear algebra interface based on the BLAS

[wg21bot]

P1673R0 A free function linear algebra interface based on the BLAS (Mark Hoemmen, Daisy Hollman, Christian Trott, Daniel Sunderland, Nevin Liber, Siva Rajamanickam, Li-Ta Lo, Graham Lopez, Peter Caday, Sarah Knepper, Piotr Luszczek, Timothy Costa)

[tituswinters]

This R0 needs to be forwarded by some/all of the relevant SGs before going to LEWG for design review. Removing the LEWG label.

[brycelelbach]

Mark, if you want some early design review, we can look at this in LEWGI.

[brycelelbach]

Cologne 2019-07 LEWGI Minutes

P1673R0 & P1674R0 C++ BLAS Library: Design Feedback

Champion: Christian Trott

Minute Taker: Tobias Loew

Start Overview: 07-16 8:52

Start Review: 9:20

Algorithms are parameterized on a concept, iterators. The proposed linear algebra algorithms are parameterized on a concrete view type, mdspan.

Do people need to spell scaled_scalar? Can it just be an implementation-defined type with certain properties.

SG1 needs to review this from a parallelization/vectorization angle.

How should we handle outputs in the API?

Explore a range-style lazy view API.

End: 10:17

CONSENSUS: Bring a revision of P1673R0, exploring conceptification of mdspan and a range-style lazy API, to LEWGI for further design feedback.

[wg21bot]

P1673R1 A free function linear algebra interface based on the BLAS (Mark Hoemmen, Daisy Hollman, Christian Trott, Daniel Sunderland, Nevin Liber, Siva Rajamanickam, Li-Ta Lo, Graham Lopez, Peter Caday, Sarah Knepper, Piotr Luszczek, Timothy Costa)

[Cpp-Lisa]

Removing SG6 tag

[brycelelbach]

Note that the LEWGI feedback from Cologne didn't make it clear that SG1 needs to see this, but they certainly do. Sorry, my mistake.

[brycelelbach]

This was forwarded to LEWG by SG14 at Cologne.

[brycelelbach]

Belfast 2019-11 LEWGI Minutes

P1673R1 Free Function Linear Algebra Library

Champion: Mark Hoemmen

Minute Taker: Andreas Weis

Start Review: 11-05 8:55

Are the algorithms inplace (like sort) or non-modifying (like transform)?

Should dimensional errors be checked? If we know them all at compile time, we should check them.

Make these constexpr? May not be possible/easy in version 1 - constexprification is ongoing, and you may have dependencies on that.

Walking through the matrix multiply code example was very useful.

Add more examples showing the use of the views.

Do we need to have the mdarray overloads in version 1?

Start Polling: 10:12

POLL: We are comfortable with the design of "mdspan transformations" in P1673 (scaled_view, transpose_view, etc).

Strongly For	Weakly For	Neutral	Weakly Against	Strongly Against
0	11	0	0	1

Attendance: 16

That's got consensus.

POLL: Dimensional errors should be checked at compile time (e.g. Mandates clauses) if all dimensions are known at compile time; otherwise, they should be preconditions (e.g. Expects clauses).

NO OBJECTION TO UNANIMOUS CONSENT

Attendance: 16

POLL: Making the linear algebra algorithms in P1673 constexpr should be a priority for the initial release.

Strongly For	Weakly For	Neutral	Weakly Against	Strongly Against
1	2	7	3	0

Attendance: 16

No consensus.

POLL: mdarray overloads should be a priority for the initial release.

Strongly For	Weakly For	Neutral	Weakly Against	Strongly Against
1	1	4	3	1

Attendance: 17

Weak consensus against.

POLL: Conceptifying input and output arguments should be a priority for the initial release.

Strongly For	Weakly For	Neutral	Weakly Against	Strongly Against
1	2	3	3	1

Attendance: 16

No consensus.

Break: 10:22

Resume: 10:44

Start Polling: 11:15

POLL: Packed layouts are a priority for the initial release.

Strongly For	Weakly For	Neutral	Weakly Against	Strongly Against
3	6	4	0	1

Attendance: 18

SA: I want to start with something as simple as possible.

That has consensus.

POLL: We are comfortable with the design of layout_blas_general and layout_blas_packed in P1673.

NO OBJECTION TO UNANIMOUS CONSENT

Attendance: 18

POLL: Linear algebra algorithms that produce a single scalar output should return the output.

Strongly For	Weakly For	Neutral	Weakly Against	Strongly Against
2	8	2	1	2

Attendance: 18

That has consensus.

POLL: The intermediate type used for reduction-style linear algebra algorithms should be the type of the initial value, or the element type derived from the inputs if there is no initial value (e.g. what std::reduce/std::accumulate does).

Strongly For	Weakly For	Neutral	Weakly Against	Strongly Against
2	7	3	1	0

Attendance: 18

That has consensus.

A: I want an explicit template parameter.

POLL: Send P1673 (Free Function Linear Algebra Library) to LEWG and SG1.

Strongly For	Weakly For	Neutral	Weakly Against	Strongly Against
12	1	3	0	0

Attendance: 18

That has consensus.

After speaking with the SG1 chair, SG1 doesn't need to see this.

End: 11:31

CONSENSUS: LEWGI sends P1673R1 (Free Function Linear Algebra Library), with the guidance below, to LEWG.

• Add the walkthrough of the matrix multiply code to the presentation.
• Add more examples showing the use of mdspan transformations.
• Rename "mdspan views" in the paper to something else (such as "mdspan transformations").
• Propose batched linear algebra in a separate paper, not this one.
• Remove mdarray overloads.
• Check dimensional errors at compile time (e.g. Mandates clauses) if all dimensions are known at compile time; otherwise, make them precondition violations (e.g. Expects clauses).
• Make linear algebra algorithms that produce a single scalar output return that output.
• Make the intermediate type used for reduction-style linear algebra algorithms the type of the initial value, or the element type derived from the inputs if there is no initial value (e.g. what std::reduce/std::accumulate does).

[wg21bot]

P1673R2 A free function linear algebra interface based on the BLAS (Mark Hoemmen, Daisy Hollman, Christian Trott, Daniel Sunderland, Nevin Liber, Siva Rajamanickam, Li-Ta Lo, Damien Lebrun-Grandie, Graham Lopez, Peter Caday, Sarah Knepper, Piotr Luszczek, Timothy Costa)

[wg21bot]

P1673R10 A free function linear algebra interface based on the BLAS (Mark Hoemmen, Daisy Hollman,Christian Trott,Daniel Sunderland,Nevin Liber,Alicia KlinvexLi-Ta Lo,Damien Lebrun-Grandie,Graham Lopez,Peter Caday,Sarah Knepper,Piotr Luszczek,Timothy Costa)

[FabioFracassi]

2022-11-10 13:00 to 15:00 UTC-10 Kona Library Evolution Meeting

P1673R10: A free function linear algebra interface based on the BLAS

2022-11-10 13:00 to 15:00 UTC-10 Kona Library Evolution Minutes

Champion: Christian Trott (in-person)

Chair: Bryce Adelstein Lelbach & Corentin Jabot

Minute Taker: Steve Downey

Start: 2022-11-10 13:10 UTC-10

Could in_vector_t be a contiguous range? No, we need access to things like stride.

Open Questions:

• Conceptification.
  • Of element types.
  • Of mdspans.
• Named type requirements vs concepts for expressing constraints.
• get_mdspan customization point.
• Interoperability with P1385.
• What and how should we specify that the operations do?

Action Items:

• Use CamelCase for the type requirement names.
• Remove the _t from the type requirement names.
• Rename the named type requirements to avoid matrix/vector terminology to avoid conflicts with P1385.
• Explore expressing constraints with concepts instead of named type requirements.

POLL: The specification of linear algebra algorithms can be hand-wavy (draft poll not taken).

POLL: The specification of linear algebra algorithms can be in terms of operators that must exist (draft poll not taken).

David Stone formulated the options for this poll.

POLL: The description of linear algebra algorithms should be (vote for the options you can tolerate, vote as many times as you like):

Options	Votes
Syntax only (status quo)	15
Hand wavy do math	27
Do math	2

Break: 15:00

Resume: 15:24

End: 15:57

Next Steps

Revise the paper, with the guidance below, and return to Library Evolution for further design review.

• Use CamelCase for the type requirement names.
• Remove the _t from the type requirement names.
• Rename the named type requirements to avoid matrix/vector terminology to avoid conflicts with P1385.
• Explore expressing constraints with concepts instead of named type requirements.

[wg21bot]

P1673R11 A free function linear algebra interface based on the BLAS (Mark Hoemmen, Daisy Hollman,Christian Trott,Daniel Sunderland,Nevin Liber,Alicia KlinvexLi-Ta Lo,Damien Lebrun-Grandie,Graham Lopez,Peter Caday,Sarah Knepper,Piotr Luszczek,Timothy Costa)

[brycelelbach]

2023-02-10 13:00 to 17:30 Issaquah Library Evolution Meeting

P1673R11: Linear Algebra

2023-02-10 13:00 to 17:30 UTC-8 Issaquah Library Evolution Minutes

Champion: Mark Hoemmen (R)

Chair: Bryce Adelstein Lelbach (IP) & Billy Baker (IP)

Minute Taker: Steve Downey (IP)

Start: 2023-02-10 13:04 UTC-8

Break: 15:15

Resume: 15:32

Does this paper have:

• Examples?
  • Yes.
• Field experience?
  • Implementation experience?
    • Yes, reference implementation and in NVC++.
  • Usage experience?
    • Yes, reference implementation and in NVC++.
  • Deployment experience?
    • Yes, reference implementation and in NVC++.
• Performance considerations?
  • Yes.
• Discussion of prior art?
  • Yes.
• Changes Library Evolution previously requested?
  • Yes.
• Wording?
  • Yes.
• Breaking changes?
  • No.
• Feature test macro?
  • Yes.
• Freestanding?
  • ?
• Format and/or iostream support?
  • Not applicable
• Hash support?
  • Not applicable
• Allocator support?
  • ?

Should this be freestanding?

Can any algorithms allocate? If so, should they support allocators?

Should out-vector subsume in-vector? Should we add a base vector exposition-only concept? We don't seem to have strong feelings on this - it's exposition only.

POLL: Vote once for the names you like:

Option	Votes
vector_norm2, matrix_one_norm, matrix_inf_norm	0
vector_two_norm, matrix_one_norm, matrix_inf_norm	14
vector_norm2, matrix_norm1, matrix_norm_inf	1
vector_norm_two, matrix_norm_one, matrix_norm_inf	2

Attendance: 19 (in person) + 6 (remote)

# of Authors: 3

POLL: Explore replacing symmetric_matrix_vector_product with an overload of matrix_vector_product.

Strongly Favor	Weakly Favor	Neutral	Weakly Against	Strongly Against
0	0	2	4	11

Attendance: 20 (in person) + 7 (remote)

# of Authors: 3

Author Position: 3 SA

Outcome: Consensus against.

POLL: Use words instead of digits for numbers in linear algebra function names - matrix_rank_1_update would become matrix_rank_one_update, symmetric_matrix_rank_2k_update would become symmetric_matrix_rank_two_k_update.

Strongly Favor	Weakly Favor	Neutral	Weakly Against	Strongly Against
2	3	6	8	1

Attendance: 18 (in person) + 7 (remote)

# of Authors: 3

Author Position: WA, 2 N

Outcome: No consensus for change.

POLL: Replace symmetric_matrix_(left|right)_product with symmetric_matrix_product(B, A, tri, C) and symmetric_matrix_product(A, tri, B, C).

Strongly Favor	Weakly Favor	Neutral	Weakly Against	Strongly Against
7	9	1	0	1

Attendance: 19 (in person) + 7 (remote)

# of Authors: 3

Author Position: 2 SF, 1 WF

Outcome: Consensus in favor.

SA: I think the original made more sense to me.

POLL: Remove the constexpr variables of (row|column)_major_t type and rename (row|column)_major_t to (row|column)_major.

Strongly Favor	Weakly Favor	Neutral	Weakly Against	Strongly Against
0	2	8	7	2

Attendance: 19 (in person) + 8 (remote)

# of Authors: 3

Author Position: 1 WA, 2 SA

Outcome: No consensus for change.

Wording Volunteers:

• Zach Laine
• David Olsen

Editorial Action Items:

• Split [linalg.reqs.val] into two sections: one on requirements on the value type and one on implementatio nfreedom.
• Rephrase [linalg.reqs.val] (3.7) to use "The type T must" language.
• Fix typo in [linalg.reqs.val] (1.1): delete "and reference".
• Change the name of the T init parameters and the parameter to sum_of_squares_result to allow simplification of [linalg.reqs.val] 1.
• In [linalg.reqs.val], say "a value initialized object acts as the additive identity".
• Move the reference to "Fast Linear Algebra Is Stable" into the bibliography.
• Remove "Throws: Nothing" from givens_rotation_setup because it doesn't need to be explicitly stated.
• Remove const from by-value parameters.
• Hoist ExecutionPolicy wording to apply to both this and the parallel algorithms.
• Simplify preconditions that can be stated as an equality comparison of the extents.
• Add a requirement that if not otherwise permitted, in-vectors/in-matrixs may not be the same object as out-vectors/out-matrixs.
• Change remarks about aliasing to say "object" instead of "vector" and "matrix".
• Rephrase aliasing wording.
• Audit "Equivalent to" wording and use the new "as if" wording.

Design Action Items:

• In out-vector/inout-vector/out-matrix/inout-matrix, instead of checking if the element type is const, check if you can assign the element type to the reference type.
• Make given_rotations_setup noexcept.
• Consistently pass complex by value.
• Rename vector_norm2 to vector_two_norm.
• Replace (hermitian|symmetric|triangular)_matrix_(left|right)_product with (hermitian|symmetric|triangular)_matrix_product(B, A, tri, C) and (hermitian|symmetric|triangular)_matrix_product(A, tri, B, C).
• Ensure the BLAS 3 *_update functions have an overload that does not take a scaling parameter.
• Don't remove const from the element type in transposed when you have default_accessor.

POLL: Modify P1673R11 (Linear Algebra) as described below, and then send the revised paper to Library for C++26 classified as B3 - addition, to be confirmed with a Library Evolution electronic poll.

• In out-vector/inout-vector/out-matrix/inout-matrix, instead of checking if the element type is const, check if you can assign the element type to the reference type.
• Make given_rotations_setup noexcept.
• Consistently pass linear algebra value types by value.
• Rename vector_norm2 to vector_two_norm.
• Replace (hermitian|symmetric|triangular)_matrix_(left|right)_product with (hermitian|symmetric|triangular)_matrix_product(B, A, tri, C) and (hermitian|symmetric|triangular)_matrix_product(A, tri, B, C).
• Ensure the BLAS 3 *_update functions have an overload that does not take a scaling parameter.
• Don't remove const from the element type in transposed when you have default_accessor.

Strongly Favor	Weakly Favor	Neutral	Weakly Against	Strongly Against
16	5	0	0	0

Attendance: 19 (in person) + 7 (remote)

# of Authors: 3

Author Position: 3 SF

Outcome: Unanimous consensus in favor

End: 17:40

Summary

We extensively reviewed P1673, the BLAS Linear Algebra proposal. We've seen this paper a number of times before; our goal in this review was to identify any remaining open design questions.

We first reviewed the exposition-only concepts, and decided to check if the element type can be assigned to the reference type, instead of checking if the element type is const. We also didn't remove const from the element type in transposed when you have default_accessor.

We found a number of places in the specification where linear algebra value types are passed by const reference where they should be passed by value. The authors will fix this so that we consistently pass these types by value.

We discussed some inconsistencies in the spelling of algorithms that contain "norm" in their name. To resolve this, we decided to rename vector_norm2 to vector_two_norm.

When reviewing the (hermitian|symmetric|triangular)_matrix_(left|right)_product family of algorithms, we decided that instead of having left and right versions, we should have a single overload of each that takes the triangle parameter in a different place in the parameter list.

We also wanted to add overloads that do not take a scaling parameter to the BLAS 3 *_update functions.

At the conclusion of our review, we had not identified any significant open questions or design changes. So, we took a vote to make the requested modifications to the paper and then send it to Library for C++23, and we had unanimous consensus in favor.

Next Steps

Modify P1673R11 (Linear Algebra) as described below, and then take a Library Evolution electronic poll to send the revised paper to Library for C++26 classified as B3 - addition,

• In out-vector/inout-vector/out-matrix/inout-matrix, instead of checking if the element type is const, check if you can assign the element type to the reference type.
• Make given_rotations_setup noexcept.
• Consistently pass linear algebra value types by value.
• Rename vector_norm2 to vector_two_norm.
• Replace (hermitian|symmetric|triangular)_matrix_(left|right)_product with (hermitian|symmetric|triangular)_matrix_product(B, A, tri, C) and (hermitian|symmetric|triangular)_matrix_product(A, tri, B, C).
• Ensure the BLAS 3 *_update functions have an overload that does not take a scaling parameter.
• Don't remove const from the element type in transposed when you have default_accessor.

[wg21bot]

P1673R12 A free function linear algebra interface based on the BLAS (Mark Hoemmen, Daisy Hollman,Christian Trott,Daniel Sunderland,Nevin Liber,Alicia KlinvexLi-Ta Lo,Damien Lebrun-Grandie,Graham Lopez,Peter Caday,Sarah Knepper,Piotr Luszczek,Timothy Costa)

[brycelelbach]

2023-05 Library Evolution Electronic Poll Outcomes

Poll 1: Send [[P1673R12]] BLAS Linear Algebra to Library Working Group for C++26.

Strongly Favor	Weakly Favor	Neutral	Weakly Against	Strongly Against
5	9	2	1	1

Outcome: Consensus in favor.