CWG3134 [expr.cond] Unclear behavior of bit-fields in conditional operators

[Halalaluyafail3]

Full name of submitter (unless configured in github; will be published with the issue): Jay Ghiron

Reference (section label): [expr.cond]/5

If the second and third operands are glvalues of the same value category and have the same type, the result is of that type and value category and it is a bit-field if the second or the third operand is a bit-field, or if both are bit-fields.

Issue description:

Currently, if the second and third operands of a conditional operator have the same type and value category then the resulting expression has the same type and value category as both the second and third operands. Additionally, if both the second and third operands are glvalues and either is a bit-field then the resulting expression is also a bit-field. This wording creates two confusing situations. First, if both the second and third operands are bit-field glvalues of the same value category and type but do not have the same widths. The wording requires the resulting expression to be a bit-field glvalue, but does not specify what the width is. Secondly, if either the second or third operand is a bit-field glvalue and the other is a non-bit-field glvalue of the same value category and type. The wording requires the resulting expression to be a bit-field glvalue, even though it might not end up referring to a bit-field object.

#include<limits>
static_assert(std::numeric_limits<int>::max()<std::numeric_limits<long long>::max());
int main(){
    struct{int x:3,y:4;}a;
    struct{unsigned x:std::numeric_limits<unsigned>::digits-1,y:std::numeric_limits<unsigned>::digits;}b{};
    int c;
    //a.y can represent 7, a.x cannot represent 7
    (0?a.x:a.y)=7;//does this surely store 7 in a.y?
    (1?a.x:a.y)=7;//if the answer to the above is yes, does this store an implementation-defined value in a.x or is a specific value guaranteed?
    auto d=(1?b.x:b.y)-1;
    d<0;//is this expression true or false? that is, does promoting (1?b.x:b.y) result in int or unsigned
    (1?c:a.x)=7;//does this surely store 7 in c?
    (0?c:a.x)=7;//if the answer to the above is yes, does this store an implementation-defined value in a.x or is a specific value guaranteed?
    (1?c:a.x)=std::numeric_limits<int>::max()+1LL;//if 7 was surely stored in c before, does this surely store std::numeric_limits<int>::min() in c or is an implementation-defined value stored?
    //the expressions above which assigned to c did so with bit-field expressions, what memory location was modified by those assignments?
    unsigned e{};
    auto f=(1?b.x:e)-1;
    f<0;//is this expression true? that is, does promoting (1?b.x:e) result in int or unsigned
}

GCC, Clang, and MSVC all agree on this behavior:
(0?a.x:a.y)=7 does store 7
(1?a.x:a.y)=7 stores -1 (implementation-defined value)
d<0 is false, it promotes to unsigned
(1?c:a.x)=7 does store 7
(0?c:a.x)=7 stores -1 (implementation-defined value)
(1?c:a.x)=std::numeric_limits<int>::max()+1LL does store std::numeric_limits<int>::min()
f<0 is false, it promotes to unsigned

The first 4 assignments I think can have their behavior reasonably intuited from the current wording (using the width of the selected branch), even if it is not very clear. The last assignment appears to assign an implementation-defined value from the current wording, though I cannot imagine that is the intent and no implementations appear to assign a different value than would occur with wraparound. There is no clear wording to judge whether or not d<0 and f<0 should be true or false. I have not tested how any implementation views the assignments to c in terms of which memory locations are modified.

GCC, Clang, and MSVC also disagree with the intuitive interpretation of what should happen with a conditional operator where the second and third operands are both bit-field glvalues of the same value category, type, and width.

int main(){
    struct{unsigned u:1;}u{};
    auto y=(1?u.u:u.u)-1;
    y<0;//is this expression true? that is, does promoting (1?u.u:u.u) result in int or unsigned
}

All three implementations agree that y<0 is false, that is (1?u.u:u.u) promotes to unsigned. Both GCC and Clang agree that with u.u instead of (1?u.u:u.u), the promoted type should be int instead. MSVC also has u.u promote to unsigned, but that is surely a bug. So it seems like even if the widths match in a conditional operator, implementations still intentionally disregard that when determining the promoted type. GCC and Clang diverge about how to promote bit-fields inside of conditional operators.

int main(){
    struct{unsigned u:1;}u{};
    auto y=(1?u.u:0)-1;
    y<0;//true with Clang, false with GCC and MSVC
}

This also seems like a bug, the language is clear here that the usual arithmetic conversions should take place.

All three implementations also have issues implementing conditional operators that result in bit-fields when the resulting expression is assigned to, even when both operands are bit-fields and their widths match.

int main(){
    struct{int x:4;}a;
    struct{int y:2,x:4;}b;
    int r=1;
    (r?a.x=0:b.x=0)+=1;
}

This is rejected by all three implementations, for different reasons. GCC has issues ensuring the correct evaluation order in these cases, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=122772. Clang just says "cannot compile this conditional operator yet". MSVC does not recognize (r?a.x=0:b.x=0) as an lvalue, and this even changes the meaning of declarations such as decltype(r?a.x=0:b.x=0)q; which MSVC accepts (surely a bug). MSVC seems to be able to correctly handle this program if either the assignment to a.x or b.x is removed, or both.

Considering the implementation difficulty, it might be worth drastically changing the semantics of conditional operators that currently result in bit-fields. No proposed wording is provided, but here are some possible resolutions (from most to least severe):

• Reject all conditional expressions that currently are specified as resulting in bit-fields, or change them to prvalues instead.
• Reject conditional expressions that mix different widths, bit-fields and non-bit-fields, or bit-fields of different classes; or change them to prvalues instead. Clarify that the width of the bit-field from a conditional operator is the same as its operands.
• Reject conditional expressions that mix different widths or bit-fields and non-bit-fields, or change them to prvalues instead. Clarify that the width of the bit-field from a conditional operator is the same as its operands.
• Reject conditional expressions that mix bit-fields and non-bit-fields, or change them to prvalues instead. Clarify the widths of bit-fields from conditional operators, including when the widths are different.
• No changes to value categories or what expressions are allowed. Clarify the widths of bit-fields from conditional operators, including when the widths are different or bit-fields and non-bit-fields are mixed. Clarify what it means to have a bit-field expression refer to a non-bit-field object.

[lprv]

Looks like the wording issue here boils down to the fact that [conv.prom]/5, [dcl.init.list]/7.4.2, and [expr.assign]/4 talk about bit-field width as a static property of an expression, but the rest of the wording does not specify it.

The fix for the latter should be pretty straightforward:

When the result of the left operand is a bit-field ...

For the other two cases, one option would be to modify every occurrence of "[expression] is a bit-field" to also specify the width.

[Halalaluyafail3]

For the other two cases, one option would be to modify every occurrence of "[expression] is a bit-field" to also specify the width.

There are only a few places that do this. CWG324 was the issue that introduced the problematic wording and it seems to have been done only when considering whether taking the address is valid.

[jensmaurer]

There is nothing wrong with CWG324 per se.

An expression has some properties (e.g. type and value category). Furthermore, an lvalue can be a bit-field. When CWG324 was drafted, it was apparently believed it's enough to specify the boolean property "bit-field". However, we've now found places where we actually need the width of the bit-field lvalue. And that part is underspecified and needs to be fixed.

[jensmaurer]

CWG3134

[t3nsor]

I think there's remaining implementation difficulty even with the "take the minimum of the two widths" rule. The OP pointed out to me that bit-field expressions have a hidden additional property of offset within their memory location, which affects the generated code for e.g. storing into the bit-field. So even with width becoming a static property of a bit-field expression we still have the problem that the conditional operator can create bit-field expressions that have offset as a dynamic property.

Like the OP, I don't know what we should do about this issue; I think this is one of those cases where we just have to talk to all the implementors and find out what they want. If we could just make the result a prvalue in all cases where either the second or third operand is a bit-field, that would surely be the easiest rule to implement, and the implementors would be best-placed to advise as to how much pain it would cause for their users.

[Halalaluyafail3]

CWG3134

I don't think this wording would be a good resolution:

If both operands are bit-fields of widths N and M, respectively, the result is a bit field of width K, where K is the minimum of N and M.

#include<limits>
static_assert(std::numeric_limits<unsigned char>::max()<std::numeric_limits<int>::max());
int main(){
    struct{unsigned x:std::numeric_limits<unsigned char>::digits,y:std::numeric_limits<unsigned char>::digits+1;}a;
    struct{unsigned x:std::numeric_limits<unsigned>::digits-1,y:std::numeric_limits<unsigned>::digits;}b;
    a.y=std::numeric_limits<unsigned char>::max()+1;
    unsigned char c{0?a.x:a.y};//not narrowing with this wording, even though the value isn't representable
    //implementations as is consider this narrowing, regardless of the widths chosen; indicating that they just discard the widths here
    //MSVC even rejects unsigned char c{a.x}; due to narrowing, but that is surely a bug
    b.y=std::numeric_limits<unsigned>::max();
    auto d=+(0?b.x:b.y);//promotion changes value with this wording, which promotion is not meant to do
}

Additionally, [expr.assign]/4 doesn't even mention the width so it's not clear if this change would answer the original question. To make promotion and narrowing make sense, it would surely have to be the maximum of the widths instead of the minimum of the widths. Assignment is more tricky, but I think if it's to be allowed then it should choose the selected branch so (a?b:c)=d is always the same as a?b=d:c=d.

Also, the issue of bit-field expressions referring to non-bit-field objects should be mentioned too.

[jensmaurer]

The lack of spaces in your examples makes it hard for me to read them.

We can certainly track both the min (for safe assignment) and the max (for narrowing) through the expression tree, if we want to go there.

Since this is all at the expression level (bit-field-ness can't propagate across function calls, for example), I don't see a problem with forwarding a (hidden) runtime offset, similar to forwarding the hidden pointer when non-bi-tfield lvalues are involved.

I'm not seeing how we can specify that

(a ? b : c) = d is always the same as a ? b = d : c = d

given that d must be evaluated only once. Is the flattening going to grow exponentially as the number of nested conditional expressions increases? That might be unfriendly.

[jensmaurer]

I've fixed CWG3134 to also update [expr.assign]p4 with our tried-and-trusted "hypothetical extended integer type of width x" wording.

If we go with forwarding min and max through the expression tree (which seems eminently possible), assignment needs to use the min and narrowing checking and promotion needs to use the max.

[jensmaurer]

Like the OP, I don't know what we should do about this issue

That sounds very much like an EWG question, then. Maybe CWG324 was misguided to forward bit-field-ness through the conditional operator.

[Halalaluyafail3]

I'm not seeing how we can specify that

(a ? b : c) = d is always the same as a ? b = d : c = d

given that d must be evaluated only once. Is the flattening going to grow exponentially as the number of nested conditional expressions increases? That might be unfriendly.

d is only evaluated once in the second option, since the conditional operator will never evaluate both the second and third operands. Also, there is already precedent for just saying "except ... is only evaluated once" and specifying it in a way that if literally flattened would increase the size exponentially: [expr.assign]/6.

We can certainly track both the min (for safe assignment) and the max (for narrowing) through the expression tree, if we want to go there.

That would make sense, though if the minimum is to be used for assignment then I think it should be stated that if it's not representable in the minimum size bit-field then the value stored in the bit-field can be anything it can represent.

int main(){
    struct{unsigned x:1,y:2;}a;
    (1?a.x:a.y)=2;//can only store 0 or 1
    (0?a.x:a.y)=2;//can store 0, 1, 2, or 3
}

After thinking about this again it can also occur in the case of mixing bit-fields and non-bit-fields:

#include<limits>
int main(){
    struct{unsigned x:1;}a;
    unsigned b=std::numeric_limits<unsigned>::max();
    auto c=0?a.x:b;//promoting to int here too changes the value
    unsigned char d{0?a.x:b};//should be recognized as narrowing too
}

[jensmaurer]

If we go with flattening for the assignment case (which needs a bit of novel specification machinery, I think), what do we do for the narrowing checking and promotion situations? Looks like we'd need to track the maximum width through the expression tree for those.

[t3nsor]

I don't think we should actually flatten in the wording. If we wanted to make implementations flatten, we could just say that the width of the result is the width of whichever operand was chosen. (In other words, by making the width a dynamic property, we would force implementations to generate different code for the assignment depending on which operand was chosen by the conditional expression, which is sort of like making the assignment itself part of what gets chosen based on the first operand to the conditional operator.) In fact, I think that's what the current wording (unclearly) says. Which then raises the issue that perhaps, for over a decade, implementors have found this too annoying to actually implement.

[jensmaurer]

for over a decade, implementors have found this too annoying to actually implement.

Well, this core issue would serve as a nice prodding then, I guess.

I've fixed CWG3134 to use "result of" for the assignment operator (which causes dynamic tracking, i.e. flattening), and I've changed the "width" property to be the maximum, which fixes promotion and narrowing checking.

[Halalaluyafail3]

From testing it appears that all three compilers also fail to propagate the width in a conditional expression where one operand is a throw-expression:

int main(){
    struct{unsigned x:1;}a{};
    auto b=(0?throw 0:a.x)-1;
    b<0;//GCC, Clang, and MSVC all agree this is false, i.e. (0?throw 0:a.x) promotes to unsigned not int
}