[intro.abstract] p8 Is the external function that was originally written in C++ regulated by the standard?

[xmh0511]

Full name of submitter (unless configured in github; will be published with the issue): Jim X

[intro.abstract] p8 says:

The following specify the observable behavior of the program:

• Accesses through volatile glvalues are evaluated strictly according to the rules of the abstract machine.
• Data is delivered to the host environment to be written into files (See also: ISO/IEC 9899:2024, 7.23.3).
• The input and output dynamics of interactive devices shall take place in such a fashion that prompting output is actually delivered before a program waits for input. What constitutes an interactive device is implementation-defined.

This is an exhaustive list that specifies which are observable behaviors. In other words, the stuff not in the list can be eliminated by a conforming implementation as per the "as-if" rule.

Consider this example:

bool network_request(char const *);
int main(){
  network_request("hello");
}

If the external function network_request was written in, for example, assembly, in which world, there is certainly no "object", let alone a volatile object, and anything is either a register, a memory address, an interrupt, or so on. In this case, a conforming implementation can eliminate the call to network_request because it touches nothing that is specified as observable behavior from the perspective of the C++ standard.

If the external function was originally written in C++, however, its definition does not appear in the translation unit of the current program that uses it, the implementation knows nothing about this black box function; it's vague to say whether the network_request has an observable behavior or not. Generally, the burden of proof is on the side that says it exists. If the implementation cannot prove the black box network_request has observable behavior, then it does not have, as per Russell's Teapot.

Suggested Resolution

[intro.abstract] p8 seems to need to give implementations a chance that define extra observable behaviors; these observable behaviors are not anticipated by the standard. For example, network requests, database modifications, interruptions, which affect the real-world behavior

[eisenwave]

If the external function network_request was written in, for example, assembly ...

That sounds like it's written in something that is not standard C++. It's not meaningful to lament whether a C++ program containing random bits of foreign languages can be optimized one way or the other.

If the external function was originally written in C++, however, its definition does not appear in the translation unit of the current program that uses it, the implementation knows nothing about this black box function; it's vague to say whether the network_request has an observable behavior or not. Generally, the burden of proof is on the side that says it exists. If the implementation cannot prove the black box network_request has observable behavior, then it does not have, as per Russell's Teapot.

What you're discussing here has to do with optimization barriers. Compilers cannot optimize network_request if they don't know its definition, so they have to be pessimistic and assume it contains observable behavior.

However, there is nothing vague or unclear in the standard about this. From the perspective of standard C++, there is simply a program made of translation units. The fact that compilers build separate object files in parallel and don't do whole-program optimization is an implementation detail. A heroic implementation could look into the definition of all functions in all TUs; we don't mandate that, but it's possible.

[intro.abstract] p8 seems to need to give implementations a chance that define extra observable behaviors; these observable behaviors are not anticipated by the standard.

You're specifically pointing out observable behaviors that are outside of the scope of the C++ standard, like assembly, and fall into compiler extensions or undefined behavior. There is no defect to be fixed; conforming implementations can already have inline assembly as an extension and consider that to be observable behavior.

[eisenwave]

Another way to look at it is through https://eel.is/c++draft/basic.def.odr#12.sentence-1:

Every program shall contain at least one definition of every function or variable that is odr-used in that program outside of a discarded statement; no diagnostic required.

If you define network_request in say, Assembly, then there exists no C++ definition of that function. No diagnostic is required, so a conforming implementation can link network_request without errors/warnings, but a mixed C++/Assembly program is not considered a standard C++ program anymore.

[xmh0511]

What you're discussing here has to do with optimization barriers. Compilers cannot optimize network_request if they don't know its definition, so they have to be pessimistic and assume it contains observable behavior.

If network_request was originally written in C++, however, its definition does not appear in the TU of the program, for example, when linking the function with a dynamic library. By your last comment, the program is ill-formed; no diagnosis is required? If this is the case, according to [intro.compliance.general] p11, the implementation shall issue a diagnosis

A conforming implementation may have extensions (including additional library functions), provided they do not alter the behavior of any well-formed program. Implementations are required to diagnose programs that use such extensions that are ill-formed according to this document.

AFAIK, when linking a dynamic library(or even an assembly file), no real implementations would issue a diagnosis. Or, do you mean such an external function that was written in C++ is always considered to have a C++ definition, regardless of how it is currently supplied in whatever approach(i.e., dynamic or static library)? But a dynamic or static library seems not to be considered as C++'s translation unit.

[frederick-vs-ja]

The description looks quite invalid to me. I don't think we can request the C++ standard to reason about cross-language interoperability.

By your last comment, the program is ill-formed; no diagnosis is required?

Yeah. Lack of definition of an odr-used function generally causes the program IFNDR.

If this is the case, according to [intro.compliance.general] p11, the implementation shall issue a diagnosis

No. As said in #357, requirement for diagnostic is suppressed in such a case.

But a dynamic or static library seems not to be considered as C++'s translation unit.

I don't see why can't a static library be considered as a translated translation unit or a combination of multiple translated translation units. For dynamic libraries, there seem to be some real issues since nothing in [lex.phases] seems to permit such dynamicity.

[xmh0511]

No. As said in #357, requirement for diagnostic is suppressed in such a case.

The diagnosis is not triggered by the IFNDR itself.

[intro.compliance.general] p11 says:

Implementations are required to diagnose programs that use such extensions that are ill-formed according to this document.

"ill-formed, no diagnosis required" is also ill-formed. In other words, [intro.compliance.general] p11 doesn't exclude IFNDR. So, [intro.compliance.general] p11 applies if the implementation uses that extension.

[eisenwave]

"ill-formed, no diagnosis required" is also ill-formed. In other words, [intro.compliance.general] p11 doesn't exclude IFNDR. So, [intro.compliance.general] p11 applies if the implementation uses that extension.

"ill-formed, no diagnosis required" is also ill-formed. In other words, [intro.compliance.general] p11 doesn't exclude IFNDR. So, [intro.compliance.general] p11 applies if the implementation uses that extension.

As I've said above, a way to interpret the situation is according to https://eel.is/c++draft/basic.def.odr#12.sentence-1. An assembly definition of network_request is not a C++ definition, so it's like there's no definition, and no diagnostic is required in that case, despite the program being ill-formed.

Inline assembly in network_request would be a compiler extension though, and a diagnostic would be required in that case. The "compiler extension" construct is only relevant in C++ when you put things into C++ source files that go beyond the standard. It arguably doesn't apply to linking with foreign language constructs.

Either way, the standard provides ways to conform while still mixing C++ with other languages.

[xmh0511]

The "compiler extension" construct is only relevant in C++ when you put things into C++ source files that go beyond the standard.

Do you mean "such an extension that are ill-formed according to this document" refers to the construct that appears in the C++ source file and violates the syntax or semantics defined by the standard? Not the stuff that supplies aid for the construct that is ill-formed.

[villevoutilainen]

Generally, the burden of proof is on the side that says it exists. If the implementation cannot prove the black box network_request has observable behavior, then it does not have, as per Russell's Teapot.

No, the burden is not on that side. The observable behavior of the program includes whatever network_request() does, and unless you can somehow prove what that behavior is, then you can't skip the call.

What tool is used to produce a TU that contains the definition, or the equivalent thereof, of network_request() is irrelevant.

[xmh0511]

What you're discussing here has to do with optimization barriers. Compilers cannot optimize network_request if they don't know its definition, so they have to be pessimistic and assume it contains observable behavior.

However, a conforming implementation can still eliminate all calls to such functions that are linked with foreign language constructs. These functions do not have the observable behavior specified in the list. Eliminating these calls does not violate the standard, doesn't it?

[villevoutilainen]

However, a conforming implementation can still eliminate all calls to such functions that are linked with foreign language constructs. These functions do not have the observable behavior specified in the list. Eliminating these calls does not violate the standard, doesn't it?

The presumption that foreign language constructs cannot perform volatile accesses is just incorrect, they may have an API/ABI that lets them do that in a manner observable to the rest of the C++ program executing on the abstract machine implementation, and the same goes for files and other i/o.

The whole premise of this issue is incorrect.

[eisenwave]

Do you mean "such an extension that are ill-formed according to this document" refers to the construct that appears in the C++ source file and violates the syntax or semantics defined by the standard? Not the stuff that supplies aid for the construct that is ill-formed.

Yeah, that sounds about right. It's not really an extension in the [intro.copmliance] sense if you link a C++ program with an external blob of assembly that was never C++ code to begin with.

However, a conforming implementation can still eliminate all calls to such functions that are linked with foreign language constructs. These functions do not have the observable behavior specified in the list. Eliminating these calls does not violate the standard, doesn't it?

The standard doesn't say whether it's okay or not okay to optimize out calls to Fortran or JavaScript or Brainfuck. You're not dealing with standard C++ there, and the implementation can do whatever it wants.

[xmh0511]

No, the burden is not on that side. The observable behavior of the program includes whatever network_request() does, and unless you can somehow prove what that behavior is, then you can't skip the call.

@villevoutilainen All observable behaviors are exhaustively listed in [intro.abstract] p8. The burden of proof is that you should prove that the black box function contains any one of them(i.e., existence of one observable behavior); otherwise, it does not exist. (https://en.wikipedia.org/wiki/Russell%27s_teapot)

[villevoutilainen]

The burden of proof is that you should prove that the black box function contains any one of them

You can keep claiming that, but it's not correct no matter how many times you repeat it. In order to skip a function call and claim that that results in equivalent behavior, you need to prove that the behavior you ended up with is actually equivalent.

[xmh0511]

The standard doesn't say whether it's okay or not okay to optimize out calls to Fortran or JavaScript or Brainfuck. You're not dealing with standard C++ there, and the implementation can do whatever it wants.

@eisenwave So, eliminating all calls or being conservative to preserve all calls, you think, does either of them is a conforming implementation?

[eisenwave]

So, eliminating all calls or being conservative to preserve all calls, you think, does either of them is a conforming implementation?

When those calls are linked with Fortran, the program is IFNDR anyway, and either option conforms, with or without diagnostics.

When they are linked with C++, the implementation typically has to be conservative with optimizations, but that's an implementation detail, not mandated behavior.