malloc is optimized out for release build

[llvmbot]

Bugzilla Link	28416
Resolution	INVALID
Resolved on	May 26, 2021 07:02
Version	3.8
OS	MacOS X
Reporter	LLVM Bugzilla Contributor
CC	@majnemer,@DougGregor,@hfinkel,@yaupon0

Extended Description

the following code behaves incorrectly

int main(int argc, const char * argv[]) {
size_t need_size = 0x1000000000000;
void *data = malloc(need_size);
if(data == NULL) {
printf("data == NULL\n");
return 1;
} else {
printf("data != NULL\n");
}
free(data);
return 0;
}

it returns

data != NULL
Program ended with exit code: 0

but it should return

data == NULL
Program ended with exit code: 1

gcc handles that situation correctly

clang shouldn't optimize it out, because result of the malloc is used and

caller of my program relies on the exit code

Note, clang flag -fno-builtin prevents clang from malloc optimize out

and solve the problem.

[majnemer]

LLVM has not optimized your program into an inconsistent state because calling malloc is not an observable side effect in and of itself. If the compiler can satisfy your use of malloc in a way which doesn't end up calling malloc, that's ok.

If you did something else with the malloc'd pointer like called malloc_usable_size (or escaped it through any other mechanism) then that would be a different story.

[llvmbot]

we have a use case which is far more complicated than example I used in the bug report, but even in my example

result of the malloc was used in the exit code of the program, clang shouldn't

alter logic of the program

it MUST return NULL if memory wasn't allocated and was used in the

next line of the code

from malloc man page:

RETURN VALUES
If successful, calloc(), malloc(), realloc(), reallocf(), and valloc() functions return a pointer to
allocated memory. If there is an error, they return a NULL pointer and set errno to ENOMEM.

so clang behavior isn't compliant to the malloc documentation

you're saying: "If the compiler can satisfy your use of malloc in a way which doesn't end up calling malloc, that's ok."

problem is that in my example compiler didn't satisfy my use of malloc and

alter code flow which isn't acceptable

[llvmbot]

See https://groups.google.com/forum/#!topic/llvm-dev/lV30rcmF0ss

[llvmbot]

thanks for the link

I disagree with statement about LLVM malloc implementation

compiler shouldn't call any function at the compile time

and compiler can't know what will be outcome of malloc(sz)

(and in my example code I do use result of malloc call)

therefore compiler optimization isn't good

also I agree with opinion that setting error code is side effect and therefore

malloc can't be optimized out. It's not clear to me why malloc should never

fail (if by failing we understand returning NULL). If we follow that logic then

operator new never throws which is clearly

in disagreement with C++ standard

[majnemer]

thanks for the link

I disagree with statement about LLVM malloc implementation

compiler shouldn't call any function at the compile time

and compiler can't know what will be outcome of malloc(sz)

(and in my example code I do use result of malloc call)

therefore compiler optimization isn't good

also I agree with opinion that setting error code is side effect and
therefore

malloc can't be optimized out.

We rewrite the call to malloc as-if it succeeded which means that errno is not changed.

It's not clear to me why malloc should never

fail (if by failing we understand returning NULL). If we follow that logic
then

operator new never throws which is clearly

in disagreement with C++ standard

Actually, the C++ standard is carefully worded to permit us optimizing away calls to operator new.

[llvmbot]

thanks
yes you can optimize malloc where it is possible to do

in the example I provided it is simply wrong thing to do

we have real case (it's MATLAB BTW) where malloc for huge number of required

memory returns not NULL and we continue to execute and eventually our

application (MATLAB) simply crashes. That is not acceptable by any mean

real case isn't possible to reproduce here - it's too complicate

it requires install of MATLAB and Simulink, some specific

Simulink model and few other things. so when I found that simple case

I decided to submit it as bug. It's worth mention that

consider the following C++ code

int main(int argc, const char * argv[]) {
size_t need_size = 0x1000000000000;
unsigned char *chars = nullptr;
try {
chars = new unsigned char[need_size];
std::fill(chars, chars + need_size, 'a');
} catch (std::bad_alloc &ex) {
std::cout << "allocation error: " << ex.what() << std::endl;
chars = nullptr;
}
if(chars != nullptr) {
std::cout << "alloction success: <<" << static_cast(chars[need_size / 2]) << ">>" << std::endl;
delete [] chars;
chars = nullptr;
}
return 0;
}

that code behaves incorrectly and doesn't throw exception, however

the following code

int main(int argc, const char * argv[]) {
size_t need_size = 0x1000000000000;
unsigned char *chars = nullptr;
std::vector v;
try {
v.assign(need_size, 'a');
chars = v.data();
} catch (std::bad_alloc &ex) {
std::cout << "allocation error: " << ex.what() << std::endl;
chars = nullptr;
}
if(chars != nullptr) {
std::cout << "alloction success: <<" << static_cast(chars[need_size / 2]) << ">>" << std::endl;
chars = nullptr;
}
return 0;
}

behaves as expected, simply speaking

std::vector v(0x1000000000000); throws exception

but std::unique_ptr v(new unsigned char[0x1000000000000]);

doesn't throw any exception

thanks

[llvmbot]

I meant

std::unique_ptr<unsigned char []> v(new unsigned char[0x1000000000000]);

[hfinkel]

thanks
yes you can optimize malloc where it is possible to do

in the example I provided it is simply wrong thing to do

we have real case (it's MATLAB BTW) where malloc for huge number of required

memory returns not NULL and we continue to execute and eventually our

application (MATLAB) simply crashes. That is not acceptable by any mean

real case isn't possible to reproduce here - it's too complicate

it requires install of MATLAB and Simulink, some specific

Simulink model and few other things.

Is MATLAB crashing because it assumes that malloc must eventually fail even if it does not make use of the memory?

[llvmbot]

sorry for the troubles

apparently our problem has nothing with malloc optimization

code I attached to the bug report doesn't really represent our use case,

it's probably related to memory overcommitting

so I'm closing that report

sorry again

dm

[Kojoley]

*** Bug llvm/llvm-bugzilla-archive#50482 has been marked as a duplicate of this bug. ***

[Kojoley]

mentioned in issue llvm/llvm-bugzilla-archive#50482