Skip to content

Commit

Permalink
[Clang] Improve the handling of large arrays evaluation.
Browse files Browse the repository at this point in the history
This is a temporary fix (for clang 17) that caps the size of
any array we try to constant evaluate:

    There are 2 limits:
      * We cap to UINT_MAX the size of ant constant evaluated array,
        because the constant evaluator does not support size_t.
      * We cap to `-fconstexpr-steps` elements the size of each individual
        array and dynamic array allocations.
        This works out because the number of constexpr steps already limits
        how many array elements can be initialized, which makes this new
        limit conservatively generous.
        This ensure that the compiler does not crash when attempting to
        constant-fold valid programs.

    If the limit is reached by a given array, constant evaluation will fail,
    and the program will be ill-formed, until a bigger limit is given.
    Or, constant folding will fail and the array will be evaluated at runtime.

    Fixes #63562

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D155955
  • Loading branch information
cor3ntin committed Jul 28, 2023
1 parent a457067 commit 45ab2b4
Show file tree
Hide file tree
Showing 7 changed files with 162 additions and 11 deletions.
5 changes: 5 additions & 0 deletions clang/docs/ReleaseNotes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,11 @@ Bug Fixes to Attribute Support
Bug Fixes to C++ Support
^^^^^^^^^^^^^^^^^^^^^^^^

- Clang limits the size of arrays it will try to evaluate at compile time
to avoid memory exhaustion.
This limit can be modified by `-fconstexpr-steps`.
(`#63562 <https://github.com/llvm/llvm-project/issues/63562>`_)

Bug Fixes to AST Handling
^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down
4 changes: 3 additions & 1 deletion clang/docs/UsersManual.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3348,7 +3348,9 @@ Controlling implementation limits
.. option:: -fconstexpr-steps=N

Sets the limit for the number of full-expressions evaluated in a single
constant expression evaluation. The default is 1048576.
constant expression evaluation. This also controls the maximum size
of array and dynamic array allocation that can be constant evaluated.
The default is 1048576.

.. option:: -ftemplate-depth=N

Expand Down
2 changes: 2 additions & 0 deletions clang/include/clang/AST/Type.h
Original file line number Diff line number Diff line change
Expand Up @@ -3144,6 +3144,8 @@ class ConstantArrayType final
QualType ElementType,
const llvm::APInt &NumElements);

unsigned getNumAddressingBits(const ASTContext &Context) const;

/// Determine the maximum number of active bits that an array's size
/// can require, which limits the maximum size of the array.
static unsigned getMaxSizeBits(const ASTContext &Context);
Expand Down
3 changes: 3 additions & 0 deletions clang/include/clang/Basic/DiagnosticASTKinds.td
Original file line number Diff line number Diff line change
Expand Up @@ -351,6 +351,9 @@ def note_constexpr_new_negative : Note<
"cannot allocate array; evaluated array bound %0 is negative">;
def note_constexpr_new_too_large : Note<
"cannot allocate array; evaluated array bound %0 is too large">;
def note_constexpr_new_exceeds_limits : Note<
"cannot allocate array; evaluated array bound %0 exceeds the limit (%1); "
"use '-fconstexpr-steps' to increase this limit">;
def note_constexpr_new_too_small : Note<
"cannot allocate array; evaluated array bound %0 is too small to hold "
"%1 explicitly initialized elements">;
Expand Down
59 changes: 49 additions & 10 deletions clang/lib/AST/ExprConstant.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1019,6 +1019,34 @@ namespace {
return false;
}

bool CheckArraySize(SourceLocation Loc, unsigned BitWidth,
uint64_t ElemCount, bool Diag) {
// FIXME: GH63562
// APValue stores array extents as unsigned,
// so anything that is greater that unsigned would overflow when
// constructing the array, we catch this here.
if (BitWidth > ConstantArrayType::getMaxSizeBits(Ctx) ||
ElemCount > uint64_t(std::numeric_limits<unsigned>::max())) {
if (Diag)
FFDiag(Loc, diag::note_constexpr_new_too_large) << ElemCount;
return false;
}

// FIXME: GH63562
// Arrays allocate an APValue per element.
// We use the number of constexpr steps as a proxy for the maximum size
// of arrays to avoid exhausting the system resources, as initialization
// of each element is likely to take some number of steps anyway.
uint64_t Limit = Ctx.getLangOpts().ConstexprStepLimit;
if (ElemCount > Limit) {
if (Diag)
FFDiag(Loc, diag::note_constexpr_new_exceeds_limits)
<< ElemCount << Limit;
return false;
}
return true;
}

std::pair<CallStackFrame *, unsigned>
getCallFrameAndDepth(unsigned CallIndex) {
assert(CallIndex && "no call index in getCallFrameAndDepth");
Expand Down Expand Up @@ -3583,6 +3611,14 @@ static bool lifetimeStartedInEvaluation(EvalInfo &Info,
llvm_unreachable("unknown evaluating decl kind");
}

static bool CheckArraySize(EvalInfo &Info, const ConstantArrayType *CAT,
SourceLocation CallLoc = {}) {
return Info.CheckArraySize(
CAT->getSizeExpr() ? CAT->getSizeExpr()->getBeginLoc() : CallLoc,
CAT->getNumAddressingBits(Info.Ctx), CAT->getSize().getZExtValue(),
/*Diag=*/true);
}

namespace {
/// A handle to a complete object (an object that is not a subobject of
/// another object).
Expand Down Expand Up @@ -3757,6 +3793,9 @@ findSubobject(EvalInfo &Info, const Expr *E, const CompleteObject &Obj,
if (O->getArrayInitializedElts() > Index)
O = &O->getArrayInitializedElt(Index);
else if (!isRead(handler.AccessKind)) {
if (!CheckArraySize(Info, CAT, E->getExprLoc()))
return handler.failed();

expandArray(*O, Index);
O = &O->getArrayInitializedElt(Index);
} else
Expand Down Expand Up @@ -6491,6 +6530,9 @@ static bool HandleDestructionImpl(EvalInfo &Info, SourceLocation CallLoc,
uint64_t Size = CAT->getSize().getZExtValue();
QualType ElemT = CAT->getElementType();

if (!CheckArraySize(Info, CAT, CallLoc))
return false;

LValue ElemLV = This;
ElemLV.addArray(Info, &LocE, CAT);
if (!HandleLValueArrayAdjustment(Info, &LocE, ElemLV, ElemT, Size))
Expand Down Expand Up @@ -6681,7 +6723,7 @@ static bool HandleDestruction(EvalInfo &Info, SourceLocation Loc,
return HandleDestructionImpl(Info, Loc, LV, Value, T);
}

/// Perform a call to 'perator new' or to `__builtin_operator_new'.
/// Perform a call to 'operator new' or to `__builtin_operator_new'.
static bool HandleOperatorNewCall(EvalInfo &Info, const CallExpr *E,
LValue &Result) {
if (Info.checkingPotentialConstantExpression() ||
Expand Down Expand Up @@ -6727,13 +6769,12 @@ static bool HandleOperatorNewCall(EvalInfo &Info, const CallExpr *E,
return false;
}

if (ByteSize.getActiveBits() > ConstantArrayType::getMaxSizeBits(Info.Ctx)) {
if (!Info.CheckArraySize(E->getBeginLoc(), ByteSize.getActiveBits(),
Size.getZExtValue(), /*Diag=*/!IsNothrow)) {
if (IsNothrow) {
Result.setNull(Info.Ctx, E->getType());
return true;
}

Info.FFDiag(E, diag::note_constexpr_new_too_large) << APSInt(Size, true);
return false;
}

Expand Down Expand Up @@ -9617,14 +9658,12 @@ bool PointerExprEvaluator::VisitCXXNewExpr(const CXXNewExpr *E) {

// -- its value is such that the size of the allocated object would
// exceed the implementation-defined limit
if (ConstantArrayType::getNumAddressingBits(Info.Ctx, AllocType,
ArrayBound) >
ConstantArrayType::getMaxSizeBits(Info.Ctx)) {
if (!Info.CheckArraySize(ArraySize.value()->getExprLoc(),
ConstantArrayType::getNumAddressingBits(
Info.Ctx, AllocType, ArrayBound),
ArrayBound.getZExtValue(), /*Diag=*/!IsNothrow)) {
if (IsNothrow)
return ZeroInitialization(E);

Info.FFDiag(*ArraySize, diag::note_constexpr_new_too_large)
<< ArrayBound << (*ArraySize)->getSourceRange();
return false;
}

Expand Down
5 changes: 5 additions & 0 deletions clang/lib/AST/Type.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,11 @@ unsigned ConstantArrayType::getNumAddressingBits(const ASTContext &Context,
return TotalSize.getActiveBits();
}

unsigned
ConstantArrayType::getNumAddressingBits(const ASTContext &Context) const {
return getNumAddressingBits(Context, getElementType(), getSize());
}

unsigned ConstantArrayType::getMaxSizeBits(const ASTContext &Context) {
unsigned Bits = Context.getTypeSize(Context.getSizeType());

Expand Down
95 changes: 95 additions & 0 deletions clang/test/SemaCXX/cxx2a-constexpr-dynalloc-limits.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
// RUN: %clang_cc1 -std=c++20 -verify -fconstexpr-steps=1024 -Wvla %s

namespace std {
using size_t = decltype(sizeof(0));
}

void *operator new(std::size_t, void *p) { return p; }

namespace std {
template<typename T> struct allocator {
constexpr T *allocate(size_t N) {
return (T*)operator new(sizeof(T) * N); // #alloc
}
constexpr void deallocate(void *p) {
operator delete(p);
}
};
template<typename T, typename ...Args>
constexpr void construct_at(void *p, Args &&...args) { // #construct
new (p) T((Args&&)args...);
}
}

namespace GH63562 {

template <typename T>
struct S {
constexpr S(unsigned long long N)
: data(nullptr){
data = alloc.allocate(N); // #call
for(std::size_t i = 0; i < N; i ++)
std::construct_at<T>(data + i, i); // #construct_call
}
constexpr T operator[](std::size_t i) const {
return data[i];
}

constexpr ~S() {
alloc.deallocate(data);
}
std::allocator<T> alloc;
T* data;
};

constexpr std::size_t s = S<std::size_t>(1099511627777)[42]; // expected-error {{constexpr variable 's' must be initialized by a constant expression}} \
// expected-note@#call {{in call to 'this->alloc.allocate(1099511627777)'}} \
// expected-note@#alloc {{cannot allocate array; evaluated array bound 1099511627777 is too large}} \
// expected-note {{in call to 'S(1099511627777)'}}
// Check that we do not try to fold very large arrays
std::size_t s2 = S<std::size_t>(1099511627777)[42];
std::size_t s3 = S<std::size_t>(~0ULL)[42];

// We can allocate and initialize a small array
constexpr std::size_t ssmall = S<std::size_t>(100)[42];

// We can allocate this array but we hikt the number of steps
constexpr std::size_t s4 = S<std::size_t>(1024)[42]; // expected-error {{constexpr variable 's4' must be initialized by a constant expression}} \
// expected-note@#construct {{constexpr evaluation hit maximum step limit; possible infinite loop?}} \
// expected-note@#construct_call {{in call}} \
// expected-note {{in call}}



constexpr std::size_t s5 = S<std::size_t>(1025)[42]; // expected-error{{constexpr variable 's5' must be initialized by a constant expression}} \
// expected-note@#alloc {{cannot allocate array; evaluated array bound 1025 exceeds the limit (1024); use '-fconstexpr-steps' to increase this limit}} \
// expected-note@#call {{in call to 'this->alloc.allocate(1025)'}} \
// expected-note {{in call}}


// Check we do not perform constant initialization in the presence
// of very large arrays (this used to crash)

template <auto N>
constexpr int stack_array() {
[[maybe_unused]] char BIG[N] = {1}; // expected-note 3{{cannot allocate array; evaluated array bound 1025 exceeds the limit (1024); use '-fconstexpr-steps' to increase this limit}}
return BIG[N-1];
}

int a = stack_array<~0U>();
int c = stack_array<1024>();
int d = stack_array<1025>();
constexpr int e = stack_array<1024>();
constexpr int f = stack_array<1025>(); // expected-error {{constexpr variable 'f' must be initialized by a constant expression}} \
// expected-note {{in call}}
void ohno() {
int bar[stack_array<1024>()];
int foo[stack_array<1025>()]; // expected-warning {{variable length arrays are a C99 feature}} \
// expected-note {{in call to 'stack_array()'}}

constexpr int foo[stack_array<1025>()]; // expected-warning {{variable length arrays are a C99 feature}} \
// expected-error {{constexpr variable cannot have non-literal type 'const int[stack_array<1025>()]'}} \
// expected-note {{in call to 'stack_array()'}}
}

}

0 comments on commit 45ab2b4

Please sign in to comment.