Skip to content

Commit

Permalink
Adding a width of the GEP index to the Data Layout.
Browse files Browse the repository at this point in the history 
Making a width of GEP Index, which is used for address calculation, to be one of the pointer properties in the Data Layout.
p[address space]:size:memory_size:alignment:pref_alignment:index_size_in_bits.
The index size parameter is optional, if not specified, it is equal to the pointer size.

Till now, the InstCombiner normalized GEPs and extended the Index operand to the pointer width.
It works fine if you can convert pointer to integer for address calculation and all registered targets do this.
But some ISAs have very restricted instruction set for the pointer calculation. During discussions were desided to retrieve information for GEP index from the Data Layout.
http://lists.llvm.org/pipermail/llvm-dev/2018-January/120416.html

I added an interface to the Data Layout and I changed the InstCombiner and some other passes to take the Index width into account.
This change does not affect any in-tree target. I added tests to cover data layouts with explicitly specified index size.

Differential Revision: https://reviews.llvm.org/D42123

llvm-svn: 325102
  • Loading branch information
Elena Demikhovsky committed Feb 14, 2018
1 parent 5ecea9f commit 945b7e5
Show file tree
Hide file tree
Showing 30 changed files with 1,630 additions and 131 deletions.
47 changes: 24 additions & 23 deletions llvm/docs/LangRef.rst
View file
Expand Up @@ -1065,7 +1065,7 @@ Currently, only the following parameter attributes are defined:
for return values.

.. _attr_align:

``align <n>``
This indicates that the pointer value may be assumed by the optimizer to
have the specified alignment.
Expand Down Expand Up @@ -1908,10 +1908,12 @@ as follows:
``A<address space>``
Specifies the address space of objects created by '``alloca``'.
Defaults to the default address space of 0.
``p[n]:<size>:<abi>:<pref>``
``p[n]:<size>:<abi>:<pref>:<idx>``
This specifies the *size* of a pointer and its ``<abi>`` and
``<pref>``\erred alignments for address space ``n``. All sizes are in
bits. The address space, ``n``, is optional, and if not specified,
``<pref>``\erred alignments for address space ``n``. The fourth parameter
``<idx>`` is a size of index that used for address calculation. If not
specified, the default index size is equal to the pointer size. All sizes
are in bits. The address space, ``n``, is optional, and if not specified,
denotes the default address space 0. The value of ``n`` must be
in the range [1,2^23).
``i<size>:<abi>:<pref>``
Expand Down Expand Up @@ -2281,7 +2283,7 @@ Fast-Math Flags
LLVM IR floating-point operations (:ref:`fadd <i_fadd>`,
:ref:`fsub <i_fsub>`, :ref:`fmul <i_fmul>`, :ref:`fdiv <i_fdiv>`,
:ref:`frem <i_frem>`, :ref:`fcmp <i_fcmp>`) and :ref:`call <i_call>`
may use the following flags to enable otherwise unsafe
may use the following flags to enable otherwise unsafe
floating-point transformations.

``nnan``
Expand All @@ -2308,11 +2310,11 @@ floating-point transformations.

``afn``
Approximate functions - Allow substitution of approximate calculations for
functions (sin, log, sqrt, etc). See floating-point intrinsic definitions
for places where this can apply to LLVM's intrinsic math functions.
functions (sin, log, sqrt, etc). See floating-point intrinsic definitions
for places where this can apply to LLVM's intrinsic math functions.

``reassoc``
Allow reassociation transformations for floating-point instructions.
Allow reassociation transformations for floating-point instructions.
This may dramatically change results in floating point.

``fast``
Expand Down Expand Up @@ -6853,10 +6855,10 @@ Both arguments must have identical types.
Semantics:
""""""""""

Return the same value as a libm '``fmod``' function but without trapping or
Return the same value as a libm '``fmod``' function but without trapping or
setting ``errno``.

The remainder has the same sign as the dividend. This instruction can also
The remainder has the same sign as the dividend. This instruction can also
take any number of :ref:`fast-math flags <fastmath>`, which are optimization
hints to enable otherwise unsafe floating-point optimizations:

Expand Down Expand Up @@ -10504,7 +10506,7 @@ Semantics:
""""""""""

The '``llvm.memset.*``' intrinsics fill "len" bytes of memory starting
at the destination location.
at the destination location.

'``llvm.sqrt.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -10538,10 +10540,10 @@ Semantics:
""""""""""

Return the same value as a corresponding libm '``sqrt``' function but without
trapping or setting ``errno``. For types specified by IEEE-754, the result
trapping or setting ``errno``. For types specified by IEEE-754, the result
matches a conforming libm implementation.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.powi.*``' Intrinsic
Expand Down Expand Up @@ -10616,7 +10618,7 @@ Semantics:
Return the same value as a corresponding libm '``sin``' function but without
trapping or setting ``errno``.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.cos.*``' Intrinsic
Expand Down Expand Up @@ -10653,7 +10655,7 @@ Semantics:
Return the same value as a corresponding libm '``cos``' function but without
trapping or setting ``errno``.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.pow.*``' Intrinsic
Expand Down Expand Up @@ -10691,7 +10693,7 @@ Semantics:
Return the same value as a corresponding libm '``pow``' function but without
trapping or setting ``errno``.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.exp.*``' Intrinsic
Expand Down Expand Up @@ -10729,7 +10731,7 @@ Semantics:
Return the same value as a corresponding libm '``exp``' function but without
trapping or setting ``errno``.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.exp2.*``' Intrinsic
Expand Down Expand Up @@ -10767,7 +10769,7 @@ Semantics:
Return the same value as a corresponding libm '``exp2``' function but without
trapping or setting ``errno``.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.log.*``' Intrinsic
Expand Down Expand Up @@ -10805,7 +10807,7 @@ Semantics:
Return the same value as a corresponding libm '``log``' function but without
trapping or setting ``errno``.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.log10.*``' Intrinsic
Expand Down Expand Up @@ -10843,7 +10845,7 @@ Semantics:
Return the same value as a corresponding libm '``log10``' function but without
trapping or setting ``errno``.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.log2.*``' Intrinsic
Expand Down Expand Up @@ -10881,7 +10883,7 @@ Semantics:
Return the same value as a corresponding libm '``log2``' function but without
trapping or setting ``errno``.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.fma.*``' Intrinsic
Expand Down Expand Up @@ -10918,7 +10920,7 @@ Semantics:
Return the same value as a corresponding libm '``fma``' function but without
trapping or setting ``errno``.

When specified with the fast-math-flag 'afn', the result may be approximated
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.

'``llvm.fabs.*``' Intrinsic
Expand Down Expand Up @@ -14558,4 +14560,3 @@ lowered to a call to the symbol ``__llvm_memset_element_unordered_atomic_*``. Wh
is replaced with an actual element size.

The optimizer is allowed to inline the memory assignment when it's profitable to do so.

4 changes: 2 additions & 2 deletions llvm/include/llvm/CodeGen/BasicTTIImpl.h
View file
Expand Up @@ -240,7 +240,7 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
bool IsJTAllowed = TLI->areJTsAllowed(SI.getParent()->getParent());

// Early exit if both a jump table and bit test are not allowed.
if (N < 1 || (!IsJTAllowed && DL.getPointerSizeInBits() < N))
if (N < 1 || (!IsJTAllowed && DL.getIndexSizeInBits(0u) < N))
return N;

APInt MaxCaseVal = SI.case_begin()->getCaseValue()->getValue();
Expand All @@ -254,7 +254,7 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
}

// Check if suitable for a bit test
if (N <= DL.getPointerSizeInBits()) {
if (N <= DL.getIndexSizeInBits(0u)) {
SmallPtrSet<const BasicBlock *, 4> Dests;
for (auto I : SI.cases())
Dests.insert(I.getCaseSuccessor());
Expand Down
2 changes: 1 addition & 1 deletion llvm/include/llvm/CodeGen/TargetLowering.h
View file
Expand Up @@ -812,7 +812,7 @@ class TargetLoweringBase {
bool rangeFitsInWord(const APInt &Low, const APInt &High,
const DataLayout &DL) const {
// FIXME: Using the pointer type doesn't seem ideal.
uint64_t BW = DL.getPointerSizeInBits();
uint64_t BW = DL.getIndexSizeInBits(0u);
uint64_t Range = (High - Low).getLimitedValue(UINT64_MAX - 1) + 1;
return Range <= BW;
}
Expand Down
24 changes: 22 additions & 2 deletions llvm/include/llvm/IR/DataLayout.h
View file
Expand Up @@ -92,10 +92,12 @@ struct PointerAlignElem {
unsigned PrefAlign;
uint32_t TypeByteWidth;
uint32_t AddressSpace;
uint32_t IndexWidth;

/// Initializer
static PointerAlignElem get(uint32_t AddressSpace, unsigned ABIAlign,
unsigned PrefAlign, uint32_t TypeByteWidth);
unsigned PrefAlign, uint32_t TypeByteWidth,
uint32_t IndexWidth);

bool operator==(const PointerAlignElem &rhs) const;
};
Expand Down Expand Up @@ -165,7 +167,8 @@ class DataLayout {
unsigned getAlignmentInfo(AlignTypeEnum align_type, uint32_t bit_width,
bool ABIAlign, Type *Ty) const;
void setPointerAlignment(uint32_t AddrSpace, unsigned ABIAlign,
unsigned PrefAlign, uint32_t TypeByteWidth);
unsigned PrefAlign, uint32_t TypeByteWidth,
uint32_t IndexWidth);

/// Internal helper method that returns requested alignment for type.
unsigned getAlignment(Type *Ty, bool abi_or_pref) const;
Expand Down Expand Up @@ -321,6 +324,9 @@ class DataLayout {
/// the backends/clients are updated.
unsigned getPointerSize(unsigned AS = 0) const;

// Index size used for address calculation.
unsigned getIndexSize(unsigned AS) const;

/// Return the address spaces containing non-integral pointers. Pointers in
/// this address space don't have a well-defined bitwise representation.
ArrayRef<unsigned> getNonIntegralAddressSpaces() const {
Expand All @@ -345,13 +351,22 @@ class DataLayout {
return getPointerSize(AS) * 8;
}

/// Size in bits of index used for address calculation in getelementptr.
unsigned getIndexSizeInBits(unsigned AS) const {
return getIndexSize(AS) * 8;
}

/// Layout pointer size, in bits, based on the type. If this function is
/// called with a pointer type, then the type size of the pointer is returned.
/// If this function is called with a vector of pointers, then the type size
/// of the pointer is returned. This should only be called with a pointer or
/// vector of pointers.
unsigned getPointerTypeSizeInBits(Type *) const;

/// Layout size of the index used in GEP calculation.
/// The function should be called with pointer or vector of pointers type.
unsigned getIndexTypeSizeInBits(Type *Ty) const;

unsigned getPointerTypeSize(Type *Ty) const {
return getPointerTypeSizeInBits(Ty) / 8;
}
Expand Down Expand Up @@ -453,6 +468,11 @@ class DataLayout {
/// are set.
unsigned getLargestLegalIntTypeSizeInBits() const;

/// \brief Returns the type of a GEP index.
/// If it was not specified explicitly, it will be the integer type of the
/// pointer width - IntPtrType.
Type *getIndexType(Type *PtrTy) const;

/// \brief Returns the offset from the beginning of the type for the specified
/// indices.
///
Expand Down
40 changes: 20 additions & 20 deletions llvm/lib/Analysis/ConstantFolding.cpp
View file
Expand Up @@ -286,7 +286,7 @@ bool llvm::IsConstantOffsetFromGlobal(Constant *C, GlobalValue *&GV,
APInt &Offset, const DataLayout &DL) {
// Trivial case, constant is the global.
if ((GV = dyn_cast<GlobalValue>(C))) {
unsigned BitWidth = DL.getPointerTypeSizeInBits(GV->getType());
unsigned BitWidth = DL.getIndexTypeSizeInBits(GV->getType());
Offset = APInt(BitWidth, 0);
return true;
}
Expand All @@ -305,7 +305,7 @@ bool llvm::IsConstantOffsetFromGlobal(Constant *C, GlobalValue *&GV,
if (!GEP)
return false;

unsigned BitWidth = DL.getPointerTypeSizeInBits(GEP->getType());
unsigned BitWidth = DL.getIndexTypeSizeInBits(GEP->getType());
APInt TmpOffset(BitWidth, 0);

// If the base isn't a global+constant, we aren't either.
Expand Down Expand Up @@ -808,26 +808,26 @@ Constant *SymbolicallyEvaluateGEP(const GEPOperator *GEP,
// If this is a constant expr gep that is effectively computing an
// "offsetof", fold it into 'cast int Size to T*' instead of 'gep 0, 0, 12'
for (unsigned i = 1, e = Ops.size(); i != e; ++i)
if (!isa<ConstantInt>(Ops[i])) {

// If this is "gep i8* Ptr, (sub 0, V)", fold this as:
// "inttoptr (sub (ptrtoint Ptr), V)"
if (Ops.size() == 2 && ResElemTy->isIntegerTy(8)) {
auto *CE = dyn_cast<ConstantExpr>(Ops[1]);
assert((!CE || CE->getType() == IntPtrTy) &&
"CastGEPIndices didn't canonicalize index types!");
if (CE && CE->getOpcode() == Instruction::Sub &&
CE->getOperand(0)->isNullValue()) {
Constant *Res = ConstantExpr::getPtrToInt(Ptr, CE->getType());
Res = ConstantExpr::getSub(Res, CE->getOperand(1));
Res = ConstantExpr::getIntToPtr(Res, ResTy);
if (auto *FoldedRes = ConstantFoldConstant(Res, DL, TLI))
Res = FoldedRes;
return Res;
if (!isa<ConstantInt>(Ops[i])) {

// If this is "gep i8* Ptr, (sub 0, V)", fold this as:
// "inttoptr (sub (ptrtoint Ptr), V)"
if (Ops.size() == 2 && ResElemTy->isIntegerTy(8)) {
auto *CE = dyn_cast<ConstantExpr>(Ops[1]);
assert((!CE || CE->getType() == IntPtrTy) &&
"CastGEPIndices didn't canonicalize index types!");
if (CE && CE->getOpcode() == Instruction::Sub &&
CE->getOperand(0)->isNullValue()) {
Constant *Res = ConstantExpr::getPtrToInt(Ptr, CE->getType());
Res = ConstantExpr::getSub(Res, CE->getOperand(1));
Res = ConstantExpr::getIntToPtr(Res, ResTy);
if (auto *FoldedRes = ConstantFoldConstant(Res, DL, TLI))
Res = FoldedRes;
return Res;
}
}
return nullptr;
}
return nullptr;
}

unsigned BitWidth = DL.getTypeSizeInBits(IntPtrTy);
APInt Offset =
Expand Down
4 changes: 2 additions & 2 deletions llvm/lib/Analysis/InlineCost.cpp
View file
Expand Up @@ -372,7 +372,7 @@ void CallAnalyzer::disableLoadElimination() {
/// Returns false if unable to compute the offset for any reason. Respects any
/// simplified values known during the analysis of this callsite.
bool CallAnalyzer::accumulateGEPOffset(GEPOperator &GEP, APInt &Offset) {
unsigned IntPtrWidth = DL.getPointerTypeSizeInBits(GEP.getType());
unsigned IntPtrWidth = DL.getIndexTypeSizeInBits(GEP.getType());
assert(IntPtrWidth == Offset.getBitWidth());

for (gep_type_iterator GTI = gep_type_begin(GEP), GTE = gep_type_end(GEP);
Expand Down Expand Up @@ -1619,7 +1619,7 @@ ConstantInt *CallAnalyzer::stripAndComputeInBoundsConstantOffsets(Value *&V) {
return nullptr;

unsigned AS = V->getType()->getPointerAddressSpace();
unsigned IntPtrWidth = DL.getPointerSizeInBits(AS);
unsigned IntPtrWidth = DL.getIndexSizeInBits(AS);
APInt Offset = APInt::getNullValue(IntPtrWidth);

// Even though we don't look through PHI nodes, we could be called on an
Expand Down
10 changes: 5 additions & 5 deletions llvm/lib/Analysis/InstructionSimplify.cpp
View file
Expand Up @@ -3762,7 +3762,7 @@ static Value *SimplifyGEPInst(Type *SrcTy, ArrayRef<Value *> Ops,
// The following transforms are only safe if the ptrtoint cast
// doesn't truncate the pointers.
if (Ops[1]->getType()->getScalarSizeInBits() ==
Q.DL.getPointerSizeInBits(AS)) {
Q.DL.getIndexSizeInBits(AS)) {
auto PtrToIntOrZero = [GEPTy](Value *P) -> Value * {
if (match(P, m_Zero()))
return Constant::getNullValue(GEPTy);
Expand Down Expand Up @@ -3802,10 +3802,10 @@ static Value *SimplifyGEPInst(Type *SrcTy, ArrayRef<Value *> Ops,
if (Q.DL.getTypeAllocSize(LastType) == 1 &&
all_of(Ops.slice(1).drop_back(1),
[](Value *Idx) { return match(Idx, m_Zero()); })) {
unsigned PtrWidth =
Q.DL.getPointerSizeInBits(Ops[0]->getType()->getPointerAddressSpace());
if (Q.DL.getTypeSizeInBits(Ops.back()->getType()) == PtrWidth) {
APInt BasePtrOffset(PtrWidth, 0);
unsigned IdxWidth =
Q.DL.getIndexSizeInBits(Ops[0]->getType()->getPointerAddressSpace());
if (Q.DL.getTypeSizeInBits(Ops.back()->getType()) == IdxWidth) {
APInt BasePtrOffset(IdxWidth, 0);
Value *StrippedBasePtr =
Ops[0]->stripAndAccumulateInBoundsConstantOffsets(Q.DL,
BasePtrOffset);
Expand Down
4 changes: 2 additions & 2 deletions llvm/lib/Analysis/Loads.cpp
View file
Expand Up @@ -80,7 +80,7 @@ static bool isDereferenceableAndAlignedPointer(
if (const GEPOperator *GEP = dyn_cast<GEPOperator>(V)) {
const Value *Base = GEP->getPointerOperand();

APInt Offset(DL.getPointerTypeSizeInBits(GEP->getType()), 0);
APInt Offset(DL.getIndexTypeSizeInBits(GEP->getType()), 0);
if (!GEP->accumulateConstantOffset(DL, Offset) || Offset.isNegative() ||
!Offset.urem(APInt(Offset.getBitWidth(), Align)).isMinValue())
return false;
Expand Down Expand Up @@ -146,7 +146,7 @@ bool llvm::isDereferenceableAndAlignedPointer(const Value *V, unsigned Align,

SmallPtrSet<const Value *, 32> Visited;
return ::isDereferenceableAndAlignedPointer(
V, Align, APInt(DL.getTypeSizeInBits(VTy), DL.getTypeStoreSize(Ty)), DL,
V, Align, APInt(DL.getIndexTypeSizeInBits(VTy), DL.getTypeStoreSize(Ty)), DL,
CtxI, DT, Visited);
}

Expand Down

0 comments on commit 945b7e5

Please sign in to comment.