Skip to content

[AMDGPU] Enable "amdgpu-uniform-intrinsic-combine" pass in pipeline. #162819

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
+1,065 −1,977
Open

[AMDGPU] Enable "amdgpu-uniform-intrinsic-combine" pass in pipeline. #162819

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9259485
[pre-commit] Update the test check affected after adding pass to llc
PankajDwivedi-25 Oct 10, 2025
d92fd81
added OPM support and added pass into llc pipeline
PankajDwivedi-25 Oct 10, 2025
920ffc4
skip optimizing function has optnone attribute
PankajDwivedi-25 Oct 13, 2025
388b593
address review
PankajDwivedi-25 Oct 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions llvm/lib/Target/AMDGPU/AMDGPU.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -562,6 +562,10 @@ class AMDGPURewriteAGPRCopyMFMAPass
void initializeAMDGPURewriteAGPRCopyMFMALegacyPass(PassRegistry &);
extern char &AMDGPURewriteAGPRCopyMFMALegacyID;

void initializeAMDGPUUniformIntrinsicCombineLegacyPass(PassRegistry &);
extern char &AMDGPUUniformIntrinsicCombineLegacyPassID;
ModulePass *createAMDGPUUniformIntrinsicCombineLegacyPass();

struct AMDGPUUniformIntrinsicCombinePass
: public PassInfoMixin<AMDGPUUniformIntrinsicCombinePass> {
PreservedAnalyses run(Module &M, ModuleAnalysisManager &AM);
Expand Down
4 changes: 4 additions & 0 deletions llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -618,6 +618,7 @@ extern "C" LLVM_ABI LLVM_EXTERNAL_VISIBILITY void LLVMInitializeAMDGPUTarget() {
initializeAMDGPUPreloadKernArgPrologLegacyPass(*PR);
initializeAMDGPUWaitSGPRHazardsLegacyPass(*PR);
initializeAMDGPUPreloadKernelArgumentsLegacyPass(*PR);
initializeAMDGPUUniformIntrinsicCombineLegacyPass(*PR);
}

static std::unique_ptr<TargetLoweringObjectFile> createTLOF(const Triple &TT) {
Expand Down Expand Up @@ -1310,6 +1311,9 @@ void AMDGPUPassConfig::addIRPasses() {
isPassEnabled(EnableImageIntrinsicOptimizer))
addPass(createAMDGPUImageIntrinsicOptimizerPass(&TM));

if (EnableUniformIntrinsicCombine)
addPass(createAMDGPUUniformIntrinsicCombineLegacyPass());

// This can be disabled by passing ::Disable here or on the command line
// with --expand-variadics-override=disable.
addPass(createExpandVariadicsPass(ExpandVariadicsMode::Lowering));
Expand Down
71 changes: 71 additions & 0 deletions llvm/lib/Target/AMDGPU/AMDGPUUniformIntrinsicCombine.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,13 +141,73 @@ static bool runUniformIntrinsicCombine(Module &M, ModuleAnalysisManager &AM) {
for (User *U : make_early_inc_range(F.users())) {
auto *II = cast<IntrinsicInst>(U);
Function *ParentF = II->getFunction();
if (ParentF->hasFnAttribute(Attribute::OptimizeNone))
continue;
const auto &UI = FAM.getResult<UniformityInfoAnalysis>(*ParentF);
IsChanged |= optimizeUniformIntrinsic(*II, UI, Tracker);
}
}
return IsChanged;
}

// Legacy PM version
static bool runUniformIntrinsicCombine(Module &M, ModulePass &P) {
bool IsChanged = false;
ValueMap<const Value *, bool> Tracker;

for (Function &F : M) {
switch (F.getIntrinsicID()) {
case Intrinsic::amdgcn_permlane64:
case Intrinsic::amdgcn_readfirstlane:
case Intrinsic::amdgcn_readlane:
case Intrinsic::amdgcn_ballot:
break;
default:
continue;
}

for (User *U : make_early_inc_range(F.users())) {
auto *II = cast<IntrinsicInst>(U);
Function *ParentF = II->getFunction();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still running on optnone functions, another issue with doing this as a module pass

Copy link
Contributor Author

@PankajDwivedi-25 PankajDwivedi-25 Oct 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a simple check over ParentF->hasFnAttribute(Attribute::OptimizeNone) should be sufficient here?

if (ParentF->hasFnAttribute(Attribute::OptimizeNone))
continue;
auto &UI = P.getAnalysis<UniformityInfoWrapperPass>(*ParentF)
.getUniformityInfo();
IsChanged |= optimizeUniformIntrinsic(*II, UI, Tracker);
}
}
return IsChanged;
}

namespace {
class AMDGPUUniformIntrinsicCombineLegacy : public ModulePass {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This belongs with the base support PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its ok to have separately. Initially, it was part of the base PR. after reviews I dropped it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should have gone there, and the IR test needs both new and old PM run lines

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OPM is really not required as such, as both the pipeline OPT supports NPM by default, and very soon backend will also be default NPM. This is just for temporary until we do not have NPM support in LLC.

public:
static char ID;
AMDGPUUniformIntrinsicCombineLegacy() : ModulePass(ID) {
initializeAMDGPUUniformIntrinsicCombineLegacyPass(
*PassRegistry::getPassRegistry());
}

private:
bool runOnModule(Module &M) override;
void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.setPreservesCFG();
AU.addRequired<UniformityInfoWrapperPass>();
AU.addRequired<TargetPassConfig>();
}
};
} // namespace

char AMDGPUUniformIntrinsicCombineLegacy::ID = 0;
char &llvm::AMDGPUUniformIntrinsicCombineLegacyPassID =
AMDGPUUniformIntrinsicCombineLegacy::ID;

bool AMDGPUUniformIntrinsicCombineLegacy::runOnModule(Module &M) {
if (skipModule(M))
return false;
return runUniformIntrinsicCombine(M, *this);
}

PreservedAnalyses
AMDGPUUniformIntrinsicCombinePass::run(Module &M, ModuleAnalysisManager &AM) {
if (!runUniformIntrinsicCombine(M, AM))
Expand All @@ -157,3 +217,14 @@ AMDGPUUniformIntrinsicCombinePass::run(Module &M, ModuleAnalysisManager &AM) {
PA.preserve<UniformityInfoAnalysis>();
return PA;
}

INITIALIZE_PASS_BEGIN(AMDGPUUniformIntrinsicCombineLegacy, DEBUG_TYPE,
"AMDGPU Uniform Intrinsic Combine", false, false)
INITIALIZE_PASS_DEPENDENCY(UniformityInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(TargetPassConfig)
INITIALIZE_PASS_END(AMDGPUUniformIntrinsicCombineLegacy, DEBUG_TYPE,
"AMDGPU Uniform Intrinsic Combine", false, false)

ModulePass *llvm::createAMDGPUUniformIntrinsicCombineLegacyPass() {
return new AMDGPUUniformIntrinsicCombineLegacy();
}
Loading
Loading