Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mlir] gpu-to-llvm Pass crashed with error message "Assertion failed: (!replacements.count(op) && "operation was already replaced")" #59182

Open
Colloportus0 opened this issue Nov 24, 2022 · 8 comments
Assignees

Comments

@Colloportus0
Copy link

MLIR built at commit 12ebfca
Reproduced with:
mlir-opt --gpu-to-llvm temp.mlir

temp.mlir:

module {
  func.func @func1(){
    %c1 = arith.constant 1 : index
    %alloc_6 = memref.alloc() : memref<2x3xf32>
    memref.alloca_scope  {
      %41 = memref.generic_atomic_rmw %alloc_6[%c1, %c1] : memref<2x3xf32> {
      ^bb0(%arg0: f32):
        memref.atomic_yield %arg0 : f32
      }
    }
    return
  }
}

trace:

Assertion failed: (!replacements.count(op) && "operation was already replaced"), function notifyOpReplaced, file DialectConversion.cpp, line 1404.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: mlir-opt -gpu-to-llvm temp.mlir
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  mlir-opt                 0x00000001055fff5c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 72
1  mlir-opt                 0x0000000105600478 PrintStackTraceSignalHandler(void*) + 28
2  mlir-opt                 0x00000001055fe588 llvm::sys::RunSignalHandlers() + 148
3  mlir-opt                 0x0000000105601d48 SignalHandler(int) + 252
4  libsystem_platform.dylib 0x000000019fc254c4 _sigtramp + 56
5  libsystem_pthread.dylib  0x000000019fc0dee0 pthread_kill + 288
6  libsystem_c.dylib        0x000000019fb48340 abort + 168
7  libsystem_c.dylib        0x000000019fb47754 err + 0
8  mlir-opt                 0x0000000109346380 mlir::detail::ConversionPatternRewriterImpl::notifyOpReplaced(mlir::Operation*, mlir::ValueRange) + 208
9  mlir-opt                 0x0000000109347758 mlir::ConversionPatternRewriter::eraseOp(mlir::Operation*) + 300
10 mlir-opt                 0x000000010834561c (anonymous namespace)::GenericAtomicRMWOpLowering::moveOpsRange(mlir::ValueRange, mlir::ValueRange, llvm::ilist_iterator<llvm::ilist_detail::node_options<mlir::Operation, true, false, void>, false, false>, llvm::ilist_iterator<llvm::ilist_detail::node_options<mlir::Operation, true, false, void>, false, false>, mlir::ConversionPatternRewriter&) const + 304
11 mlir-opt                 0x0000000108344630 (anonymous namespace)::GenericAtomicRMWOpLowering::matchAndRewrite(mlir::memref::GenericAtomicRMWOp, mlir::memref::GenericAtomicRMWOpAdaptor, mlir::ConversionPatternRewriter&) const + 1600
12 mlir-opt                 0x0000000108343f28 mlir::ConvertOpToLLVMPattern<mlir::memref::GenericAtomicRMWOp>::matchAndRewrite(mlir::Operation*, llvm::ArrayRef<mlir::Value>, mlir::ConversionPatternRewriter&) const + 192
13 mlir-opt                 0x0000000109348b50 mlir::ConversionPattern::matchAndRewrite(mlir::Operation*, mlir::PatternRewriter&) const + 368
14 mlir-opt                 0x000000010a06cbb8 mlir::PatternApplicator::matchAndRewrite(mlir::Operation*, mlir::PatternRewriter&, llvm::function_ref<bool (mlir::Pattern const&)>, llvm::function_ref<void (mlir::Pattern const&)>, llvm::function_ref<mlir::LogicalResult (mlir::Pattern const&)>) + 1356
15 mlir-opt                 0x0000000109369a1c (anonymous namespace)::OperationLegalizer::legalizeWithPattern(mlir::Operation*, mlir::ConversionPatternRewriter&) + 328
16 mlir-opt                 0x0000000109369170 (anonymous namespace)::OperationLegalizer::legalize(mlir::Operation*, mlir::ConversionPatternRewriter&) + 996
17 mlir-opt                 0x0000000109368760 (anonymous namespace)::OperationConverter::convert(mlir::ConversionPatternRewriter&, mlir::Operation*) + 64
18 mlir-opt                 0x000000010934cc60 (anonymous namespace)::OperationConverter::convertOperations(llvm::ArrayRef<mlir::Operation*>, llvm::function_ref<void (mlir::Diagnostic&)>) + 568
19 mlir-opt                 0x000000010934c990 mlir::applyPartialConversion(llvm::ArrayRef<mlir::Operation*>, mlir::ConversionTarget&, mlir::FrozenRewritePatternSet const&, llvm::DenseSet<mlir::Operation*, llvm::DenseMapInfo<mlir::Operation*, void>>*) + 124
20 mlir-opt                 0x000000010934ce84 mlir::applyPartialConversion(mlir::Operation*, mlir::ConversionTarget&, mlir::FrozenRewritePatternSet const&, llvm::DenseSet<mlir::Operation*, llvm::DenseMapInfo<mlir::Operation*, void>>*) + 72
21 mlir-opt                 0x00000001080bc8d4 (anonymous namespace)::GpuToLLVMConversionPass::runOnOperation() + 384
22 mlir-opt                 0x000000010920c524 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) + 512
23 mlir-opt                 0x000000010920cbf4 mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) + 364
24 mlir-opt                 0x000000010920ee38 mlir::PassManager::runPasses(mlir::Operation*, mlir::AnalysisManager) + 108
25 mlir-opt                 0x000000010920ec10 mlir::PassManager::run(mlir::Operation*) + 864
26 mlir-opt                 0x00000001091f4010 performActions(llvm::raw_ostream&, bool, bool, llvm::SourceMgr&, mlir::MLIRContext*, llvm::function_ref<mlir::LogicalResult (mlir::PassManager&)>, bool, bool) + 560
27 mlir-opt                 0x00000001091f3ba4 processBuffer(llvm::raw_ostream&, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, bool, bool, bool, bool, bool, bool, llvm::function_ref<mlir::LogicalResult (mlir::PassManager&)>, mlir::DialectRegistry&, llvm::ThreadPool*) + 496
28 mlir-opt                 0x00000001091f396c mlir::MlirOptMain(llvm::raw_ostream&, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::function_ref<mlir::LogicalResult (mlir::PassManager&)>, mlir::DialectRegistry&, bool, bool, bool, bool, bool, bool, bool)::$_0::operator()(std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&) const + 204
29 mlir-opt                 0x00000001091f3880 mlir::LogicalResult llvm::function_ref<mlir::LogicalResult (std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>::callback_fn<mlir::MlirOptMain(llvm::raw_ostream&, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::function_ref<mlir::LogicalResult (mlir::PassManager&)>, mlir::DialectRegistry&, bool, bool, bool, bool, bool, bool, bool)::$_0>(long, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&) + 80
30 mlir-opt                 0x00000001093fea44 llvm::function_ref<mlir::LogicalResult (std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>::operator()(std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&) const + 96
31 mlir-opt                 0x00000001093fe528 mlir::splitAndProcessBuffer(std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::function_ref<mlir::LogicalResult (std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>, llvm::raw_ostream&, bool, bool) + 128
32 mlir-opt                 0x00000001091f12c0 mlir::MlirOptMain(llvm::raw_ostream&, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::function_ref<mlir::LogicalResult (mlir::PassManager&)>, mlir::DialectRegistry&, bool, bool, bool, bool, bool, bool, bool) + 320
33 mlir-opt                 0x00000001091f14c8 mlir::MlirOptMain(llvm::raw_ostream&, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, mlir::PassPipelineCLParser const&, mlir::DialectRegistry&, bool, bool, bool, bool, bool, bool, bool, bool) + 296
34 mlir-opt                 0x00000001091f208c mlir::MlirOptMain(int, char**, llvm::StringRef, mlir::DialectRegistry&, bool) + 2912
35 mlir-opt                 0x0000000104e21fa0 main + 148
36 dyld                     0x0000000122ab5088 start + 516
zsh: abort      mlir-opt -gpu-to-llvm temp.mlir
@llvmbot
Copy link
Collaborator

llvmbot commented Nov 24, 2022

@llvm/issue-subscribers-mlir

@joker-eph
Copy link
Collaborator

@ThomasRaoux may know who can look into this?

@ThomasRaoux
Copy link
Contributor

This is not gpu specific, looks like a bug in the lowering of generic_atomic_rmw to llvm. This code was added by @pifon2a who is probably the best person to look at it.

@Colloportus0
Copy link
Author

still reproducible at commit e09c76f

@Colloportus0
Copy link
Author

Hi @pifon2a, I want to kindly bring this to your attention in case you forgot to look at this issue.

@pifon2a
Copy link
Contributor

pifon2a commented Mar 8, 2023

I did not even see it. Thanks for pinging me. I ll try to fix it on Friday or Monday.

@pifon2a
Copy link
Contributor

pifon2a commented Mar 13, 2023

Here is the fix for the main issue here https://reviews.llvm.org/D145901, but it won't fix the lowering completely, because cmpxchg does not accept floats. @ftynse, is there any workaround to make it work? I found this related discussion https://lists.llvm.org/pipermail/llvm-dev/2020-August/144314.html

@ftynse
Copy link
Member

ftynse commented Mar 28, 2023

Can't the lowering go to llvm.atomic_rmw?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants