Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ORC] Add absoluteSymbolsLinkGraph to expose absolute symbols to platform #77008

Merged
merged 1 commit into from
Jan 5, 2024

Conversation

benlangmuir
Copy link
Collaborator

@benlangmuir benlangmuir commented Jan 4, 2024

Adds a function to create a LinkGraph of absolute symbols, and a callback in dynamic library search generators to enable using it to expose its symbols to the platform/orc runtime. This allows e.g. using __orc_rt_run_program to run a precompiled function that was found via dlsym. Ideally we would use this in llvm-jitlink's own search generator, but it will require more work to align with the Process/Platform JITDylib split, so not handled here.

As part of this change we need to handle LinkGraphs that only have absolute symbols.

case Triple::aarch64:
case llvm::Triple::riscv64:
case Triple::x86_64:
PointerSize = 8;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not thrilled with this, but didn't see an existing method for calculating pointer size for the execution session/target.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah -- I don't think this is a blocker here.

I think the triple was added to the LinkGraph after the pointer-size / endianness pair, but now that we have the triple the pointer-size and endianness seem redundant -- is there any situation where we can't derive them from the triple? Maybe we can just get rid of them in a follow-up patch.

Copy link
Contributor

@lhames lhames left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tryToGenerate just has to add the definitions to the target MU, so what if you replaced

using AbsoluteSymbolsMUFactory =
      unique_function<std::unique_ptr<MaterializationUnit>(SymbolMap)>;

with

using AddAbsoluteSymbolsFn =
      unique_function<Error(JITDylib&, SymbolMap)>;

then you could replace absoluteSymbolsObjectLayer with a more generic absoluteSymbolsLinkGraph function that just returns the graph, and your adder could be:

[&ObjLinkingLayer](JITDylib &JD, SymbolMap Symbols) {
  // Build graph.
  return ObjLinkingLayer.add(JD, std::move(G));
}

That'd avoid exposing ObjectLinkingLayer::createLinkGraphMaterializationUnit.

case Triple::aarch64:
case llvm::Triple::riscv64:
case Triple::x86_64:
PointerSize = 8;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah -- I don't think this is a blocker here.

I think the triple was added to the LinkGraph after the pointer-size / endianness pair, but now that we have the triple the pointer-size and endianness seem redundant -- is there any situation where we can't derive them from the triple? Maybe we can just get rid of them in a follow-up patch.

@benlangmuir
Copy link
Collaborator Author

Updated per review; changing LinkGraph itself to use the triple to compute pointer width is left as a potential follow up

@benlangmuir benlangmuir changed the title [ORC] Add absoluteSymbolsObjectLayer to expose absolute symbols to platform [ORC] Add absoluteSymbolsLinkGraph to expose absolute symbols to platform Jan 5, 2024
Copy link
Contributor

@lhames lhames left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @benlangmuir!

Comment on lines 51 to 57
Ctx->getMemoryManager().allocate(
Ctx->getJITLinkDylib(), *G,
[S = std::move(Self)](AllocResult AR) mutable {
// FIXME: Once MSVC implements c++17 order of evaluation rules for calls
// this can be simplified to
// S->linkPhase2(std::move(S), std::move(AR));
auto *TmpSelf = S.get();
TmpSelf->linkPhase2(std::move(S), std::move(AR));
});
auto LinkPhase2 = [S = std::move(Self)](AllocResult AR) mutable {
// FIXME: Once MSVC implements c++17 order of evaluation rules for calls
// this can be simplified to
// S->linkPhase2(std::move(S), std::move(AR));
auto *TmpSelf = S.get();
TmpSelf->linkPhase2(std::move(S), std::move(AR));
};

if (G->allocActions().empty() && llvm::all_of(G->sections(), [](Section &S) {
return S.getMemLifetime() == orc::MemLifetime::NoAlloc;
})) {
LinkPhase2(nullptr);
} else {
Ctx->getMemoryManager().allocate(Ctx->getJITLinkDylib(), *G,
std::move(LinkPhase2));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively this could also be:

  // Skip straight to phase 2 if the graph is empty and has no associated actions.
  if (G->allocActions().empty() && llvm::all_of(G->sections(), [](Section &S) {
        return S.getMemLifetime() == orc::MemLifetime::NoAlloc;
      })) {
    linkPhase2(std::move(Self), nullptr);
    return;
  }
  
  Ctx->getMemoryManager().allocate(Ctx->getJITLinkDylib(), *G,
                                   [S = std::move(Self)](AllocResult AR) mutable {
      // FIXME: Once MSVC implements c++17 order of evaluation rules for calls
      // this can be simplified to
      //          S->linkPhase2(std::move(S), std::move(AR));
      auto *TmpSelf = S.get();
      TmpSelf->linkPhase2(std::move(S), std::move(AR));
    });

Similarly for the call to phase 4.

That reads more clearly to me, but only marginally.

…form

Adds a function to create a LinkGraph of absolute symbols, and a
callback in dynamic library search generators to enable using it to
expose its symbols to the platform/orc runtime. This allows e.g. using
__orc_rt_run_program to run a precompiled function that was found via
dlsym. Ideally we would use this in llvm-jitlink's own search generator,
but it will require more work to align with the Process/Platform
JITDylib split, so not handled here.

As part of this change we need to handle LinkGraphs that only have
absolute symbols.
@benlangmuir benlangmuir merged commit 08c5f1f into llvm:main Jan 5, 2024
4 checks passed
@benlangmuir benlangmuir deleted the absolute-symbols-object-layer branch January 5, 2024 23:32
chapuni added a commit that referenced this pull request Jan 6, 2024
justinfargnoli pushed a commit to justinfargnoli/llvm-project that referenced this pull request Jan 28, 2024
…form (llvm#77008)

Adds a function to create a LinkGraph of absolute symbols, and a
callback in dynamic library search generators to enable using it to
expose its symbols to the platform/orc runtime. This allows e.g. using
__orc_rt_run_program to run a precompiled function that was found via
dlsym. Ideally we would use this in llvm-jitlink's own search generator,
but it will require more work to align with the Process/Platform
JITDylib split, so not handled here.

As part of this change we need to handle LinkGraphs that only have
absolute symbols.
justinfargnoli pushed a commit to justinfargnoli/llvm-project that referenced this pull request Jan 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants