[compiler-rt][ctx_profile] Add the instrumented contextual profiling APIs #89838

mtrofin · 2024-04-23T21:58:18Z

APIs for contextual profiling. ContextNode is the call context-specific counter buffer. ContextRoot is associated to those functions that constitute roots into interesting call graphs, and is the object on which we hang off Arenas for allocating ContextNodes, as well as the ContextNode corresponding to such functions. Graphs of ContextNodes are accessible by one thread at a time.

(Tracking Issue: #89287, more details in the RFC referenced there)

…APIs APIs for contextual profiling. (Tracking Issue: llvm#89287, RFC referenced there)

github-actions · 2024-04-30T23:48:24Z

✅ With the latest revision this PR passed the C/C++ code formatter.

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

teresajohnson · 2024-05-06T02:44:16Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+  }
+
+  uint64_t *counters() {
+    ContextNode *addr_after = &(this[1]);


Can you add a comment here and in the subContexts method and maybe in the class header about the expected layout of the counters and subContexts, since they aren't data members?

teresajohnson · 2024-05-06T03:06:05Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+};
+
+/// ContextRoots are allocated by LLVM for entrypoints. LLVM is only concerned
+/// with allocating and zero-initializing the global value for it.


What do you mean by "the global value for it"?

as in GlobalValue. fixed.

teresajohnson · 2024-05-06T03:08:49Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+//
+/// TLS where LLVM stores the pointer of the called value, as part of lowering a
+/// llvm.instrprof.callsite
+extern __thread void *volatile __llvm_ctx_profile_expected_callee[2];


Why are there 2 values for the expected callee? Ditto for the __llvm_ctx_profile_callsite below.

It says in the first comment under extern "C" - position 1 is for scratch. It's so we don't pollute the values for normal collection when signal handlers happen. Normally we'd implement a stack - but we don't need to because we don't care about anything that happens when we go to sctach.

I only see index [0] being accessed for both __llvm_ctx_profile_expected_callee and __llvm_ctx_profile_callsite, other than in the tests. When does index [1] get used?

on the llvm side. See PR #90821. The runtime never looks at index 1, but having the 2 indices simplifies the lowering logic: LLVM determines the index upon getting the value from here (so in the function entry basic block) and then just writes at that index for each callsite without needing to test every time if it's scratch.

teresajohnson · 2024-05-06T03:21:18Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+/// Used to obtain the profile. The Writer is called for each root ContextNode,
+/// with the ContextRoot::Taken taken. The Writer is responsible for traversing
+/// the structure underneath.
+/// The Writer's first parameter is an object it knows about, which is what the


Not sure what you mean by "an object it knows about"?

Oh, it's the closure... rephrased.

teresajohnson · 2024-05-06T15:37:41Z

compiler-rt/lib/ctx_profile/tests/CtxInstrProfilingTest.cpp

+TEST_F(ContextTest, ScratchNoCollection) {
+  EXPECT_EQ(__llvm_ctx_profile_current_context_root, nullptr);
+  int OpaqueValue = 0;
+  // this would be the very first function executing this. the TLS is empty,


Can you clarify - I assume it is scratch because start context hasn't been called?

yes. clarified.

teresajohnson · 2024-05-06T15:38:01Z

compiler-rt/lib/ctx_profile/tests/CtxInstrProfilingTest.cpp

+  __llvm_ctx_profile_expected_callee[0] = &OpaqueValue;
+  __llvm_ctx_profile_callsite[0] = &Ctx->subContexts()[2];
+  auto *Subctx = __llvm_ctx_profile_get_context(&OtherOpaqueValue, 2, 3, 1);
+  EXPECT_TRUE(isScratch(Subctx));


Comment about why

I guess because you passed a different pointer to get_context, but it would be good to add a comment.

teresajohnson · 2024-05-06T15:40:51Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+/// called for any other function than entry points, in the entry BB of such
+/// function. Same consideration about LSB of returned value as .._start_context
+__ctx_profile::ContextNode *
+__llvm_ctx_profile_get_context(void *Callee, __ctx_profile::GUID Guid,


What type is Callee in practice?

a function address. So void * is appropriate.

teresajohnson · 2024-05-06T15:41:54Z

compiler-rt/lib/ctx_profile/tests/CtxInstrProfilingTest.cpp

+  int OpaqueValue = 0;
+  const bool IsScratch = isScratch(Ctx);
+  EXPECT_FALSE(IsScratch);
+  __llvm_ctx_profile_expected_callee[0] = &OpaqueValue;


The OpaqueValues in these tests are stand ins for callee values?

yes. should I rename - maybe "FakeCallee"?

Yeah FakeCallee would be more self explanatory

teresajohnson · 2024-05-06T15:43:13Z

compiler-rt/lib/ctx_profile/tests/CtxInstrProfilingTest.cpp

+  __llvm_ctx_profile_callsite[1] = &Subctx->subContexts()[0];
+
+  auto *Subctx2 = __llvm_ctx_profile_get_context(&ThirdOpaqueValue, 3, 0, 0);
+  EXPECT_TRUE(isScratch(Subctx2));


Ditto about adding comment

snehasish · 2024-05-06T22:22:28Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+///
+/// The layout is as follows:
+///
+/// [[statically-declared fields][counters vector][subcontexts vector]]


I think interpreting a subcontext ContextNode as part of the same block of memory immediately following counters of width uint64_t is undefined behaviour. In this case I think we get lucky because the first element of the ContextNode struct is a GUID (uint64_t) and thus the alignment of the ContextNode struct and the alignment of each individual counter matches.

I am far from an expert on undefined behaviour and my understanding is based on reading (5) in https://en.cppreference.com/w/cpp/language/reinterpret_cast and https://en.cppreference.com/w/c/language/object. Please feel free to correct me if you think otherwise.

I think the motivation is that the single chunk based layout simplifies the instrumentation GEP computations. A few questions:

Can we test the ContextNode runtime implementation with sanitizers to expose any alignment issues?

Have you considered an alternative design which does not make it easy to run into such issues?

I don't follow. The subcontext vector is of pointers to subcontexts. Maybe I should clarify that.

That being said - the rest isn't by accident, but we can statically check that the alignment checks out (added). Basically:

the Arenas are 8-aligned because that's how __sanitizer::InternalAlloc returns by default, but added the explicit flag for clarity

the memory we start bump-allocating in the Arena is 8-aligned by construction - added a static_assert for maintainability

the choice of types for ContextNode's header is explicit (to maintain the invariant) - and added a static assert.

Okay, then this does not apply since we are reinterpreting 64-bit values as pointers. Yes, enhancing the documentation to clarify would help. I still think it's a good idea to run (some of) the unit tests through ubsan.

snehasish · 2024-05-06T22:23:13Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+  // The counters vector starts right after the static header.
+  uint64_t *counters() {
+    ContextNode *addr_after = &(this[1]);
+    return reinterpret_cast<uint64_t *>(reinterpret_cast<char *>(addr_after));


Why is the additional reinterpret_cast to char* required?

It's not. Fixed.

snehasish · 2024-05-06T22:46:31Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.cpp

+
+// utility to taint a pointer by setting the LSB. There is an assumption
+// throughout that the addresses of contexts are even (really, they should be
+// align(8), but "even"-ness is the minimum assumption)


I think the align(8) nature is load bearing, see my other comment about reinterpreting the address immediately following the counters array.

Yes, and __sanitizer::InternalAlloc align-8s if you pass 0 alignment (the default). We can force that though, for clarity.

snehasish · 2024-05-06T23:31:30Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+///
+/// The layout is as follows:
+///
+/// [[statically-declared fields][counters vector][subcontexts vector]]


Okay, then this does not apply since we are reinterpreting 64-bit values as pointers. Yes, enhancing the documentation to clarify would help. I still think it's a good idea to run (some of) the unit tests through ubsan.

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

snehasish · 2024-05-06T23:33:43Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.cpp

+  for (uint32_t I = 0; I < NrCounters; ++I)
+    counters()[I] = 0;


Maybe just use memset since you know the begin and end?

__sanitizer::internal_memset is the best we can do. I'm not super-sure we really want that though (would be happy with std::memset) - because see FIXME here about removing more of the deps to sanitizer_common - not that this would be too hard to fix; but also don't we have a pass in LLVM that would effectivelly recognize this as memset and replace with the intrinsic? I.e. may not achieve much.

Left a FIXME to do std::memset though.

Sounds good. I don't think the compiler will be able to reason through the counters() implementation to be able to do this automatically though.

teresajohnson

lgtm but wait for snehasish to approve as well

snehasish

lgtm with some minor comments.

snehasish · 2024-05-07T16:03:40Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.cpp

+  for (uint32_t I = 0; I < NrCounters; ++I)
+    counters()[I] = 0;


Sounds good. I don't think the compiler will be able to reason through the counters() implementation to be able to do this automatically though.

snehasish · 2024-05-07T16:25:29Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+
+// Verify maintenance to ContextNode doesn't change this invariant, which makes
+// sure the inlined vectors are appropriately aligned.
+static_assert(sizeof(ContextNode) % Alignment == 0);


This should be alignof(ContextNode). See the example in https://en.cppreference.com/w/c/language/object for structures.

good point, fixed here and in a few more places.

snehasish · 2024-05-07T16:29:44Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+  // Note that this is node-by-node aggregation, i.e. summing counters of nodes
+  // at the same position in the graph, not flattening.
+  // Threads that cannot lock Taken (fail TryLock) are given a "scratch context"
+  // - a buffer they can clobber, safely from a memory access perspective.


So then the data written to the scratch context is just dropped?

Yes - and expanded in the comment.

The design allows for that not be the case - because "scratch"-ness is first and foremost about not trying to build subcontexts, and is captured by tainting the pointer value (pointer to the memory treated as context), but right now, we drop that info.

snehasish · 2024-05-07T16:31:52Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.h

+  // regardless.
+  ::__sanitizer::StaticSpinMutex Taken;
+
+  // Avoid surprises due to (unlikely) StaticSpinMutex changes.


A more descriptive message of why we care about the size here is preferred.

snehasish · 2024-05-07T17:07:12Z

compiler-rt/lib/ctx_profile/tests/CtxInstrProfilingTest.cpp

+  EXPECT_NE(Ctx, nullptr);
+  EXPECT_NE(Root.CurrentMem, nullptr);
+  EXPECT_EQ(Root.FirstMemBlock, Root.CurrentMem);
+  EXPECT_EQ(Ctx->size(), sizeof(ContextNode) + 10 * sizeof(uint64_t) +


Ctx->size() will segfault even if the expectation on L35 fails since that does not fail the test. L35 should be an ASSERT.

snehasish · 2024-05-07T17:14:05Z

compiler-rt/lib/ctx_profile/CtxInstrProfiling.cpp

+    auto *Root = AllContextRoots[I];
+    __sanitizer::GenericScopedLock<__sanitizer::StaticSpinMutex> TakenLock(
+        &Root->Taken);
+    if (!validate(Root)) {


Do you want to validate on all builds? It might be problematic for server apps which expect profile dumping to be quick.

It's quick right now, we can relax later, but this wasn't a problem so far.

Follow-up from PR ##89838. Some build bots warn-as-error about signed/unsigned comparison in CtxInstrProfilingTest. Example: https://lab.llvm.org/buildbot/#/builders/37/builds/34610

[compiler-rt][ctx_profile] Add the instrumented contextual profiling …

75161d4

…APIs APIs for contextual profiling. (Tracking Issue: llvm#89287, RFC referenced there)

mtrofin requested review from snehasish, teresajohnson, david-xl and xur-llvm April 23, 2024 21:58

llvmbot added the compiler-rt label Apr 23, 2024

more comments

50449f5

formatting

de7676c

mtrofin mentioned this pull request May 2, 2024

[llvm][ctx_profile] Add instrumentation lowering #90821

Merged

teresajohnson reviewed May 6, 2024

View reviewed changes

more comments / addressed feedback

48afbe3

snehasish reviewed May 6, 2024

View reviewed changes

mtrofin added 3 commits May 6, 2024 16:44

more comments, asserts about alignment

658facc

about memset

1c07880

typo

509ee74

teresajohnson approved these changes May 7, 2024

View reviewed changes

snehasish approved these changes May 7, 2024

View reviewed changes

mtrofin added 3 commits May 7, 2024 11:22

feedback

5f95dc2

fixes

f063427

Merge branch 'main' into context_node

913d62c

mtrofin merged commit ccf765c into llvm:main May 7, 2024
4 checks passed

[compiler-rt][ctx_profile] Add the instrumented contextual profiling APIs #89838

[compiler-rt][ctx_profile] Add the instrumented contextual profiling APIs #89838

Conversation

mtrofin commented Apr 23, 2024

github-actions bot commented Apr 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

teresajohnson left a comment

Choose a reason for hiding this comment

snehasish left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtrofin May 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Apr 30, 2024 •

edited

Loading

mtrofin May 7, 2024 •

edited

Loading