Skip to content

Walk compressed IL -> Native map for EventTrace and stacktrace symbolication scenarios #116031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
a1a9fd8
First pass at removing use of the debugger copy of the IL to native map
davidwrighton May 27, 2025
00c702a
Get the helper to work correctly
davidwrighton May 27, 2025
2b2e88a
More fixes
davidwrighton May 27, 2025
90b15ae
Remove accidental character added to PR
davidwrighton May 27, 2025
1e03ed6
- Make the NibbleReader a bit faster by loading data in pointer sized…
davidwrighton May 29, 2025
f13394d
Missed change to COOPERATIVE mode in contract
davidwrighton May 29, 2025
fe8a4e5
Fix epilog handling
davidwrighton May 29, 2025
1acbbd1
Add assertions around correctness of offset data
davidwrighton Jun 16, 2025
c682847
Merge branch 'main' of https://github.com/dotnet/runtime into UseWalk…
davidwrighton Jun 16, 2025
0a38876
Fix IL offset walk for epilog discovery
davidwrighton Jun 17, 2025
be19771
Get valid result details more correct.
davidwrighton Jun 17, 2025
c329fa8
CALL_INSTRUCTION mapping data affects the greatest IL offset found ca…
davidwrighton Jun 18, 2025
2475c19
Faster nibble reading in checked builds
davidwrighton Jun 18, 2025
531b4f0
Faster underlying compressed format
davidwrighton Jun 20, 2025
6c0427a
Fix Unix build breaks
davidwrighton Jun 20, 2025
375a4fc
Fix Unix build break
davidwrighton Jun 26, 2025
575e762
Fix assertion failures due to missing case in VarLoc comparison routine
davidwrighton Jun 26, 2025
6bb34be
Enable R2R to have the new compressed format
davidwrighton Jun 27, 2025
0b98cbc
Update ILOffsets format for R2R at the reader side
davidwrighton Jun 27, 2025
7c9a558
Merge branch 'main' of https://github.com/dotnet/runtime into UseWalk…
davidwrighton Jul 3, 2025
2aca1f4
Fix merge issue
davidwrighton Jul 3, 2025
e5529fa
Fix errors noted in testing
davidwrighton Jul 4, 2025
830ef11
- Fix interpreter scenarios
davidwrighton Jul 7, 2025
f8316bd
Handle Instrumented IL mappings
davidwrighton Jul 8, 2025
ee28f16
Fix some more issues
davidwrighton Jul 8, 2025
749d1d1
Self-feedback on the PR
davidwrighton Jul 8, 2025
bf58582
Reduce down to only 1 implementation of bounds processing through usi…
davidwrighton Jul 8, 2025
32f085c
Fix comparison failures found in GCC build leg
davidwrighton Jul 8, 2025
18d91df
Fix wasm build
davidwrighton Jul 8, 2025
0d2d790
Fix comparison failures by always processing all the data even though…
davidwrighton Jul 8, 2025
b6907ff
Feedback
davidwrighton Jul 9, 2025
e7cdcbd
Merge branch 'UseWalkILOffsetInsteadOfDebuggerData' of https://github…
davidwrighton Jul 9, 2025
f7efd5c
Merge branch 'main' into UseWalkILOffsetInsteadOfDebuggerData
davidwrighton Jul 9, 2025
d8fb3fa
Merge branch 'main' of https://github.com/dotnet/runtime into UseWalk…
davidwrighton Jul 10, 2025
bb02c93
Fixup changes from merge
davidwrighton Jul 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/coreclr/debug/daccess/daccess.cpp
Original file line number Diff line number Diff line change
@@ -5990,6 +5990,7 @@ ClrDataAccess::GetMethodVarInfo(MethodDesc* methodDesc,
BOOL success = DebugInfoManager::GetBoundariesAndVars(
request,
DebugInfoStoreNew, NULL, // allocator
BoundsType::Instrumented,
NULL, NULL,
&countNativeVarInfo, &nativeVars);

@@ -6052,6 +6053,7 @@ ClrDataAccess::GetMethodNativeMap(MethodDesc* methodDesc,
BOOL success = DebugInfoManager::GetBoundariesAndVars(
request,
DebugInfoStoreNew, NULL, // allocator
BoundsType::Instrumented,
&countMapCopy, &mapCopy,
NULL, NULL);

141 changes: 4 additions & 137 deletions src/coreclr/debug/daccess/dacdbiimpl.cpp
Original file line number Diff line number Diff line change
@@ -868,6 +868,7 @@ void DacDbiInterfaceImpl::GetNativeVarData(MethodDesc * pMethodDesc,

BOOL success = DebugInfoManager::GetBoundariesAndVars(request,
InfoStoreNew, NULL, // allocator
BoundsType::Instrumented,
NULL, NULL,
&entryCount, &nativeVars);

@@ -879,76 +880,6 @@ void DacDbiInterfaceImpl::GetNativeVarData(MethodDesc * pMethodDesc,
} // GetNativeVarData


//-----------------------------------------------------------------------------
// Given a instrumented IL map from the profiler that maps:
// Original offset IL_A -> Instrumentend offset IL_B
// And a native mapping from the JIT that maps:
// Instrumented offset IL_B -> native offset Native_C
// This function merges the two maps and stores the result back into the nativeMap.
// The nativeMap now maps:
// Original offset IL_A -> native offset Native_C
// pEntryCount is the number of valid entries in nativeMap, and it may be adjusted downwards
// as part of the composition.
//-----------------------------------------------------------------------------
void DacDbiInterfaceImpl::ComposeMapping(const InstrumentedILOffsetMapping * pProfilerILMap, ICorDebugInfo::OffsetMapping nativeMap[], ULONG32* pEntryCount)
{
// Translate the IL offset if the profiler has provided us with a mapping.
// The ICD public API should always expose the original IL offsets, but GetBoundaries()
// directly accesses the debug info, which stores the instrumented IL offsets.

ULONG32 entryCount = *pEntryCount;
// The map pointer could be NULL or there could be no entries in the map, in either case no work to do
if (pProfilerILMap && !pProfilerILMap->IsNull())
{
// If we did instrument, then we can't have any sequence points that
// are "in-between" the old-->new map that the profiler gave us.
// Ex, if map is:
// (6 old -> 36 new)
// (8 old -> 50 new)
// And the jit gives us an entry for 44 new, that will map back to 6 old.
// Since the map can only have one entry for 6 old, we remove 44 new.

// First Pass: invalidate all the duplicate entries by setting their IL offset to MAX_ILNUM
ULONG32 cDuplicate = 0;
ULONG32 prevILOffset = (ULONG32)(ICorDebugInfo::MAX_ILNUM);
for (ULONG32 i = 0; i < entryCount; i++)
{
ULONG32 origILOffset = TranslateInstrumentedILOffsetToOriginal(nativeMap[i].ilOffset, pProfilerILMap);

if (origILOffset == prevILOffset)
{
// mark this sequence point as invalid; refer to the comment above
nativeMap[i].ilOffset = (ULONG32)(ICorDebugInfo::MAX_ILNUM);
cDuplicate += 1;
}
else
{
// overwrite the instrumented IL offset with the original IL offset
nativeMap[i].ilOffset = origILOffset;
prevILOffset = origILOffset;
}
}

// Second Pass: move all the valid entries up front
ULONG32 realIndex = 0;
for (ULONG32 curIndex = 0; curIndex < entryCount; curIndex++)
{
if (nativeMap[curIndex].ilOffset != (ULONG32)(ICorDebugInfo::MAX_ILNUM))
{
// This is a valid entry. Move it up front.
nativeMap[realIndex] = nativeMap[curIndex];
realIndex += 1;
}
}

// make sure we have done the bookkeeping correctly
_ASSERTE((realIndex + cDuplicate) == entryCount);

// Final Pass: derecement entryCount
entryCount -= cDuplicate;
*pEntryCount = entryCount;
}
}


//-----------------------------------------------------------------------------
@@ -983,38 +914,14 @@ void DacDbiInterfaceImpl::GetSequencePoints(MethodDesc * pMethodDesc,

ULONG32 entryCount;
BOOL success = DebugInfoManager::GetBoundariesAndVars(request,
InfoStoreNew, NULL, // allocator
InfoStoreNew,
NULL, // allocator
BoundsType::Uninstrumented,
&entryCount, &mapCopy,
NULL, NULL);
if (!success)
ThrowHR(E_FAIL);

#ifdef FEATURE_REJIT
CodeVersionManager * pCodeVersionManager = pMethodDesc->GetCodeVersionManager();
ILCodeVersion ilVersion;
NativeCodeVersion nativeCodeVersion = pCodeVersionManager->GetNativeCodeVersion(dac_cast<PTR_MethodDesc>(pMethodDesc), (PCODE)startAddr);
if (!nativeCodeVersion.IsNull())
{
ilVersion = nativeCodeVersion.GetILCodeVersion();
}

// if there is a rejit IL map for this function, apply that in preference to load-time mapping
if (!ilVersion.IsNull() && !ilVersion.IsDefaultVersion())
{
const InstrumentedILOffsetMapping * pRejitMapping = ilVersion.GetInstrumentedILMap();
ComposeMapping(pRejitMapping, mapCopy, &entryCount);
}
else
{
#endif
// if there is a profiler load-time mapping and not a rejit mapping, apply that instead
InstrumentedILOffsetMapping loadTimeMapping =
pMethodDesc->GetAssembly()->GetModule()->GetInstrumentedILOffsetMapping(pMethodDesc->GetMemberDef());
ComposeMapping(&loadTimeMapping, mapCopy, &entryCount);
#ifdef FEATURE_REJIT
}
#endif

pSeqPoints->InitSequencePoints(entryCount);

// mapCopy and pSeqPoints have elements of different types. Thus, we
@@ -1024,46 +931,6 @@ void DacDbiInterfaceImpl::GetSequencePoints(MethodDesc * pMethodDesc,

} // GetSequencePoints

// ----------------------------------------------------------------------------
// DacDbiInterfaceImpl::TranslateInstrumentedILOffsetToOriginal
//
// Description:
// Helper function to convert an instrumented IL offset to the corresponding original IL offset.
//
// Arguments:
// * ilOffset - offset to be translated
// * pMapping - the profiler-provided mapping between original IL offsets and instrumented IL offsets
//
// Return Value:
// Return the translated offset.
//

ULONG DacDbiInterfaceImpl::TranslateInstrumentedILOffsetToOriginal(ULONG ilOffset,
const InstrumentedILOffsetMapping * pMapping)
{
SIZE_T cMap = pMapping->GetCount();
ARRAY_PTR_COR_IL_MAP rgMap = pMapping->GetOffsets();

_ASSERTE((cMap == 0) == (rgMap == NULL));

// Early out if there is no mapping, or if we are dealing with a special IL offset such as
// prolog, epilog, etc.
if ((cMap == 0) || ((int)ilOffset < 0))
{
return ilOffset;
}

SIZE_T i = 0;
for (i = 1; i < cMap; i++)
{
if (ilOffset < rgMap[i].newOffset)
{
return rgMap[i - 1].oldOffset;
}
}
return rgMap[i - 1].oldOffset;
}

//- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
// Function Data
//- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
7 changes: 0 additions & 7 deletions src/coreclr/debug/daccess/dacdbiimpl.h
Original file line number Diff line number Diff line change
@@ -183,13 +183,6 @@ class DacDbiInterfaceImpl :
CORDB_ADDRESS startAddr,
SequencePoints * pNativeMap);

// Helper to compose a IL->IL and IL->Native mapping
void ComposeMapping(const InstrumentedILOffsetMapping * pProfilerILMap, ICorDebugInfo::OffsetMapping nativeMap[], ULONG32* pEntryCount);

// Helper function to convert an instrumented IL offset to the corresponding original IL offset.
ULONG TranslateInstrumentedILOffsetToOriginal(ULONG ilOffset,
const InstrumentedILOffsetMapping * pMapping);

public:
//----------------------------------------------------------------------------------
// class MapSortILMap: A template class that will sort an array of DebuggerILToNativeMap.
13 changes: 8 additions & 5 deletions src/coreclr/debug/ee/debugger.cpp
Original file line number Diff line number Diff line change
@@ -2508,7 +2508,10 @@ void Debugger::JITComplete(NativeCodeVersion nativeCodeVersion, TADDR newAddress
// Can be called on managed thread only
// This API Implements DebugInterface

#ifndef DEBUG // We need to do this in debug builds so that the ValidateILOffsets method can be called
// TODO: We may wish to remove this in the future.
if (CORDebuggerAttached())
#endif
{
// Populate the debugger's cache of DJIs. Normally we can do this lazily,
// the only reason we do it here is b/c the MethodDesc is not yet officially marked as "jitted",
@@ -2876,10 +2879,11 @@ HRESULT Debugger::GetILToNativeMapping(PCODE pNativeCodeStartAddress, ULONG32 cM
{
DebugInfoRequest diq;
diq.InitFromStartingAddr(fd, pNativeCodeStartAddress);
// TODO This is currently returning the instrumented boundaries, but the path for non-dynamic methods returns uninstrumented ones.

if (cMap == 0)
{
if (DebugInfoManager::GetBoundariesAndVars(diq, nullptr, nullptr, pcMap, nullptr, nullptr, nullptr))
if (DebugInfoManager::GetBoundariesAndVars(diq, nullptr, nullptr, BoundsType::Instrumented, pcMap, nullptr, nullptr, nullptr))
{
return S_OK;
}
@@ -2888,7 +2892,7 @@ HRESULT Debugger::GetILToNativeMapping(PCODE pNativeCodeStartAddress, ULONG32 cM
}

ICorDebugInfo::OffsetMapping* pMap = nullptr;
if (DebugInfoManager::GetBoundariesAndVars(diq, InteropSafeNoThrowNew, nullptr, pcMap, &pMap, nullptr, nullptr))
if (DebugInfoManager::GetBoundariesAndVars(diq, InteropSafeNoThrowNew, nullptr, BoundsType::Instrumented, pcMap, &pMap, nullptr, nullptr))
{
for (ULONG32 i = 0; i < cMap; ++i)
{
@@ -2992,7 +2996,7 @@ HRESULT Debugger::GetILToNativeMapping(PCODE pNativeCodeStartAddress, ULONG32 cM
// events for the same MethodDesc (each time it's EnC'd), with each event
// corresponding to the most recent EnC version at the time.
//

#ifdef DEBUG
HRESULT Debugger::GetILToNativeMappingIntoArrays(
MethodDesc * pMethodDesc,
PCODE pNativeCodeStartAddress,
@@ -3063,8 +3067,7 @@ HRESULT Debugger::GetILToNativeMappingIntoArrays(

return S_OK;
}


#endif // DEBUG


#endif // #ifndef DACCESS_COMPILE
2 changes: 2 additions & 0 deletions src/coreclr/debug/ee/debugger.h
Original file line number Diff line number Diff line change
@@ -2105,13 +2105,15 @@ class Debugger : public DebugInterface
HRESULT GetILToNativeMapping(PCODE pNativeCodeStartAddress, ULONG32 cMap, ULONG32 *pcMap,
COR_DEBUG_IL_TO_NATIVE_MAP map[]);

#ifdef DEBUG
HRESULT GetILToNativeMappingIntoArrays(
MethodDesc * pMethodDesc,
PCODE pNativeCodeStartAddress,
USHORT cMapMax,
USHORT * pcMap,
UINT ** prguiILOffset,
UINT ** prguiNativeOffset);
#endif // DEBUG

PRD_TYPE GetPatchedOpcode(CORDB_ADDRESS_TYPE *ip);
BOOL CheckGetPatchedOpcode(CORDB_ADDRESS_TYPE *address, /*OUT*/ PRD_TYPE *pOpcode);
7 changes: 7 additions & 0 deletions src/coreclr/debug/ee/functioninfo.cpp
Original file line number Diff line number Diff line change
@@ -920,6 +920,7 @@ void DebuggerJitInfo::LazyInitBounds()
BOOL fSuccess = DebugInfoManager::GetBoundariesAndVars(
request,
InteropSafeNew, NULL, // allocator
BoundsType::Instrumented, // TODO We currently don't use the uninstrumented bounds here but we should and remove the instrumented bounds logic from the SetBoundaries function.
&cMap, &pMap,
&cVars, &pVars);

@@ -1045,6 +1046,12 @@ void DebuggerJitInfo::SetBoundaries(ULONG32 cMap, ICorDebugInfo::OffsetMapping *

DebuggerILToNativeMap *m = m_sequenceMap;

// TODO: Consider removing the handling for the InstrumentedILMap here.
// since we now have the ability to get an uninstrumented IL offset mapping
// directly from the VM. This work was not done when adding the instrumented
// IL mapping due to the work ocurring too close to the shipping deadline for .NET 10.
// If we do so, we need to change the input to this function to be the uninstrumented IL offset mapping

// For the instrumented-IL case, we need to remove all duplicate entries.
// So we keep a record of the last old IL offset. If the current old IL
// offset is the same as the last old IL offset, we remove it.
9 changes: 9 additions & 0 deletions src/coreclr/inc/clrtypes.h
Original file line number Diff line number Diff line change
@@ -370,6 +370,15 @@ inline UINT64 AlignDown(UINT64 value, UINT alignment)
return (value&~(UINT64)(alignment-1));
}

#ifdef __wasm__
inline uintptr_t AlignDown(uintptr_t value, UINT alignment)
{
STATIC_CONTRACT_LEAF;
STATIC_CONTRACT_SUPPORTS_DAC;
return (value&~(uintptr_t)(alignment-1));
}
#endif

#ifdef __APPLE__
inline SIZE_T AlignDown(SIZE_T value, UINT alignment)
{
Loading
Oops, something went wrong.
Loading
Oops, something went wrong.