Skip to content

Fix crash on out-of-line class-template member definitions and harden getVtblIdx#769

Merged
tannergooding merged 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-investigate-crash-robustness
Jul 14, 2026
Merged

Fix crash on out-of-line class-template member definitions and harden getVtblIdx#769
tannergooding merged 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-investigate-crash-robustness

Conversation

@tannergooding

Copy link
Copy Markdown
Member

Addresses two crash/robustness issues: #510 (verified fix) and #534 (native hardening, unverified — see note below).

5cef78c2 — Fix crash on out-of-line class-template member definitions (#510)

An out-of-line class-template member definition (e.g. template<class T, class P> CCallResult<T,P>::CCallResult()) is a CXXMethodDecl whose this type is dependent, so Parent (base.Parent ?? ThisObjectType.AsCXXRecordDecl) is null — the semantic parent cursor is null and ThisObjectType is a PointerType, not a CXXRecordDecl. It is not a TemplateDecl, so the !GenerateTemplateBindings exclusion doesn't catch it, and the naming code assumes a non-null parent behind Debug.Assert (a no-op in Release), so it dereferences null and crashes. Under the Steamworks SDK headers from #510 this surfaces as an ArgumentNullException in the naming path.

Fix: exclude these decls up front in IsExcludedByName on the exact Parent is null signal, before any naming runs. One guard at the exclusion layer covers every downstream naming site because IsExcluded runs first in VisitDecl. Added CXXMethodDeclarationTest.OutOfLineTemplateConstructorDefinitionTest; the test reproduces the exact ArgumentNullException when the fix is reverted and passes with it. These decls produce no output today (templates aren't generated by default), so the baselines are empty — the regression value is "doesn't crash". Full suite green.


d09fc4f8 — Harden getVtblIdx against a missing vtable index (#534)

Warning

UNVERIFIED — this commit could not be compiled or tested here (it needs an LLVM/Clang build). The predicates are reasonable but not compile-checked; API names should be confirmed against the target Clang version before merging. It is isolated on top of the #510 fix so it can be dropped independently.

#534 is an AccessViolationException in Cursor_getVtblIdx, reached via CXXMethodDecl.VtblIndex during ordering in IsExcluded. The native Itanium getMethodVTableIndex / Microsoft getMethodVFTableLocation assert the method is present in the vtable map; with asserts disabled in release they dereference an end() iterator → AV. The managed VtblIndex already guards IsVirtual; the gap is purely native. It did not reproduce on the current native lib (clang 21.1.8), so this is defensive.

This guards both vtable queries behind a computable-vtable check (RD non-null, complete definition, non-dependent context, dynamic/polymorphic class) and falls back to the existing -1 sentinel instead of dereferencing an invalid iterator.


CC. @tannergooding — the native commit needs a real LLVM/Clang build to verify; happy to drop it if you'd rather land only the verified #510 fix.

Note

This PR description was drafted by Copilot.

tannergooding and others added 2 commits July 13, 2026 23:06
An out-of-line definition of a class-template member (e.g.
	emplate<class T, class P> C<T, P>::C() {}) is a CXXMethodDecl with a
dependent 	his type, so its parent record, function type, and vtable
index all resolve to null/invalid. It isn't a TemplateDecl and so slips
past the template exclusion, then crashes the naming machinery with an
ArgumentNullException (and, on older native libs, an AccessViolation via
the vtable index path). Exclude it up front in IsExcludedByName, before
any naming runs.

Repros against the Steamworks SDK headers from dotnet#510.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…g vtable index

getMethodVFTableLocation/getMethodVTableIndex assert the decl is present
in the vtable layout; with asserts disabled (release libs) a missing
entry dereferences an end() iterator and AVs (dotnet#534,
dotnet#510). Guard the query behind a computable-vtable check (complete,
non-dependent, polymorphic parent) and fall back to the existing -1
sentinel otherwise.

This changes native libClangSharp, which requires building LLVM/Clang to
compile and verify -- not possible in this session -- so it is committed
separately and clearly marked so it can be reviewed and trivially
dropped independently of the verified managed dotnet#510 fix.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@tannergooding
tannergooding marked this pull request as ready for review July 14, 2026 16:21
@tannergooding
tannergooding merged commit f458eca into dotnet:main Jul 14, 2026
14 checks passed
@tannergooding
tannergooding deleted the tannergooding-investigate-crash-robustness branch July 14, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant