Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable basic Kept validation in NativeAOT running linker tests #78828

Merged
merged 11 commits into from
Nov 30, 2022

Conversation

vitek-karas
Copy link
Member

This implements basic Type and Method Kept attribute validation.
It adds a KeptBy property on all KeptAttribute which uses the same ProducedBy enum to specify exceptions for each target product. (Eventually after the source merge we should rename ProducedBy enum to just Tool or Platform, but it's not important and I didn't want to do it here as it would be a giant diff).

The validation is done by going over all ConstructedEETypeNode and IMethodBodyNode nodes in the final graph and remembers that list. Then it compares that list to the expected kept members as per the attributes in tests. There are some tweaks:

  • Filters out all compiler added types/members
  • Doesn't track generic instantiations - only remember definition
  • If a method body is kept, then it behaves as if the type is also kept even though there's no ConstructedEETypeNode - this is technically wrong (since NativeAOT can remove the type if for example only static methods are used from it), but it would require major changes to the linker tests (since in IL this is a necessity). If we ever need precise validation of this, we would probably add a new attribute to check just for this.

I also copied all of the "other assemblies" kept validation code from ResultChecker (were it lives in linker) to AssemblyChecker where it will need to be to actually work (and where it should probably be anyway). That code is not used yet.

Left lot of TODO/commented out code in the AssemblyChecker - lots of other validation to enable eventually.

Fixed/adapted all of the tests which were already ported to NativeAOT to pass with this new validation.
Removed a test for unresolved members, since it actually fails NativeAOT compilation, so it doesn't test anything really.

One tiny product change:
Display names now consistently include all parent types when writing out nested type names. ILLink already does this and NativeAOT was a bit inconsistent. Since the display names are used only in diagnostics, this brings the diagnostics experience closer between the two tools. The added benefit is that we can use display names to compare members between Cecil and Internal.TypeSystem more precisely.

@ghost
Copy link

ghost commented Nov 24, 2022

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

This implements basic Type and Method Kept attribute validation.
It adds a KeptBy property on all KeptAttribute which uses the same ProducedBy enum to specify exceptions for each target product. (Eventually after the source merge we should rename ProducedBy enum to just Tool or Platform, but it's not important and I didn't want to do it here as it would be a giant diff).

The validation is done by going over all ConstructedEETypeNode and IMethodBodyNode nodes in the final graph and remembers that list. Then it compares that list to the expected kept members as per the attributes in tests. There are some tweaks:

  • Filters out all compiler added types/members
  • Doesn't track generic instantiations - only remember definition
  • If a method body is kept, then it behaves as if the type is also kept even though there's no ConstructedEETypeNode - this is technically wrong (since NativeAOT can remove the type if for example only static methods are used from it), but it would require major changes to the linker tests (since in IL this is a necessity). If we ever need precise validation of this, we would probably add a new attribute to check just for this.

I also copied all of the "other assemblies" kept validation code from ResultChecker (were it lives in linker) to AssemblyChecker where it will need to be to actually work (and where it should probably be anyway). That code is not used yet.

Left lot of TODO/commented out code in the AssemblyChecker - lots of other validation to enable eventually.

Fixed/adapted all of the tests which were already ported to NativeAOT to pass with this new validation.
Removed a test for unresolved members, since it actually fails NativeAOT compilation, so it doesn't test anything really.

One tiny product change:
Display names now consistently include all parent types when writing out nested type names. ILLink already does this and NativeAOT was a bit inconsistent. Since the display names are used only in diagnostics, this brings the diagnostics experience closer between the two tools. The added benefit is that we can use display names to compare members between Cecil and Internal.TypeSystem more precisely.

Author: vitek-karas
Assignees: vitek-karas
Labels:

area-NativeAOT-coreclr

Milestone: 8.0.0

@@ -36,7 +36,8 @@ public static string GetDisplayName(this MethodDesc method)

if (method.IsConstructor)
{
sb.Append(method.OwningType.GetDisplayNameWithoutNamespace());
DefType defType = method.OwningType.GetTypeDefinition() as DefType;
sb.Append(defType?.Name ?? method.Name);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll just assume the changes here match what IL Linker is doing. But for the ?? part I want to point out that we would only take that codepath for array constructors (so e.g. new int[10, 20, 30];) and whether that part is intentional.

If so, I'd suggest we change the if above to if (method.IsConstructor && method.OwningType is DefType defType). Array constructors will fall down to the last if in the cascade.

/// This property can override that by setting only the platforms
/// which are expected to keep the target.
/// </summary>
public ProducedBy KeptBy { get; set; } = ProducedBy.TrimmerAnalyzerAndNativeAot;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: kind of weird name to repeat the name of the attribute on the property.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So just [Kept (By = Tool.Trimmer)]? (After we rename the enum...)
I can change that, sounds good.

Copy link
Member

@sbomer sbomer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, thanks! LGTM on first pass. I can do another more detailed review tomorrow if you like, but the fact that it already caught a bunch of differences between linker/nativeaot is great.

@@ -310,6 +311,7 @@ public static StringBuilder GetDisplayNameWithoutNamespace (this TypeReference t
break;
}

type = type.GetElementType ();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate more on why this is needed? I just read what the GetElementType method does, but still not fully sure why is always necessary, nor why it doesn't return null for some of the types.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Cecil this method is something like "Get me the type without the decorators". So for arrays it returns the element type, but it also works for things like pointer types (where it returns the non-pointer version), by-ref types and so on. And on all other "normal" types it just returns this.

The case I ran into was a by-ref of a nested type. Before this change we would write it out as simple NestedType&, after the change it prints out ParentType.NestedType&.

vitek-karas and others added 2 commits November 30, 2022 08:34
…Checker.cs

Co-authored-by: Tlakaelel Axayakatl Ceja <tlakaelel.ceja@microsoft.com>
@vitek-karas vitek-karas merged commit 1412ee7 into dotnet:main Nov 30, 2022
@vitek-karas vitek-karas deleted the KeptValidation branch November 30, 2022 18:30
@ghost ghost locked as resolved and limited conversation to collaborators Dec 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants