Annotations are not preserved on non-whitespace trivia after formatting

[ajaybhargavb]

Version Used: CodeAnalysis 3.8.0, dotnet sdk 5.0.100

Steps to Reproduce:

    class Program
    {
        public static string Text => @"
namespace TestApp
{
    class Test
    {
    /* __marker__ */
    }
}
";

        static void Main(string[] args)
        {
            var pos = Text.IndexOf("/* __marker__ */");
            var syntaxTree = CSharpSyntaxTree.ParseText(Text);
            var root = syntaxTree.GetRoot();

            // Create an annotation and attach it to the multiline comment trivia. Then replace the original trivia with the annotated trivia.
            var annotation = new SyntaxAnnotation("marker");
            var markerTrivia = root.FindTrivia(pos, findInsideTrivia: true);
            var annotatedMarkerTrivia = markerTrivia.WithAdditionalAnnotations(annotation);
            root = root.ReplaceTrivia(markerTrivia, annotatedMarkerTrivia);

            // Format the root node and get all trivia with the marker annotation
            var formattedRoot = Formatter.Format(root, new AdhocWorkspace());
            var annotatedTrivia = formattedRoot.GetAnnotatedTrivia("marker");

            Console.WriteLine(annotatedTrivia.Count()); // Expected 1; Actual 0
        }
    }

Expected Behavior: The annotation is preserved

Actual Behavior: The annotation is not preserved

cc @sharwell @jasonmalinowski

[CyrusNajmabadi]

i'm not sure if we preserve annotations on trivia when run under formatting. can you instead annotate a token and try to get things working that way?

[ajaybhargavb]

I'm not sure if we preserve annotations on trivia when run under formatting

@CyrusNajmabadi, I am only asking it to be preserved for non-whitespace trivia.

can you instead annotate a token and try to get things working that way?

I discussed doing that with @jasonmalinowski and @sharwell and given multiple trivia (whitespace and non-whitespace) is grouped together, I am having to resort to ugly assumptions about the relative positioning of non-whitespace trivia to figure out the indentation to aid Razor formatting. Note: The behavior isn't consistent as well. For example, if I use // __marker as my marker instead of /* __marker__ */ then the annotation is preserved.

[sharwell]

... I am having to resort to ugly assumptions about the relative positioning of non-whitespace trivia ...

I think the assumptions would be completely valid. The formatter only changes whitespace so you should not be seeing insertion/removal/reordering of non-whitespace trivia during formatting.

[ajaybhargavb]

Discussed offline with @CyrusNajmabadi. Going to file a bug for this to investigate the inconsistency mentioned above.

Edit: Filed #49492