Experimental documentation generation tool #253

cgranade · 2020-06-13T05:14:09Z

This PR opens for early feedback a possible path towards solving microsoft/qsharp-compiler#277, namely outputting documentation Markdown directly rather than as YAML. This PR packages that effort as an IRewriteStep that can be controlled by the IRewriteStep.AssemblyConstants property, and that can be included as an IsQscReference. The step also attaches documentation information as attributes that can be used by other parts of the Quantum Development Kit, such as IQ#.

Improvements over the existing approach:

Can display Q# syntax for UDT named items.
Extensible with new Q# features.
Can be used more easily by third-party libraries.
Attaches documentation data to the AST for reuse by other tools.

Remaining work:

Currently does not filter by the current compilation unit, such that all symbols included from references are documented as well.
Does not yet provide special support for @Deprecated(); this is currently supported by the non-experimental documentation generator.
Does not yet provide special support for @Attribute(), making it hard to learn how attributes should be used.
Missing unit tests.
Missing explanatory comments.
Named items don't yet have a dedicated Markdown section; this will require additions to the DocumentationParser in the qsharp-compiler repo.
Need to add comments to output Markdown files indicating that they are auto-generated.
Does not yet support making or merging toc.yml and index.md files.
Does not yet resolve unqualified names in /// # See Also blocks.

cgranade · 2020-06-16T17:55:13Z

Adding @tcNickolas and @KittyYeungQ for reference documentation impact, @anpaz-msft for build pipeline impact. If you have any time to offer feedback on the early prototype / experiment and the broader approach, I'd really appreciate it. Thank you!

bettinaheim · 2020-06-16T20:35:21Z

src/Simulation/DocumentationGenerationStep/Extensions.cs

+                )
+            );
+
+        internal static IAttributeBuilder<T> WithDocumentationAttributesFromDictionary<T>(


I believe you might want to use the tools in this PR: microsoft/qsharp-compiler#463
Let me know what attribute related things are missing, and we can see if it makes sense to add it to that static class.

Sounds good, I'll take a look at that, then. One thing I ran into was that a lot of the documentation attributes take a similar form on UDTs and callables, but even though QsCallable and QsCustomType both have an AddAttribute method, there's no common interface that allows calling AddAttribute on both. The AttributeBuilder class here was a hack around that to reduce code duplication.

src/Simulation/DocumentationGenerationStep/ProcessDocComments.cs

bettinaheim · 2020-06-16T20:43:10Z

src/Simulation/DocumentationGenerationStep/ProcessDocComments.cs

+            this.Namespaces = new ProcessDocComments.NamespaceTransformation(this, outputPath);
+        }
+
+        public QsCompilation OnCompilation(QsCompilation compilation) =>


Apply (inherited from the base class) would do that too.

renamed to OnCompilation

Does that imply that this method can be safely deleted?

bettinaheim · 2020-06-16T20:43:46Z

src/Simulation/DocumentationGenerationStep/ProcessDocComments.cs

+
+            private static QsCallable AddSummaryAttribute(QsCallable callable, string summary) =>
+                callable.AddAttribute(
+                    new QsDeclarationAttribute(


See the tools in the PR I mentioned in another comment.

bettinaheim · 2020-06-16T20:44:35Z

src/Simulation/DocumentationGenerationStep/ProcessDocComments.cs

+                    )
+                );
+
+            private async Task MaybeWriteOutput(QsCustomType type, DocComment docComment)


I would put these writers etc into a separate file.

Sounds good, I agree; this was convenient when quickly prototyping, but is not the most maintainable.

bettinaheim

What was the reason to put this on the runtime repo rather than the compiler repo? I believe you could just have a reference to the source code version of the compiler to build the rewrite step, couldn't you?

cgranade · 2020-06-16T21:41:40Z

Thanks for your feedback, @bettinaheim!

What was the reason to put this on the runtime repo rather than the compiler repo? I believe you could just have a reference to the source code version of the compiler to build the rewrite step, couldn't you?

My thinking was that this is similar in principle to how the C# generation app works, and thus to put it there, but I'm happy to move this to the compiler instead if that makes more sense.

anpaz · 2020-07-01T08:17:41Z

I think it makes sense on the compiler repo, as Bettina suggested.

Could you provide more information about usage? If I understand correctly these are Q# attributes that get added to the code directly and that would replace the current markdown docs, is that correct, are you planning to replace comments with attributes, or are these attributes go inside the comments?
I personally like much better using comments and markdown to write the documentation as we do today. I rather make enhancement to the YAML parser we use that to embed the documentation directly into the code.
Maybe you can give a couple of samples on how the new documentation would look?

Also, will these changes be backwards compatible? Switching all the documentation will be a huge PITA.

cgranade · 2020-07-01T14:20:35Z

Thanks for your comments, @anpaz-msft! Please see responses inline below.

I think it makes sense on the compiler repo, as Bettina suggested.

Sounds good, I'll move it there then and close this PR.

Could you provide more information about usage? If I understand correctly these are Q# attributes that get added to the code directly and that would replace the current markdown docs, is that correct, are you planning to replace comments with attributes, or are these attributes go inside the comments?

The new attributes are added automatically by the IRewriteStep in this PR, such that no change to Markdown documentation comments is required.

I personally like much better using comments and markdown to write the documentation as we do today. I rather make enhancement to the YAML parser we use that to embed the documentation directly into the code.

Agreed, hence this PR doesn't target changing that workflow at all, but makes the result of parsing those comments more widely available to other parts of the Quantum Development Kit (e.g.: IQ#) via attributes.

Maybe you can give a couple of samples on how the new documentation would look?

Also, will these changes be backwards compatible? Switching all the documentation will be a huge PITA.

So far the only backwards-incompatible change posed is to finally fix the casing convention for API reference UIDs to not require all-lowercase.

cgranade · 2020-08-28T22:17:28Z

Moved logic to compiler repo, closing this PR in favor of new approach.

cgranade added 2 commits June 12, 2020 22:00

Initial version of experimental docgen tool.

49ee735

Integrate with build.

0b5a695

cgranade requested review from bamarsha and bettinaheim June 13, 2020 05:14

Fix typo in manifest.

84b1f1d

cgranade requested review from anpaz, tcNickolas and KittyYeungQ June 16, 2020 17:54

bettinaheim reviewed Jun 16, 2020

View reviewed changes

src/Simulation/DocumentationGenerationStep/ProcessDocComments.cs Show resolved Hide resolved

bettinaheim reviewed Jun 16, 2020

View reviewed changes

src/Simulation/DocumentationGenerationStep/ProcessDocComments.cs Show resolved Hide resolved

bettinaheim reviewed Jun 16, 2020

View reviewed changes

This was referenced Jun 17, 2020

Some tools for adding attributes microsoft/qsharp-compiler#463

Merged

Synchronize named types and their items with the actual Q# types MicrosoftDocs/quantum-docs-pr#917

Closed

cgranade mentioned this pull request Aug 3, 2020

Fix namespace casing in API documentation MicrosoftDocs/quantum-docs-pr#991

Closed

cgranade mentioned this pull request Aug 17, 2020

Get API docs built to markdown qsharp-community/qram#27

Open

cgranade closed this Aug 28, 2020

cgranade deleted the cgranade/experimental/docgen-step branch August 28, 2020 22:20

cgranade mentioned this pull request Aug 28, 2020

Initial work on documentation generation tool as rewrite step microsoft/qsharp-compiler#594

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experimental documentation generation tool #253

Experimental documentation generation tool #253

cgranade commented Jun 13, 2020 •

edited

cgranade commented Jun 16, 2020

bettinaheim Jun 16, 2020

cgranade Jun 16, 2020

bettinaheim Jun 16, 2020

bettinaheim Jul 1, 2020

cgranade Jul 3, 2020

bettinaheim Jun 16, 2020

bettinaheim Jun 16, 2020

cgranade Jun 16, 2020

bettinaheim left a comment

cgranade commented Jun 16, 2020

anpaz commented Jul 1, 2020

cgranade commented Jul 1, 2020

cgranade commented Aug 28, 2020

Experimental documentation generation tool #253

Experimental documentation generation tool #253

Conversation

cgranade commented Jun 13, 2020 • edited

cgranade commented Jun 16, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bettinaheim left a comment

Choose a reason for hiding this comment

cgranade commented Jun 16, 2020

anpaz commented Jul 1, 2020

cgranade commented Jul 1, 2020

cgranade commented Aug 28, 2020

cgranade commented Jun 13, 2020 •

edited