Expose API for getting JSDoc nodes in TypeScript files

[evmar]

Closure adds various semantic JSDoc annotations that aren't currently modeled by TypeScript, such as @export (unrelated to TS export) and @nosideeffects.
https://developers.google.com/closure/compiler/docs/js-for-compiler#overview

As part of our tool chain to run TypeScript through Closure we'd like to be able to munge these via the TypeScript API. It appears TypeScript gathers JSDoc comments when parsing JavaScript -- is there a good reason to not gather these in TypeScript as well?

Specifically, I believe my suggestion amounts to removing the "if" statement in the below code (though I'm not certain this is the right place) in parser.ts:

        function addJSDocComment<T extends Node>(node: T): T {
            if (contextFlags & NodeFlags.JavaScriptFile) {
                ...all of the code is in here...
            }
        }

[mhegazy]

They are disabled for performance purposes, given that in TS context nothing is inferred from the comments, where as in JS types can come from comments.

so you need these for emit purposes, correct?

[evmar]

@mhegazy Oops, I never responded, sorry! Yes, we consume and generate modified JSDoc in our Closure emitter.

But this is not only for our crazy Closure compiler hacks. You can imagine other more reasonable examples; for example, a tool that generates documentation from TypeScript code would need access to the JSDoc (now that I think of it, how does VSCode get at it?). An optimizer might want access to specific JSDoc attributes; e.g. the Closure compiler can make use of @nosideeffects annotations among others.

Some ideas to fix this that don't require turning it on always:

• You could put "parse JSDoc or not" behind a flag in the TS parser API.
• You could expose the JSDoc parsing code such that a tool walking the TS AST could call back into the parser.

[evmar]

Re my second suggestion, right now we have to write our own JSDoc parser because the TS one isn't public API:
https://github.com/angular/tsickle/blob/e79f7837e2774f8fbbef695a1ab7471c27369548/src/tsickle.ts#L85

[mhegazy]

nothing stops us from doing this on demand. see https://github.com/Microsoft/TypeScript/blob/master/src/services/services.ts#L360-L366.

we could add a new API, say getJSDocComemnts that would return you a Node and will invoke the parser on the text and cache the resulting tree.

[mhegazy]

We would be open to take a PR for this.

[evmar]

It appears that TS maybe now gathers JSDoc always. That is, given a ts.Node you can call getDocumentationComment() on it and it updates the (internal property) node.jsDoc (and does some caching).

However, there's still no way to get at the parsed JSDoc there. getDocumentationComment() only returns the comment portion of the JSDoc, not the tags. So my request still stands, but now it's just a request to expose API to retrieve the full JSDoc of a node.

[tonyhallett]

I have not checked every code path but it does appear that Parser.parseSourceFile will call createNodeWithJsDoc for JsDocContainer nodes. Can we just remove the internal comment ?

export interface JSDocContainer {
        /* @internal */ jsDoc?: JSDoc[];                      // JSDoc that directly precedes this node
        /* @internal */ jsDocCache?: ReadonlyArray<JSDocTag>; // Cache for getJSDocTags
    }

To save doing
var classDeclaration:ts.ClassDeclaration=getFromSomewhere(); var jsDocs=:ts.JSDoc[]=(classDeclaration as any).jsDoc

[nadavwix]

i am experiencing the same issue, where in order to reach the JSdoc comments for some nodes i have to resort to using the private "jsDoc" member on the ast nodes

[RyanCavanaugh]

@sandersn thoughts on exposing that property?

[sandersn]

ts.getJSDocTags is the public API from the compiler, which does a full search plus cache. The private jsDoc property is just the jsdoc comment directly on a node, but jsdoc frequently appears on a node above the one it actually applies to.

The getDocumentationComment method has been broken for this purpose since I rewrote the jsdoc parser a couple of years ago.

I think it's always the right thing to do a search for applicable jsdoc tags, not just that attached directly to the node.

@nadavwix @tonyhallett @evmar can you weigh in on whether getJSDocTags is sufficient for what you want to do?

[cspotcode]

getJSDocTags is returning an empty array when I transform a .d.ts file using the transformer API via ts.transform(). My use case: I want to perform custom transformation of declarations based on JSDoc tags, for example switching methods from public to private or omitting declarations based on custom rules.

Do I need to do something different to getJSDocTags for a .d.ts file?

[sandersn]

@cspotcode I don't think so. Any differences you saw were probably a bug. Can you file a new one with more details about how you're using the transformer API?

[cspotcode]

@sandersn Turns out I need to turn on setParentNodes. After setting that to true, getJSDocTags is working. I'm not 100% sure when setParentNodes is required, or when it might be smart to disable it, but I think I'll always enable it for simplicity. Thanks for the quick reply!

[cspotcode]

When using my transformer via "ttypescript", it seems setParentNodes is not enabled. getJSDocTags doesn't return anything and I can't perform the necessary transformations.

Is getJSDocTags supposed to require parent nodes to be set?
Is there a way to enable setParentNodes during a normal compilation?
Is there a different, recommended workaround?

I know you don't support ttypescript, so I'm just wondering what is the most pragmatic solution for my situation, even if it's a hack.

[J5]

From my reading of the code setParentNodes needs to be true because getJSDocTags traverses the parent nodes to find the comment tags. I have a different but similar issue in that getJSDocTags returns empty because there are just comments with no tags on a ClassDecleration. I can clearly see JSDoc is set and it retrieved but then dropped from the output. Unfortunately getDocumentComment is not on the ClassDecleration interface. It would be nice to expose getJSDocCommentAndTags which underpin getJSDocTags.

[dsherret]

It's strange to be required to use the type checker to get the js doc comment. It would be nice to solely do this via the AST and expose the jsDoc property, which would allow people to quickly see the js doc comment's range, comment, and tags.

[cspotcode]

I think the typechecker is required because it includes the linker, which is responsible for setting up references between nodes. E.g. associating every usage of a variable to where it's declared. Linking also associates jsDoc nodes with the functions, variables, and fields that they describe.

I assume, though I'm not sure, that the linker is invoked on-demand to provide linked jsDoc nodes, but full typechecking does not need to run.

[jsdw]

I'm finding that even with setParentNodes to true, running ts.getJSDocTags(node) just returns an empty array for each of the top level nodes I iterate over. My input source looks like this:

/** foo doc comment */
export type foo = {
  /** barry! */
  bar: number
  wibble: string
  hey: 2
  another: false
  foo: 'hello' | 'there'
}

/**
 * Hello!
 */
/** hello there */
type bar = "hello"

The jsDoc field contains the comments as I'd expect. Is this a bug?

[cspotcode]

@jsdw IIRC getJSDocTags will get tags specifically. Those comments don't have any tags (e.g. @param or @deprecated)

[jsdw]

Ah, thanks, that makes sense! Do you know of a supported means to obtain the comments? I'm only interested in the text content of them.

I'm not aware of any api docs for this stuff offhand so I'm just muddling through based on the typescript types that exist on these functions!