Question about building a new Abstract Syntax Tree

[tinchoz49]

Hi everyone,

I'm working on build an Abstract Syntax Tree for JSDoc and I got the next situation problem:

I have this js file:

/**
 * @param {number} a
 * @param {number} b
 * @returns {number}
 */
function sum (a, b) {
   return a + b
}

I parse the syntax and then I have a node like:

{ 
  type: 'Function',
  name: 'sum',
  ...
}

Now my question is:

should I define the Parameter and ReturnType nodes as a children of the function (example 1)
or it's better to define custom properties for those nodes (example 2)

Example 1:

{ 
  type: 'Function',
  name: 'sum',
  children: [
     { type: 'Parameter', name: 'a', value: 'number' },
     { type: 'Parameter', name: 'b', value: 'number' },
     { type: 'ReturnType' , value: 'number' },
  ]
}

Example 2:

{ 
  type: 'Function',
  name: 'sum',
  paramaters: [
      { type: 'TypeNumber', name: 'a'  },
      { type: 'TypeNumber', name: 'b' },
  ],
  return: { type: 'TypeNumber' },
}

Thank you!!

[wooorm]

Interesting! I’m currently working with estree (the de facto JS AST). What’s your reason for making a unist based JS tree instead of estree?

I’ve also thought about it. And indeed: programming languages have different places, not just “children”. Whereas content (XML, HTML, Markdown) typically only have “children”.

[tinchoz49]

And indeed: programming languages have different places, not just “children”.

Yes, It makes sense.

[wooorm]

Interesting case! For jsdoc2markdown, do you really need a JavaScript AST? Or is it maybe more of a JSDoc AST? 🤔 In any case, pinging @ChristianMurphy who might have ideas on types

[ChristianMurphy]

One thing to consider, you may not need to write a parser or come up with an AST for TSDoc.
TSDoc already has an AST https://tsdoc.org/play

It will likely be easier/more maintainable to write a transform from tsdoc's AST to mdast directly.
Or alternatively tsdoc -> unist compatible tsdoc -> mdast if you want more layers of transforms in the process.

(this overall relates/is similar to unifiedjs/rfcs#5, which looks to drive both types and docs based off doc comments)

[tinchoz49]

I try TSDoc but from what I see it seems that they are using their own parser, that's why I preffer to use the TypeScript parser directly because it's already provides a good support for JSDoc, I will say, event better than the current JSDoc standard.

I'm using ts-morph (a basic wrapper for navigating the AST TypeScript) and building a minimal AST for TypeScript Definition Files + documentation.

Similar to the approach that you mentioned: https://dev.to/open-wc/generating-typescript-definition-files-from-javascript-5bp2

[tinchoz49]

For example, I have this file (sorry if is not the best example):

human.js

/**
 * A library for building widgets.
 *
 * @remarks
 * The `widget-lib` defines the {@link IWidget} interface and {@link Widget} class,
 * which are used to build widgets.
 *
 * @packageDocumentation
 */

/**
 * This is a callback example
 * @callback createHumanCallback
 * @param {number} age Set the age
 * @returns {Human}
 */

/**
 * This is a typedef example
 * @typedef options
 * @type {Object}
 * @prop {string} options.a some a description
 * @prop {number} options.b
 */

/**
 * It's a human factory
 *
 * @param {string} name Set a name
 * @returns {createHumanCallback} a function
 */
function humanFactory (name) {
  return function createHuman (age) {
    return new Human(name, age)
  }
}

/**
 * @example
 * const human = new Human('pepe', 20)
 * console.log(human.age)
 * @description eso es
 */
class Human {
  /**
   * @constructor
   * @param {string} name Set a name
   * @param {number} age Set the age
   */
  constructor (name, age) {
    this._name = name
    this._age = age
  }

  /**
   * @prop {string}
   */
  get name () {
    return this._name
  }

  get age () {
    return this._age
  }
}

const CONSTANT = 'test'

module.exports = { Human, humanFactory, CONSTANT }

And the declaration file created from ts-morph return this:

{
  "statements": [
    {
      "name": "createHumanCallback",
      "isExported": true,
      "isDefaultExport": false,
      "hasDeclareKeyword": false,
      "docs": [
        {
          "kind": 22,
          "description": "\nThis is a callback example",
          "tags": []
        },
        {
          "kind": 22,
          "description": "\n\nThis is a callback example\n",
          "tags": [
            {
              "kind": 23,
              "tagName": "param",
              "text": "{number}"
            },
            {
              "kind": 23,
              "tagName": "returns",
              "text": "{Human}"
            },
            {
              "kind": 23,
              "tagName": "callback",
              "text": "createHumanCallback"
            }
          ]
        }
      ],
      "type": "(age: number) => Human",
      "typeParameters": [],
      "kind": 36
    },
    {
      "name": "options",
      "isExported": true,
      "isDefaultExport": false,
      "hasDeclareKeyword": false,
      "docs": [
        {
          "kind": 22,
          "description": "\nThis is a typedef example",
          "tags": []
        },
        {
          "kind": 22,
          "description": "\n\nThis is a typedef example\n",
          "tags": [
            {
              "kind": 23,
              "tagName": "type",
              "text": "{Object}"
            },
            {
              "kind": 23,
              "tagName": "prop",
              "text": "{string}"
            },
            {
              "kind": 23,
              "tagName": "prop",
              "text": "{number}"
            },
            {
              "kind": 23,
              "tagName": "typedef",
              "text": "options"
            }
          ]
        }
      ],
      "type": "{\n    /**\n     * some a description\n     */\n    a: string;\n    b: number;\n}",
      "typeParameters": [],
      "kind": 36
    },
    {
      "decorators": [],
      "typeParameters": [],
      "docs": [
        {
          "kind": 22,
          "description": "",
          "tags": [
            {
              "kind": 23,
              "tagName": "example",
              "text": "\nconst human = new Human('pepe', 20)\nconsole.log(human.age)"
            },
            {
              "kind": 23,
              "tagName": "description",
              "text": "eso es"
            }
          ]
        }
      ],
      "isAbstract": false,
      "implements": [],
      "name": "Human",
      "isExported": true,
      "isDefaultExport": false,
      "hasDeclareKeyword": false,
      "kind": 1,
      "ctors": [
        {
          "parameters": [
            {
              "name": "name",
              "type": "string",
              "isReadonly": false,
              "decorators": [],
              "hasQuestionToken": false,
              "kind": 28,
              "isRestParameter": false
            },
            {
              "name": "age",
              "type": "number",
              "isReadonly": false,
              "decorators": [],
              "hasQuestionToken": false,
              "kind": 28,
              "isRestParameter": false
            }
          ],
          "typeParameters": [],
          "docs": [
            {
              "kind": 22,
              "description": "",
              "tags": [
                {
                  "kind": 23,
                  "tagName": "constructor"
                },
                {
                  "kind": 23,
                  "tagName": "param",
                  "text": "{string} name Set a name"
                },
                {
                  "kind": 23,
                  "tagName": "param",
                  "text": "{number} age Set the age"
                }
              ]
            }
          ],
          "kind": 3
        }
      ],
      "methods": [],
      "properties": [
        {
          "name": "_name",
          "type": "string",
          "hasQuestionToken": false,
          "hasExclamationToken": false,
          "isReadonly": false,
          "docs": [],
          "isStatic": false,
          "isAbstract": false,
          "decorators": [],
          "hasDeclareKeyword": false,
          "kind": 29
        },
        {
          "name": "_age",
          "type": "number",
          "hasQuestionToken": false,
          "hasExclamationToken": false,
          "isReadonly": false,
          "docs": [],
          "isStatic": false,
          "isAbstract": false,
          "decorators": [],
          "hasDeclareKeyword": false,
          "kind": 29
        }
      ],
      "getAccessors": [
        {
          "name": "name",
          "parameters": [],
          "returnType": "string",
          "typeParameters": [],
          "docs": [
            {
              "kind": 22,
              "description": "",
              "tags": [
                {
                  "kind": 23,
                  "tagName": "prop",
                  "text": "{string}"
                }
              ]
            }
          ],
          "isStatic": false,
          "isAbstract": false,
          "decorators": [],
          "kind": 13
        },
        {
          "name": "age",
          "parameters": [],
          "returnType": "number",
          "typeParameters": [],
          "docs": [],
          "isStatic": false,
          "isAbstract": false,
          "decorators": [],
          "kind": 13
        }
      ],
      "setAccessors": []
    },
    {
      "name": "humanFactory",
      "parameters": [
        {
          "name": "name",
          "type": "string",
          "isReadonly": false,
          "decorators": [],
          "hasQuestionToken": false,
          "kind": 28,
          "isRestParameter": false
        }
      ],
      "returnType": "createHumanCallback",
      "typeParameters": [],
      "docs": [
        {
          "kind": 22,
          "description": "\nIt's a human factory\n",
          "tags": [
            {
              "kind": 23,
              "tagName": "param",
              "text": "{string} name Set a name"
            },
            {
              "kind": 23,
              "tagName": "returns",
              "text": "{createHumanCallback} a function"
            }
          ]
        }
      ],
      "isExported": true,
      "isDefaultExport": false,
      "hasDeclareKeyword": false,
      "isGenerator": false,
      "isAsync": false,
      "kind": 11
    },
    {
      "isExported": true,
      "isDefaultExport": false,
      "hasDeclareKeyword": false,
      "docs": [],
      "kind": 39,
      "declarationKind": "const",
      "declarations": [
        {
          "name": "CONSTANT",
          "type": "\"test\"",
          "hasExclamationToken": false,
          "kind": 38
        }
      ]
    }
  ],
  "kind": 34
}

[wooorm]

Regardless of the a/b choice, I’d suggest having two children: Parameters and Body (same as estree).

Now, the Q is to place them in children or directly on a node.
One idea I’ve had is to use both: the nodes are in children but there are also getters (and setters?) defined for node.params to get the first child if it’s a Parameters node.
But what’s not so nice about that is that typically unist is JSON only: it can be serialized and parsed again and results in an equivalent tree. Getters/setters will break that.

For MDX, which includes JSX, I recently went with an attributes array, which includes nodes, and it also has normal children (for the content in those tags). Maybe you could do the same? Have body be the children, and parameters another field with nodes?

[tinchoz49]

Have body be the children, and parameters another field with nodes?

I will do that, thank you