Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support TypeScript usage without compile step #297

Closed
crutchcorn opened this issue Nov 29, 2021 · 49 comments
Closed

Support TypeScript usage without compile step #297

crutchcorn opened this issue Nov 29, 2021 · 49 comments

Comments

@crutchcorn
Copy link
Member

crutchcorn commented Nov 29, 2021

We need help testing this feature: #297 (comment)

Today, we support:

  • ESM .js plopfile
  • ESM .mjs plopfile
  • CSM .js plopfile
  • CSM .cjs plopfile

It would be nice if it also handled:

  • TS .ts plopfile

Without needing to add a compilation step. This would likely be done by:

  • Adding in a tsc to output the single JS file to a temporary directory
  • Read from project's tsconfig.json file
@amwmedia
Copy link
Member

amwmedia commented Dec 1, 2021

If the user wants to use TS, would it be safe to assume they have already added tsc to the project? If so, could logic be something like... if plopfile.ts is found and tsc is available, process the plopfile via tsc?

This adds support for TS, without adding more dependencies that are likely not needed.

Thoughts?

@Pike
Copy link
Contributor

Pike commented Dec 2, 2021

Would that work if the plopfile.ts had imports?

Also, projects might have TS loaders hooked up, like when using deno or ts-node instead of plain node, right? 296 looks like he installed the ts-node loader, but wasn't actually registering it? I haven't tried it, but https://github.com/TypeStrong/ts-node#node-flags looks like it.

@crutchcorn
Copy link
Member Author

@amwmedia this would be nice, but unfortunately @Pike is quite right - imports would fail.

What's worse, one of the reasons that ts-node won't work for our usage right now is what appears to be a lack of full ESM support:

TypeStrong/ts-node#1007

What's more - we wouldn't be able to get the return value from ts-node.

Instead, what I might suggest is that we use a Node loader from esbuild to run plop itself, similar to this:

https://github.com/unicorn-utterances/unicorn-utterances/blob/nextjs/package.json#L6

https://www.npmjs.com/package/esbuild-node-loader

This will handle .ts, .tsx, and other files OOTB for us, without having to change much. ESBuild is supported by huge projects like Vite.

@crutchcorn
Copy link
Member Author

I just looked into the viability to do this for real and there's a few minor problems we'll need to sort out first:

  1. node-plop's tsconfig is broken (missing comma) in node_modules 🤐 Sorry lol
  2. We get the following error Could not resolve "#ansi-styles"
  3. To get our E2E tests working, we need something similar to RN's LogBox.ignoreLogs but for cli-testing-library to ignore the ExperimentalWarning, which we expect to see but prints to stderr so as a result, fails

#2 is occurring because of our chalk dep reliance.

Luckily, this is already fixed for us in esbuild@0.14.1. We'd simply need to make a PR to update esbuild-node-loader to fix this problem

To see the branch I started to POC this idea (just starting with e2e tests of running plop with the ESBuild:

https://github.com/plopjs/plop/tree/ts-no-build-step

@codybrouwers
Copy link
Contributor

I was able to get esbuild-node-loader working really well with the only issue I ran into was the missing comma which is now fixed (plopjs/node-plop#215).

Here's an example repo I made that shows it working:
https://github.com/CodyBrouwers/plop-esbuild-example

Happy to help any other way I can!

@cspotcode
Copy link
Contributor

ts-node's ESM support should cover plop's use-cases, including skipping typechecking and using a native transpiler for speed. If users have configured it for other parts of their project, we'll pick up that config automatically, since it's the same. Let me know if you have questions.

@sdotson
Copy link

sdotson commented Nov 16, 2022

Any updates on this?

@Nicholaiii
Copy link

tsx presents itself as an esbuild-based alternative to ts-node. It's a lot faster and skips typechecking. I suggest using either tsx or straight esbuild in plop to solve this.

@airtonix
Copy link

so i've kinda worked around this with: "

yarn add --exact --dev ts-node

with a tsconfig.json of :

{
  "compilerOptions": {
    "module": "ESNext",
    "moduleResolution": "node"
  },
  // Most ts-node options can be specified here using their programmatic names.
  "ts-node": {
    "swc": true,
    "esm": true,
    "pretty": true,
    // It is faster to skip typechecking.
    // Remove if you want ts-node to do typechecking.
    "transpileOnly": true,
    "files": true,
    "compilerOptions": {
      "module": "CommonJS"
      // compilerOptions specified here will override those declared below,
      // but *only* in ts-node.  Useful if you want ts-node and tsc to use
      // different options with a single tsconfig.json.
    }
  }
}

then in my justfile:

...

#
# Generator 
#
alias gen  := generate
alias g  := generate
generate *ARGS:
    yarn ts-node \
    ./node_modules/plop/bin/plop.js {{ARGS}}

...

and then the plopfile.ts:

import type { NodePlopAPI } from 'plop';

module.exports = function Plopfile(plop: NodePlopAPI) {
  plop.setGenerator('test', {
    prompts: [
      {
        type: 'confirm',
        name: 'wantTacos',
        message: 'Do you want tacos?',
      },
    ],
    actions: [],
  });
};

resulting in:

Screencast.from.2023-03-26.22-25-22.webm

@konstantinB1
Copy link

I played around trying to make this work, and found success with swc compiler, since using tsc one was pretty slow.

With concurrently package in package.json - scripts:
"generate:plop": "concurrently -g --names \"swc plopfile\\,generate api\" -g \"npx swc ./.build/rtkGenerator/plopfile.ts -o ./.build/rtkGenerator/plopfile.js\" \"ts-node --transpileOnly --esm ./.build/rtkGenerator/run.ts\"",

run.ts

#!/usr/bin/env node
/* eslint-disable import/no-extraneous-dependencies */
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';

import minimist from 'minimist';
import { Plop, run } from 'plop';

const args = process.argv.slice(2);
const argv = minimist(args);

Plop.prepare(
    {
        cwd: argv?.cwd as string,
        configPath: join(
            dirname(fileURLToPath(import.meta.url)),
            'plopfile.js',
        ),
    },
    // eslint-disable-next-line @typescript-eslint/no-misused-promises
    (env) => Plop.execute(env, run),
);

Maybe someone here with more expertise can explain why when running plop with plopfile flag command, right after the compilation phase throws Plopfile not found!. Looks like configPath is null, but when i redo the comp + plop script with plopfile.js available before even next compilation starts it works fine. With wrapping plop seems to work fine, buts its an extra step.

@moltar
Copy link

moltar commented Sep 4, 2023

I can say with a high degree of certainty, that the following setup works with the following conditions:

Package Versions

❯ pnpm ls plop typescript ts-node
Legend: production dependency, optional only, dev only

devDependencies:
plop 3.1.2
ts-node 10.9.1
typescript 5.2.2

./tsconfig.plop.json

{
  "compilerOptions": {
    "verbatimModuleSyntax": true,
    "isolatedModules": true,
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "module": "CommonJS",
    "target": "ES2015",
    "moduleResolution": "node",
    "strict": true,
    "noEmit": true,
    "inlineSourceMap": true,
    "inlineSources": true
  },
  "include": [
    ".plop/**/*.ts"
  ],
  "exclude": [
    "node_modules"
  ],
  "ts-node": {
    "transpileOnly": true,
    "swc": true,
    "experimentalSpecifierResolution": "node"
  }
}

./plop/plopfile.ts

import type { NodePlopAPI } from 'plop';

module.exports = function (plop: NodePlopAPI) {
  plop.setGenerator('test', {
      description: 'This is loaded.',
      prompts: [{
        name: 'name',
        message: 'What is your name?',
        type: 'input',
      }],
      actions: [
        {
          type: 'add',
          template: 'foo {{name}}',
          path: 'foo-bar',
        }
      ]
  });
};

Running:

export TS_NODE_PROJECT=tsconfig.plop.json
export NODE_OPTIONS="--loader ts-node/esm --no-warnings"

plop --plopfile .plop/plopfile.ts

@moltar

This comment was marked as outdated.

@crutchcorn
Copy link
Member Author

@moltar I appreciate you suggesting this, but I think the fix might be even "simpler" (conceptually) than that. See, we're using Gulp's Liftoff library to detect configuration files:

https://github.com/plopjs/plop/blob/main/packages/plop/src/plop.js#L5C22-L5C29
https://github.com/plopjs/plop/blob/main/packages/plop/src/plop.js#L18-L24

Which allows you to enable TS support through this mechanism:

https://github.com/gulpjs/liftoff/blob/466e17ba75d213c968caae003eac7d9180ba9cda/README.md?plain=1#L543

@moltar
Copy link

moltar commented Sep 4, 2023

@crutchcorn Well, that is amazing! 😁 Going to mark my comment as hidden to avoid confusion.

This was referenced Sep 4, 2023
@crutchcorn
Copy link
Member Author

You'll all be happy to know that I've just implemented this functionality in Plop v4:

#396

It turns out that it was even easier than anticipated from the easy method outlined in my last comment.

@crutchcorn
Copy link
Member Author

This should be solved in Plop 4.0!

https://github.com/plopjs/plop/releases/tag/plop%404.0.0

@nzacca
Copy link

nzacca commented Sep 5, 2023

@crutchcorn

Thanks for the update! Just tried the example config from the tests and sadly could not get this to work: https://github.com/plopjs/plop/tree/main/packages/plop/tests/examples/typescript

Receiving the following error:

[PLOP] Something went wrong with reading your plop file TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for C:\dev\plopfile.ts
    at new NodeError (node:internal/errors:399:5)
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:79:11)
    at defaultGetFormat (node:internal/modules/esm/get_format:121:38)
    at defaultLoad (node:internal/modules/esm/load:81:20)
    at nextLoad (node:internal/modules/esm/loader:163:28)
    at ESMLoader.load (node:internal/modules/esm/loader:605:26)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:457:22)
    at new ModuleJob (node:internal/modules/esm/module_job:64:26)
    at ESMLoader.#createModuleJob (node:internal/modules/esm/loader:480:17)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:434:34) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

@glenkitchen
Copy link

Hi @crutchcorn
Works for me, Thx

@noahgregory-basis
Copy link

It appears @crutchorn/plop is no longer available. Is that intended?

@glenkitchen
Copy link

Hi @noahgregory-basis
Global install worked for me. I use yarn:

  yarn global add @crutchcorn/plop@4.1.0-alpha.1

@noahgregory-basis
Copy link

Hi @noahgregory-basis Global install worked for me. I use yarn:

  yarn global add @crutchcorn/plop@4.1.0-alpha.1

The package name fails to resolve. It does not appear to be public in the NPM registry (or Yarn registry for that matter).

@SkrzypMajster
Copy link

@crutchcorn @amwmedia
Hi guys 👋
I have a question - when can we expect the release of a new, stable version of the plop package containing this improvement?

I will be very grateful for your response 🙂

@EdiAfremovDemostack
Copy link

@cspotcode
Works for me, thanks!

@luciano96
Copy link

Also worked for me! Thanks @cspotcode!

@divramod
Copy link

divramod commented Nov 4, 2023

hey hey @crutchcorn, thx for the great tool and the effort to support ts! it is running fine for me!

i have a little caveat. i am in a nx project and would like to use functionalities from our lib projects, like validators, i already wrote for other parts of our project.
can you write a little instruction on how to use the @crutchcorn/plop@4.1.0-alpha.1 version with tsx or ts-node?
because plop is failing, when i import from one of our libs with a typescript path alias.

my plopgenerator looks like this

import { info } from "@org/util-node/echo/headers"
import { hello } from '../utils/test-me'

const pathBase = process.env['PATH_BASE']

export const plopGeneratorTest = {
  description: 'this is a test',
  prompts: [
    {
      type: 'input',
      name: 'name',
      message: 'What is your name?',
      validate: function (value: string): true | "name is required" {
        console.log(hello())
        info(hello())
        if (/.+/.test(value)) {
          return true
        }
        return 'name is required'
      },
    },
    {
      type: 'checkbox',
      name: 'toppings',
      message: 'What pizza toppings do you like?',
      choices: [
        {
          name: 'Cheese',
          value: 'cheese',
          checked: true,
        },
        { name: 'Pepperoni', value: 'pepperoni' },
        { name: 'Pineapple', value: 'pineapple' },
        { name: 'Mushroom', value: 'mushroom' },
        { name: 'Bacon', value: 'bacon', checked: true },
      ],
    },
  ],
  actions: [
    {
      type: 'add',
      path: `${pathBase}/test/{{name}}.js`,
      templateFile: 'templates/test.hbs',
    },
  ],
}

this fails with

plop --plopfile path/to/app/src/plugins/plop/plopfile.ts
[PLOP] Something went wrong with reading your plop file Error: Cannot find module '@org/util-node/echo/headers'
Require stack:
- /path/to/app/src/plugins/plop/generators/test.ts
- /path/to/app/src/plugins/plop/plopfile.ts
    at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
    at l.default._resolveFilename (/Users/mod/Library/pnpm/global/5/.pnpm/tsx@3.14.0/node_modules/tsx/dist/cjs/index.cjs:1:1671)
    at Module._load (node:internal/modules/cjs/loader:920:27)
    at Module.require (node:internal/modules/cjs/loader:1141:19)
    at require (node:internal/modules/cjs/helpers:110:18)
    at <anonymous> (path/to/app/src/plugins/plop/generators/test.ts:1:22)
    at Object.<anonymous> (path/to/app/src/plugins/plop/generators/test.ts:46:1)
    at Module._compile (node:internal/modules/cjs/loader:1254:14)
    at Object.j (/Users/mod/Library/pnpm/global/5/.pnpm/tsx@3.14.0/node_modules/tsx/dist/cjs/index.cjs:1:1197)
    at Module.load (node:internal/modules/cjs/loader:1117:32) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    'path/to/app/src/plugins/plop/generators/test.ts',
    'path/to/app/src/plugins/plop/plopfile.ts'
  ]
}

without the path alias import it is running fine

@sammcj
Copy link

sammcj commented Nov 14, 2023

Not having native .ts support is breaking a few projects I work on that have linting / rules to ensure there's no non-ts js files in repos.

It would be good to have this added.

@ctsstc
Copy link

ctsstc commented Dec 11, 2023

If using Deno or Bun, I'm wondering if less lifting is required, but for now I'll take TS support in whatever flavor it comes in.

@michaelfarrell76
Copy link

Screenshot 2023-12-16 at 3 04 28 PM struggling to get this to work with yarn v2/berry - did anyone get that working?

@crutchcorn
Copy link
Member Author

crutchcorn commented Dec 22, 2023

FWIW, beyond Yarn 2 support (which might be another major issue that I'd need to test first), what @divramod raised is a legit concern that I didn't consider when building; I need some way to allow users to bypass loading in TSX automatically.

Overall, I'm not super happy with how my alpha release came out, which is why it hasn't been released yet.

Apologies y'all. I'll try to come back to this in 2024 when some other priorities I have on the table shake out more.

In the meantime, if someone wanted to contribute, please feel free to take my PR and add in a argv flag to bypass loading TSX, figure out and add Yarn Berry tests, and docs. If this is done by someone else (even based on my existing work) I'm more than happy to expedite review sooner than I would be able to get to it myself.

@tobiashochguertel
Copy link

I don't know if it is helpful. I solved the typescript configuration issue by transpiling it to JavaScript. Furthermore, I used SWC as transpiler because it was just faster as tsc. Here is my setup / workaround:

yarn add -D rimraf @swc/cli @swc/core

File: package.json

{
...
  "scripts": {
   ...
    "plop": "npx rimraf ./plopfile.js && npx swc ./plopfile.ts --out-dir . && plop"
  },
...
}

File: .swcrc

{
  "$schema": "https://json.schemastore.org/swcrc",
  "minify": false,
  "module": {
    "type": "commonjs",
    "strict": false,
    "strictMode": true,
    "lazy": false,
    "noInterop": false
  },
  "jsc": {
    "parser": {
      "syntax": "typescript"
    },
    "target": "esnext",
    "loose": false,
    "externalHelpers": false,
    "keepClassNames": false
  },
  "isModule": true
}

File: tsconfig.json

{
  "compilerOptions": {
    "target": "ES6",
    "allowJs": true,
    "module": "commonjs",
    "skipLibCheck": true,
    "esModuleInterop": true,
    "noImplicitAny": true,
    "sourceMap": true,
    "baseUrl": ".",
    "outDir": "dist",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "paths": {
      "*": [
        "node_modules/*"
      ]
    },
    "jsx": "react"
  },
  "include": [
    "src/**/*"
  ]
}

File: plopfile.ts

import {NodePlopAPI} from 'plop';

module.exports = function (plop: NodePlopAPI) {

// controller generator
    plop.setGenerator('controller', {
        description: 'application controller logic',
        prompts: [{
            type: 'input',
            name: 'name',
            message: 'controller name please'
        }],
        actions: [{
            type: 'add',
            path: 'src/{{name}}.js',
            templateFile: 'plop-templates/controller.hbs'
        }]
    });
};

That works for me:

CleanShot 2024-01-26 at 11 43 37

@tobiashochguertel
Copy link

I improved my solution with the help of md5sum:

Previously

File: package.json

{
...
  "scripts": {
   ...
    "plop": "npx rimraf ./plopfile.js && npx swc ./plopfile.ts --out-dir . && plop"
  },
...
}

--> changes to a md5sum check before transpiling the typescript file to JavaScript:

...
    "scripts": {
        ...
        "plop": "md5sum --check --status plopfile.md5 || yarn run plop::transpile && plop",
        "plop::transpile": "echo 'Deleting plopfile.js and regenerate it+md5' && npx rimraf ./plopfile.js && npx swc ./plopfile.ts --out-dir . && md5sum plopfile.ts > plopfile.md5",
    }
...

It's fast(er), works great.

@benallfree
Copy link
Collaborator

A twist on @tobiashochguertel's solution, using tsup:

    "plop": "md5sum --check --status plopfile.md5 || pnpm plop:transpile && plop",
    "plop:transpile": "echo 'Deleting plopfile and regenerate it+md5' && rimraf ./plopfile.js && tsup ./plopfile.ts --format esm --out-dir . && md5sum plopfile.ts > plopfile.md5",

This was referenced Apr 5, 2024
@crutchcorn
Copy link
Member Author

@benallfree just as a heads up, I generally discourage folks from using tsup these days because their .d.ts generation has been buggy for us on TanStack projects (as a maintainer of TanStack, not consumer)

@benallfree
Copy link
Collaborator

benallfree commented Apr 6, 2024

@crutchcorn Thank you, yes #423 is a better approach:

  "scripts": {
    "plop": "cross-env NODE_OPTIONS='--import tsx' plop --plopfile=plopfile.ts",
   }

@moltar
Copy link

moltar commented Apr 6, 2024

@crutchcorn what do you recommend instead of tsup?

@Nicholaiii
Copy link

@crutchcorn what do you recommend instead of tsup?

unbuild is the 🐐 . See this starter for a usage example

@benallfree
Copy link
Collaborator

@Nicholaiii In this case tsx is preferred instead of actually generating a bundle. See this discussion.

  "scripts": {
    "plop": "cross-env NODE_OPTIONS='--import tsx' plop --plopfile=plopfile.ts",
   }

@prisis
Copy link

prisis commented Apr 11, 2024

I saw jiti in the comments, just a hint how to add support for js, cjs, mjs, ts, cts and mts

// try-require.ts

import jiti from "jiti";

// eslint-disable-next-line @typescript-eslint/no-explicit-any,@typescript-eslint/explicit-module-boundary-types
const tryRequire = (id: string, rootDirectory: string, errorReturn: any): any => {
    // eslint-disable-next-line @typescript-eslint/naming-convention
    const _require = jiti(rootDirectory, { esmResolve: true, interopDefault: true });

    try {
        return _require(id);
        // eslint-disable-next-line @typescript-eslint/no-explicit-any
    } catch (error: any) {
        if (error.code !== "MODULE_NOT_FOUND") {
            console.error(new Error(`Error trying import ${id} from ${rootDirectory}`, {
                cause: error,
            }));
        }

        return errorReturn;
    }
};

export default tryRequire;

config loding:

const config = tryRequire("./plopfile", cwd, undefined);

If you would accept a PR like this, i would be happy to add it :)

@benallfree
Copy link
Collaborator

@prisis I think #428 is the way they're going forward with TS support. Cool jiti though!

prisis added a commit to PrisisForks/plop that referenced this issue Apr 12, 2024
prisis added a commit to PrisisForks/plop that referenced this issue Apr 12, 2024
@crutchcorn
Copy link
Member Author

Regrettably, at this time we're moving forward with a "We won't support this feature" in Plop without additional configuration today.

Instead, we're going forward with:

  • Finding plopfile.ts automatically
  • Updating our docs to showcase how to easily use tsx to load a TS Plopfile
  • Adding a test to make sure our docs remain accurate
  • Maybe a "See our docs on how to configure TypeScript" if an error is thrown reading a TS plopfile? (what do you think @benallfree ?)

This is done via this PR: #428 and is now merged.

@crutchcorn crutchcorn closed this as not planned Won't fix, can't repro, duplicate, stale Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.