[CLI] Get rid of Yeoman

[bajtos]

Refactor the CLI package to use inquirer and ejs directly, get rid of the dependency on Yeoman.

This will allow us to use TypeScript too, we should migrate the CLI codebase to the standard TypeScript project as part of this task.

UPDATE 2020-02-11: Let's not limit ourselves to inquirer and ejs only. The ecosystem offers many more alternatives for building CLI tools, we should research the options and pick the best one. See the comments below for inspiration.

Related:

• Refactor to play well with Yeoman's run loop strongloop/generator-loopback#139
• https://github.com/strongloop/loopback-next/pull/825/files#r159253215
• Enable json config options and non-interactive mode #1384 (comment)

List of issues compiled from the discussion:

• In the "old" loopback-cli, our "model" generator is invoking "property" generator in a loop. This is not supported by Yeoman anymore, we have a hacky workaround in place. As a result, upgrades to newer versions of Yeoman were usually painful, because the hacky workaround had to be updated to match the new internals of Yeoman.
• Invoking a generator directly from CLI is a bit of PITA, there are more hacks needed to make this happen - see e.g. https://github.com/strongloop/loopback-next/blob/6cc83b666e4b2ce2f979eab44dec78b7c01c41be/packages/cli/bin/cli.js#L66-L68
• I find individual generators difficult to test in isolation. Verifying how CLI arguments are handled is even more difficult.
• AFAICT, there is no way how to test non-trivial prompting logic - for example the loop asking for multiple properties. A mocked prompt can define only a single set of answers. Running the generator/CLI end-to-end and parsing stdout is brittle because Inquirer uses lots of ANSI control codes to control what's displayed on the screen.
• Another missing feature is inability to verify configuration of individual prompts. For example when using a choices question populated dynamically (e.g. the datasource prompt showing all datasources known by the app), we want to verify the list of choices offered.
• I also find the necessity to pepper our generator actions with if (this.shouldExit()) return false; as a horrible code smell. (See e.g. here and here and here). I was took a quick glance at other generators, not all actions start with shouldExit check, so we may already have subtle bugs there!
• Subjectively, I dislike the conventions forced by Yeoman:
  • All generators live in different index.js files, which makes navigation difficult to me. Instead of app/index.js, I'd like to have app.generator.js file.
  • Implementing individual steps as named methods on a generator object made sense back in ES5 days, because it simplified async flow control. Today with ES2017's async/await, I prefer to write idiomatic JavaScript instead.
• Reading from stdin has to use readline or complex workarounds, it's not possible to read entire input into a string using the usual streams API.
• We need to call process.chdir in our tests, which makes it impossible to run the tests in parallel via worker threads. See run Mocha's tests concurrently mochajs/mocha#2839 (We found a solution for this.)

[privateOmega]

@bajtos I would like to help however I can on this issue.

[bajtos]

@raymondfeng Here are the issues I was experiencing in the old generator-loopback frequently:

• Our "model" generator is invoking "property" generator in a loop. This is not supported by Yeoman anymore, we have a hacky workaround in place. As a result, upgrades to newer versions of Yeoman were usually painful, because the hacky workaround had to be updated to match the new internals of Yeoman.
• Invoking a generator directly from CLI is a bit of PITA, there are more hacks needed to make this happen - see e.g. https://github.com/strongloop/loopback-next/blob/6cc83b666e4b2ce2f979eab44dec78b7c01c41be/packages/cli/bin/cli.js#L66-L68
• I find individual generators difficult to test in isolation. Verifying how CLI arguments are handled are even more difficult.
• AFAICT, there is no way how to test non-trivial prompting logic - for example the loop asking for multiple properties. A mocked prompt can define only a single set of answers. Running the generator/CLI end-to-end and parsing stdout is brittle because Inquirer uses lots of ANSI control codes to control what's displayed on the screen.
• Subjectively, I dislike the conventions forced by Yeoman:
  • All generators live in different index.js files, which makes navigation difficult to me. Instead of app/index.js, I'd like to have app.js file.
  • Implementing individual steps as named methods on a generator object made sense back in ES5 days, because it simplified async flow control. Today with ES2017's async/await, I prefer to write idiomatic JavaScript instead.

@privateOmega thank you for chiming in! Right now, we are debating whether we actually want to drop Yeoman or whether it's perhaps not that bad. Once we reach consensus, I can show you pointers where to start.

[bajtos]

AFAICT, there is no way how to test non-trivial prompting logic - for example the loop asking for multiple properties. A mocked prompt can define only a single set of answers. Running the generator/CLI end-to-end and parsing stdout is brittle because Inquirer uses lots of ANSI control codes to control what's displayed on the screen.

Another missing feature is inability to verify configuration of individual prompts. For example when using a choices question populated dynamically (e.g. the datasource prompt showing all datasources known by the app), we want to verify the list of choices offered.

[bajtos]

I also find the necessity to pepper our generator actions with if (this.shouldExit()) return false; as a horrible code smell. (See e.g. here and here and here). I was took a quick glance at other generators, not all actions start with shouldExit check, so we may already have subtle bugs there!