implement CLI commands- app, new, test, build, map #27

threepointone · 2020-06-30T22:25:30Z

(This is pretty yucky, but I'm trying to share and land stuff incrementally. )

This commit adds an implementation for the key cli commands.

modular app : creates a new repository , with yarn's
workspaces enabled. It then creates a package 'app', as a default
create-react-app installation. This is then modified to work with yarn's
workspaces. We rewrite the app's webpack config to read from a widgets
folder, where we'll host other components. We also copy an
implementation for a faux 'store' to save configurations, tuples of
[component, props].

modular new : This creates a new 'widget' under widgets/.

modular test: runs all tests in the repository.

modular build: runs a full build, output in app/build

modular map: generates a mapping from widget names -> React.lazy() components
for those apps. This will be used in conjunction with the store to
dynamically load components as needed.

This also sets up eslint and prettier in the repo, and can be called
with yarn lint and yarn prettier, respectively.

.gitignore

packages/cli/README.md

packages/cli/cli.js

NMinhNguyen · 2020-07-01T02:58:18Z

packages/cli/cli.js

+      '  ',
+    ),
+  );
+  shell.mkdir('widgets');


I'm a little wary of using widgets and dashboard semantics - reminds me of enterprise-y apps. But maybe I'm overthinking it because you can have like iOS widgets that have nothing to do with banks.

yeah, open to ideas for what this word should be. I started with 'components', but don't want folks to think that they have to do it for literally every component. this might be a question for matthew.

We had big discussions around terminology, I'll share the link with you regarding what we've decided internally.

packages/cli/cli.js

NMinhNguyen · 2020-07-01T03:00:30Z

packages/cli/cli.js

+  map();
+}
+
+function test() {


My worry with using .js for this is that if these functions are rewritten to be async, ESLint won't tell us we need to add await test() at the call sites and we'll have runaway promises. Whereas @typescript-eslint is able to catch that.

technically we might actually have long running processes (eg, running server). either way, happy to rewrite this in ts once it becomes a little more fleshed out and I start adding tests. didn't want to block this work tho, and it's not consumed by any other modules, so exposed types aren't really important.

so exposed types aren't really important.

It's more about the internal types than public types, especially given it's a CLI without a programmatic API. Consumers will be invoking it as a CLI. I'm more worried about us making mistakes when iterating on it.

happy to rewrite this in ts once it becomes a little more fleshed out and I start adding tests.

packages/cli/craco.config.js

packages/cli/generateComponentImports.js

threepointone · 2020-07-01T10:13:55Z

btw, I've removed the need for PR review before merging. I trust everyone to use their best judgement.

This commit adds an implementation for the key cli commands. modular app <name>: creates a new repository <name>, with yarn's workspaces enabled. It then creates a package 'app', as a default create-react-app installation. This is then modified to work with yarn's workspaces. We rewrite the app's webpack config to read from a widgets folder, where we'll host other components. We also copy an implementation for a faux 'store' to save configurations, tuples of [component, props]. modular new <name>: This creates a new 'widget' under widgets/<name>. modular test: runs all tests in the repository. modular build: runs a full build, output in app/build modular map: generates a mapping from widget names -> React.lazy() components for those apps. This will be used in conjunction with the store to dynamically load components as needed. This also sets up eslint and prettier in the repo, and can be called with yarn lint and parn prettier, respectively.

sebinsua · 2020-07-06T17:15:00Z

packages/cli/cli.js

+  );
+  shell.exec('yarn add prettier --dev -W');
+  shell.exec(
+    `yarn add ~/code/work/modular/packages/eslint-config-modular --dev -W`,


This breaks obv.

sebinsua · 2020-07-06T17:16:40Z

packages/cli/cli.js

+  shell.exec('git init'); // TODO - make an initial commit?
+}
+
+function widget(name) {


Btw, it wasn't originally clear to me that (1) there can only be one app within the repository (and this had a .git directory, etc), and (2) the widget command only operates within this folder and crashes otherwise.

the app shouldn't have a .git directory? but we should be able to modify this for multiple apps in one repo, I simply hadn't planned for it.

I think it should have a .git directory. What I'm saying is that before I ran the commands I had misunderstandings of what they did and where they should be run.

threepointone · 2020-07-06T17:17:00Z

Oh I'd promised to land this PR. I'll clean up and land this by tomorrow evening.

sebinsua · 2020-07-06T17:17:01Z

packages/cli/cli.js

+        workspaces: ['app', 'widgets/*'],
+        modular: {
+          app: 'app',
+          widgets: 'widgets',


Do we even want this to be configurable?

good question. maybe not.

sebinsua · 2020-07-06T17:18:28Z

packages/cli/cli.js

+    const modularConfig = JSON.parse(fs.readFileSync('./package.json', 'utf8'))
+      .modular;
+    if (
+      typeof modularConfig.app === 'string' &&


We fail if there is no modular key within the package.json?

open to other ideas to detect "this is a modular repo", this was just one idea.

sebinsua · 2020-07-09T17:27:11Z

@threepointone I made some changes in a separate branch, I can do a PR into this or just merge them directly in? And also, feel free to tell me to not do some of these things, because I got carried away 👀 (hence the commit separation is super bad. I am sorry.)

What I did:

Rewrote in TypeScript.
I have a preference towards using fs-extra and execa over fs and shelljs. Both are in use within create-react-app already, and I trust that they work well. I think they're meant to be 'robust', but one benefit that I care about is that it's more clear which functions are sync -- I found it very surprising that shelljs is sync by default, and that it doesn't say this against the functions. (Also, I'm OK with letting everything be sync for now, becase I can't see anything that would seriously benefit from being async.)
I wanted to make sure that the monorepo commands can still execute when you're in the wrong folder, so I am using find-up to find the parent folder with a package.json#modular and using this to define the root of a project. Related to this, I avoided paths which were relative in the code, by working out where things should be run and passing the correct directory into each exec as cwd, and making all relative paths absolute by joining to this. (If we need to print out directories, I think this will be easier to debug.)
I pulled the code and README.md that were copied into new repositories into a template/{widgets,app}/* folder structure. I think this makes it clearer which files are executed and which files are copied and rewritten.

Btw, it was kind of a pain to work with, because linking didn't bring across necessary binaries, and I kept forgetting to setup links again. Not sure if you had a better debug process than I...?!

Also, I agree with your comments. Happy to look into dropping meow? I'd perhaps keep craco for easy hacking for now, but agree that should also be dropped eventually, too.

A few thoughts:

We could have a babel-macro or webpack-virtual-modules instead of having a manual modular map command that has to be run. I think the benefit would be that it'd be more invisible to users, but the con is that it's not fully statically analyseable by an IDE/TypeScript (we'd have to have a general definition). Another possibility is that we could just not have this at all, and expect people to manually create the lazy(() => import('Widget_1')) -- but perhaps then they might forget to use dynamic imports? (We could write a lint rule for that if it was important though.)
Why do we need to use PascalCase for the widget import/package names. Can't we use kebab-case for these as is normal for packages names?
Is it weird for us to have a modular app [app-name] command? If we grab create-modular-app (which is free) we could then use yarn create modular-app to start a new modular project. Additionally, perhaps we could also change modular new [widget-name] to modular add [widget-name] (and we could chuck these in a package modular-scripts)?

threepointone · 2020-07-10T12:15:21Z

replies inline.

@threepointone I made some changes in a separate branch, I can do a PR into this or just merge them directly in? And also, feel free to tell me to not do some of these things, because I got carried away 👀 (hence the commit separation is super bad. I am sorry.)

Feel free to commandeer/overwrite this PR.

What I did:

Rewrote in TypeScript.

Dope.

I have a preference towards using fs-extra and execa over fs and shelljs. Both are in use within create-react-app already, and I trust that they work well. I think they're meant to be 'robust', but one benefit that I care about is that it's more clear which functions are sync -- I found it very surprising that shelljs is sync by default, and that it doesn't say this against the functions. (Also, I'm OK with letting everything be sync for now, becase I can't see anything that would seriously benefit from being async.)

Dope.

I wanted to make sure that the monorepo commands can still execute when you're in the wrong folder, so I am using find-up to find the parent folder with a package.json#modular and using this to define the root of a project. Related to this, I avoided paths which were relative in the code, by working out where things should be run and passing the correct directory into each exec as cwd, and making all relative paths absolute by joining to this. (If we need to print out directories, I think this will be easier to debug.)

Dope.

I pulled the code and README.md that were copied into new repositories into a template/{widgets,app}/* folder structure. I think this makes it clearer which files are executed and which files are copied and rewritten.

Dope.

Btw, it was kind of a pain to work with, because linking didn't bring across necessary binaries, and I kept forgetting to setup links again. Not sure if you had a better debug process than I...?!

Also, I agree with your comments. Happy to look into dropping meow? I'd perhaps keep craco for easy hacking for now, but agree that should also be dropped eventually, too.

It's not urgent, so I'd prioritise anything else at the moment. As long as it's not exposed externally, it shouldn't matter.

A few thoughts:

We could have a babel-macro or webpack-virtual-modules instead of having a manual modular map command that has to be run. I think the benefit would be that it'd be more invisible to users, but the con is that it's not fully statically analyseable by an IDE/TypeScript (we'd have to have a general definition). Another possibility is that we could just not have this at all, and expect people to manually create the lazy(() => import('Widget_1')) -- but perhaps then they might forget to use dynamic imports? (We could write a lint rule for that if it was important though.)

I keep going back and forth on this. But I'm certain it has to be done automatically, not manually. It's meant to be used by code that loads component definitions from a database or whatever, and it would be annoying to do it by hand whenever . Either way, let's keep the manual command for now and attack it in a later PR.

Why do we need to use PascalCase for the widget import/package names. Can't we use kebab-case for these as is normal for packages names?

Yeah I was just extra enthusiastic when I changed the package keys on the import map. I do this we should pascal case the generated component name tho. So {'x-y': lazy(() => import('x-y'))}, but in the component's index.js function XY(){...}. Fair?

Is it weird for us to have a modular app [app-name] command? If we grab create-modular-app (which is free) we could then use yarn create modular-app to start a new modular project. Additionally, perhaps we could also change modular new [widget-name] to modular add [widget-name] (and we could chuck these in a package modular-scripts)?

Dope. This also make us not have to worry about the modular package name (tho I'm still going to try and get it).

Thanks very much for this effort, it's looking so much better than my attempt. You the best.

sebinsua · 2020-07-13T12:02:13Z

Merged as-is. I'll do those extra things as separate PRs now.

NMinhNguyen · 2020-07-15T00:23:07Z

packages/eslint-config-modular/package.json

+    "eslint-plugin-jsx-a11y": "^6.3.1",
+    "eslint-plugin-react": "^7.20.0",
+    "eslint-plugin-react-hooks": "^4.0.5",
+    "typescript": "~3.9.3"


Should we match https://github.com/facebook/create-react-app/blob/2da5517689b7510ff8d8b0148ce372782cb285d7/packages/eslint-config-react-app/package.json#L17-L27 and specify them as peerDependencies instead? You wouldn't want dependencies like typescript to be brought in via an ESLint config package.

Sounds like a good idea. I think this config needs to be redone anyway.

Sounds like a good idea.

Cool, I'll raise a PR later.

I think this config needs to be redone anyway.

Do you have specific points in mind?

Do you have specific points in mind?

I'm not sure how all of the dependencies are being used by the index.js. Are they in use?

Also, shouldn't we provide people with TypeScript linting, etc.

Are they in use?

Yeah, by CRA. All of those are CRA's peerDependencies.

Also, shouldn't we provide people with TypeScript linting, etc.

CRA does have some TS linting, yeah.

threepointone force-pushed the cli branch 2 times, most recently from 2c01e23 to 428c410 Compare June 30, 2020 22:40

NMinhNguyen reviewed Jul 1, 2020

View reviewed changes

threepointone force-pushed the cli branch 3 times, most recently from 777cc23 to 755b509 Compare July 1, 2020 06:32

threepointone force-pushed the cli branch 6 times, most recently from f3466c1 to 3c2fc68 Compare July 2, 2020 23:16

threepointone force-pushed the cli branch from 3c2fc68 to 9ee96e1 Compare July 3, 2020 12:01

sebinsua reviewed Jul 6, 2020

View reviewed changes

jpmorganchase deleted a comment from threepointone Jul 6, 2020

Rewrite in TypeScript with execa, fs-extras and find-up

d2b8a72

sebinsua merged commit 8be0e65 into master Jul 13, 2020

sebinsua deleted the cli branch July 13, 2020 12:01

NMinhNguyen reviewed Jul 15, 2020

View reviewed changes

NMinhNguyen mentioned this pull request Aug 6, 2020

Fix package.json in eslint-config-modular-app #42

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

implement CLI commands- app, new, test, build, map #27

implement CLI commands- app, new, test, build, map #27

threepointone commented Jun 30, 2020 •

edited

NMinhNguyen Jul 1, 2020

threepointone Jul 1, 2020

susanafcosta Jul 1, 2020 •

edited by threepointone

NMinhNguyen Jul 1, 2020

threepointone Jul 1, 2020

NMinhNguyen Jul 1, 2020

threepointone Jul 1, 2020

threepointone commented Jul 1, 2020

sebinsua Jul 6, 2020

sebinsua Jul 6, 2020

threepointone Jul 6, 2020

sebinsua Jul 7, 2020

threepointone commented Jul 6, 2020

sebinsua Jul 6, 2020

threepointone Jul 6, 2020

sebinsua Jul 6, 2020

threepointone Jul 6, 2020 •

edited

sebinsua commented Jul 9, 2020 •

edited

threepointone commented Jul 10, 2020

sebinsua commented Jul 13, 2020

NMinhNguyen Jul 15, 2020

sebinsua Jul 15, 2020

NMinhNguyen Jul 15, 2020

sebinsua Jul 15, 2020

NMinhNguyen Jul 15, 2020 •

edited

implement CLI commands- app, new, test, build, map #27

implement CLI commands- app, new, test, build, map #27

Conversation

threepointone commented Jun 30, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

susanafcosta Jul 1, 2020 • edited by threepointone

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

threepointone commented Jul 1, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

threepointone commented Jul 6, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

threepointone Jul 6, 2020 • edited

Choose a reason for hiding this comment

sebinsua commented Jul 9, 2020 • edited

threepointone commented Jul 10, 2020

sebinsua commented Jul 13, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NMinhNguyen Jul 15, 2020 • edited

Choose a reason for hiding this comment

threepointone commented Jun 30, 2020 •

edited

susanafcosta Jul 1, 2020 •

edited by threepointone

threepointone Jul 6, 2020 •

edited

sebinsua commented Jul 9, 2020 •

edited

NMinhNguyen Jul 15, 2020 •

edited