Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[internal] Add Prettier support for the experimental Javascript backend #15480

Merged
merged 7 commits into from
May 16, 2022

Conversation

sureshjoshi
Copy link
Member

@sureshjoshi sureshjoshi commented May 15, 2022

This PR builds off of the recently added experimental Javascript targets, by adding the Prettier code formatter.

A NodeJS subsystem downloads the latest LTS Node runtime, which contains the npx cli tool, allowing NPM packages to be downloaded and run directly (with arguments). The way this practically works out is that we call:
node {path to npx cli} prettier {prettier args}

No explicit or specialized caching semantics have been considered yet, other than placing the NPM cache inside of Pant's named_caches.

(Closes #15456, #15489)

[ci skip-rust]
[ci skip-build-wheels]

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
- Need to review how nodejs is downloaded/installed
- Update NodeJS version
- Update PrettierJS version

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@sureshjoshi sureshjoshi added the category:internal CI, fixes for not-yet-released features, etc. label May 15, 2022
@sureshjoshi sureshjoshi changed the title [WIP] Add Prettier support for the experimental Javascript backend [WIP] [internal] Add Prettier support for the experimental Javascript backend May 15, 2022
@sureshjoshi
Copy link
Member Author

  • Still need to update to the latest LTS NodeJS
  • Eventually (but not part of this PR) it might be nice to leverage the user's system/NVM NodeJS version
  • Should any of this end up in a named cache? Not sure what precipitates named caches in the user's home dir

@Eric-Arellano
Copy link
Contributor

which contains the npx cli tool, allowing NPM packages to be downloaded and run directly (with arguments).

Good idea! I've had really bad performance when I tried running things this way two years ago (outside of Pants). I think everytime I ran, it took like 20 seconds to re-resolve. Do you know if there's a performance hit?

Copy link
Contributor

@Eric-Arellano Eric-Arellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huzzah!

@sureshjoshi
Copy link
Member Author

which contains the npx cli tool, allowing NPM packages to be downloaded and run directly (with arguments).

Good idea! I've had really bad performance when I tried running things this way two years ago (outside of Pants). I think everytime I ran, it took like 20 seconds to re-resolve. Do you know if there's a performance hit?

That was my experience back in the day too - now it's... 100's of ms?

If I've already got Prettier installed and I use the system npx command, it's less than a second. Via pants, with everything already installed, we're closer to 1.3 seconds including all the Pants stuff that happens

@Eric-Arellano
Copy link
Contributor

That was my experience back in the day too - now it's... 100's of ms?

Okay - probably acceptable for proof of concept, and not blocking, but generally I expect users might not be thrilled with that. Do you know if it's possible to use NPX to install the binary once, then always point to that point (probably via Digest)?

@sureshjoshi
Copy link
Member Author

That was my experience back in the day too - now it's... 100's of ms?

Okay - probably acceptable for proof of concept, and not blocking, but generally I expect users might not be thrilled with that. Do you know if it's possible to use NPX to install the binary once, then always point to that point (probably via Digest)?

I mean, I'll say that it feels VERY quick to me - in spite of what time tells me. It went from completely useless and unusable a couple years ago to something I would definitely use at the command-line now for ad hoc commands, and wouldn't think twice about.

For Pants, I don't know if it's the resolution, or just regular pants "stuff". You can see below, that I think the overhead of calling Prettier is perfectly fine, and as a percentage of call time - NPX isn't taking up much

> time ./pants_from_sources --version
2.13.0.dev1
./pants_from_sources --version  1.84s user 0.30s system 99% cpu 2.163 total
> time ./pants_from_sources fmt hellojavascript/src/main.js
10:12:51.82 [WARN] Completed: pants.backend.javascript.lint.prettier.rules.prettier_fmt - prettier made changes.
  hellojavascript/src/main.js

+ prettier made changes.
./pants_from_sources fmt hellojavascript/src/main.js  1.96s user 0.32s system 98% cpu 2.311 total

- Updated with PR requested changes

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@sureshjoshi sureshjoshi changed the title [WIP] [internal] Add Prettier support for the experimental Javascript backend [internal] Add Prettier support for the experimental Javascript backend May 16, 2022
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
level=request.level,
env={
"PATH": f"/bin:./{nodejs_dir}/bin",
"npm_config_cache": str(named_caches_dir.val / "npm"), # Normally stored at ~/.npm
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Eric-Arellano Is this all there is to do for named caches?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh oops, I didn't see this. You also have to set append_only_caches. See pex_process.py for an example

Copy link
Contributor

@Eric-Arellano Eric-Arellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good!

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:internal CI, fixes for not-yet-released features, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add PrettierJS formatting plugin for javascript_sources
2 participants