diff --git a/accepted/0000-dependency-selector-syntax.md b/accepted/0000-dependency-selector-syntax.md new file mode 100644 index 000000000..c7f3ce891 --- /dev/null +++ b/accepted/0000-dependency-selector-syntax.md @@ -0,0 +1,395 @@ +# `npm query` Command & Dependency Selector Syntax + +## Summary + +Introduce a new `npm query` commmand which exposes a new dependency selector syntax (informed by & respecting many aspects of the [CSS Selectors 4 Spec](https://dev.w3.org/csswg/selectors4/#relational)). + +## Motivation + +- Standardize the shape of, & querying of, dependency graphs with a robust object model, metadata & selector syntax +- Leverage existing, known language syntax & operators from CSS to make disparate package information broadly accessible +- Unlock the ability to answer complex, multi-faceted questions about dependencies, their relationships & associative metadata +- Consolidate redundant logic of similar query commands in `npm` (ex. `npm fund`, `npm ls`, `npm outdated`, `npm audit` ...) + +## Detailed Explanation + +This RFC's spec & implementation should closely mimic the capabilities of existing CSS Selector specifications. Notably, we'll introduce limited net-new classes, states & syntax to ensure the widest adoption & understanding of paradigms. When deviating, we'll explicitely state why & how. + +## Implementation + +- `Arborist`'s `Node` Class will have a new `.querySelectorAll()` method + - this method will return a filtered, flattened dependency Arborist `Node` list based on a valid query selector +- Introduce a new command, `npm query`, which will take a dependency selector & output a flattened dependency Node list (output is in `json` by default, but configurable) + +### Dependency Selector Syntax + +#### Overview: + +- there is no "type" or "tag" selectors (ex. `div, h1, a`) as a dependency/target is the only type of `Node` that can be queried +- the term "dependencies" is in reference to any `Node` found in the `idealTree` returned by `Arborist` + +#### Selectors + +- `*` universal selector +- `#` dependency selector (equivalent to `[name="..."]`) +- `#@` (equivalent to `[name=]:semver()`) +- `,` selector list delimiter +- `.` class selector +- `:` pseudo class selector +- `>` direct decendent/child selector +- `~` sibling selector + +#### Pseudo Selectors +- [`:not()`](https://developer.mozilla.org/en-US/docs/Web/CSS/:not) +- [`:has()`](https://developer.mozilla.org/en-US/docs/Web/CSS/:has) +- [`:where()`](https://developer.mozilla.org/en-US/docs/Web/CSS/:where) +- [`:is()`](https://developer.mozilla.org/en-US/docs/Web/CSS/:is) +- [`:root`](https://developer.mozilla.org/en-US/docs/Web/CSS/:root) matches the root node/dependency +- [`:scope`](https://developer.mozilla.org/en-US/docs/Web/CSS/:scope) matches node/dependency it was queried against +- [`:empty`](https://developer.mozilla.org/en-US/docs/Web/CSS/:empty) when a dependency has no dependencies +- [`:private`](https://docs.npmjs.com/cli/v8/configuring-npm/package-json#private) when a dependency is private +- `:link` when a dependency is linked +- `:deduped` when a dependency has been deduped +- `:override` when a dependency is an override +- `:extraneous` when a dependency exists but is not defined as a dependency of any node +- `:invalid` when a dependency version is out of its ancestors specified range +- `:missing` when a dependency is not found on disk +- `:outdated` when a dependency is not `latest` +- `:vulnerable` when a dependency has a `CVE` +- `:semver()` matching a valid [`node-semver`](https://github.com/npm/node-semver) spec +- `:path()` [glob](https://www.npmjs.com/package/glob) matching based on dependencies path relative to the project +- `:realpath()` [glob](https://www.npmjs.com/package/glob) matching based on dependencies realpath +- `:type()` [based on currently recognized types](https://github.com/npm/npm-package-arg#result-object) + +#### [Attribute Selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_selectors) + +The attribute selector evaluates the key/value pairs in `package.json` if they are `String`s. + +- `[]` attribute selector (ie. existence of attribute) +- `[attribute=value]` attribute value is equivalant... +- `[attribute~=value]` attribute value contains word... +- `[attribute*=value]` attribute value contains string... +- `[attribute|=value]` attribute value is equal to or starts with... +- `[attribute^=value]` attribute value begins with... +- `[attribute$=value]` attribute value ends with... + +#### `Array` & `Object` Attribute Selectors + +The generic `:attr()` pseudo selector standardizes a pattern which can be used for attribute selection of `Object`s, `Array`s or `Arrays` of `Object`s accessible via `Arborist`'s `Node.package` metadata. This allows for iterative attribute selection beyond top-level `String` evaluation. + +`Array`s specifically use a special `item` keyword in place of a typical attribute name. `Arrays` also support exact `valu`e matching when a `String` is passed to the selector. See examples below: + +#### Example of an `Object`: +```css +/* return dependencies that have a `scripts.test` containing `"tap"` */ +*:attr(:scripts([test~=tap])) +``` + +#### Example of an `Array` Attribute Selection: +```css +/* return dependencies that have a keyword that begins with "react" */ +*:attr(:keywords([item^="react"])) +``` + +#### Example of an `Array` matching directly to a value: +```css +/* return dependencies that have the exact keyword "react" */ +/* this is equivalent to `*:keywords([value="react"])` */ +*:attr(:keywords("react")) +``` + +#### Example of an `Array` of `Object`s: +```css +/* returns */ +*:attr(:contributors([email="ruyadorno@github.com"])) +``` + +### Classes + +Classes are defined by the package relationships to their ancestors (ie. the dependency types that are defined in `package.json`). This approach is user-centric as the ecosystem has been taught to think about dependencies in these classifications first-and-foremost. Dependencies are allowed to have multiple classifications (ex. a `workspace` may also be a `dev` dependency & may also be `bundled` - a selector for that type of dependency would look like: `*.workspace.dev.bundled`). + +- `.prod` +- `.dev` +- `.optional` +- `.peer` +- `.bundled` +- `.workspace` + +### Command + +#### `npm query ""` (alias `q`) + +#### Options: + +- `query-output` + - Default: `json` + - Type: `json`, `list`, `explain`, `outdated`, `funding`, `audit`, `duplicates`, `file` + +#### Example Response Output + +- an array of dependency objects is returned which can contain multiple copies of the same package which may or may not have been linked or deduped + +```json +[ + { + "name": "", + "version": "", + "description": "", + "homepage": "", + "bugs": {}, + "author": {}, + "license": {}, + "funding": {}, + "files": [], + "main": "", + "browser": "", + "bin": {}, + "man": [], + "directories": {}, + "repository": {}, + "scripts": {}, + "config": {}, + "dependencies": {}, + "devDependencies": {}, + "optionalDependencies": {}, + "bundledDependencies": {}, + "peerDependencies": {}, + "peerDependenciesMeta": {}, + "engines": {}, + "os": [], + "cpu": [], + "workspaces": {}, + "keywords": [], + "ancenstry": "", + "path": "", + "realpath": "", + "parent": "", + "vulnerabilities": [], + "cwe": [] + ... + }, + … +``` + +#### Usage + +```bash +npm query ":root > .workspace > *" # get all workspace direct deps +``` + +### Extended Example Queries & Use Cases + +```stylus +// all deps +* + +// all direct deps +:root > * + +// direct production deps +:root > .prod + +// direct development deps +:root > .dev + +// any peer dep of a direct deps +:root > * > .peer + +// any workspace dep +.workspace + +// all workspaces that depend on another workspace +.workspace > .workspace + +// all workspaces that have peer deps +.workspace:has(*.peer) + +// any dep named "lodash" +// equivalent to [name="lodash"] +#lodash + +// any deps named "lodash" & within semver range ^"1.2.3" +#lodash@^1.2.3 +// equivalent to... +[name="lodash"]:semver(^1.2.3) + +// get the hoisted node for a given semver range +#lodash@^1.2.3:not(:deduped) + +// querying deps with a specific version +#lodash@2.1.5 +// equivalent to... +[name="lodash"][version="2.1.5"] + +// all deps living alongside vulnerable deps +*:vulnerable ~ *:not(:vulnerable) + +// has any deps +*:has(*) + +// has any vulnerable deps +*:has(*:vulnerable) + +// deps with no other deps (ie. "leaf" nodes) +*:empty + +// all vulnerable deps that aren't dev deps & that aren't vulnerable to CWE-1333 +*:vulnerable:not(.dev:cwe(1333)) + +// manually querying git dependencies +*[repository^="github:"], +*[repository^="git:"], +*[repository^="https://github.com"], +*[repository^="http://github.com"], +*[repository^="https://github.com"], +*[repository^="+git:..."] + +// querying for all git dependencies +*:type(git) + +// find all references to "install" scripts +*[scripts=install], +*[scripts=postinstall], +*[scripts=preinstall] + +// get production dependencies that aren't also dev deps +.prod:not(.dev) + +// get dependencies with specific licenses +*[license="MIT"], *[license="ISC"] + +// find all packages that have @ruyadorno as a contributor +*:attr(:contributors([email=ruyadorno@github.com])) +``` + +## Next Steps + +### Command Mapping to `query-output` + +Previous commands with similar behaivours will now be able to utilize `Aborist` `Node.querySelectorAll()` under-the-hood & will fast-follow the `npm query` implementation. + +#### `npm list` + +```bash +npm list # equivalent to... +npm query ":root > *" + +npm list --all # equivalent to... +npm query "*" --query-output list + +npm list # equivalent to... +npm query "#" --query-output list +``` + +#### `npm explain` + +```bash +npm explain # equivalent to... +npm query "#" --query-output explain +``` + +#### `npm outdated` + +```bash +npm outdated # equivalent to... +npm query ":root > *:outdated" --query-output outdated + +npm outdated --all # equivalent to... +npm query "*:outdated" --query-output outdated +``` + +#### `npm fund` + +```bash +npm fund # equivalent to... +npm query ":root > *[funding]" --query-output funding + +npm fund --all # equivalent to... +npm query "*[funding]" --query-output funding + +npm fund # equivalent to... +npm query "#" --query-output funding +``` + +#### `npm audit` + +```bash +npm audit # equivalent to... +npm query ":root > *:vulnerable" --query-output audit + +npm audit --all # equivalent to... +npm query "*:vulnerable" --query-output audit + +npm audit # equivalent to... +npm query "#" --query-output audit +``` + +#### `npm find-dupes` + +```bash +npm find-dupes # equivalent to... +npm query "*:deduped" --query-output duplicates +``` + +#### `npm view` + +```bash +npm view # equivalent to... +npm query ":root" --query-output view + +npm view # equivalent to... +npm query "#" --query-output view +``` + +## Commands _could_ read from `stdin` + +In a future RFC, & major version bump, `npm` could begin reading from `stdin` to chain commands together with a common understand of a dependency object. All of the below commands would add the ability to execute as if they were passed package specs (our current defulat representation of packages/dependencies). + +``` +audit, bugs, ci, config, deprecate, diff, dist-tag, docs, edit, exec, +explain, explore, find-dupes, fund, get, install, install-ci-test, +install-test, link, list, outdated, pack, pkg, prune, publish, rebuild, +repo, restart, run-script, set, set-script, shrinkwrap, star, stars, +start, stop, team, test, token, uninstall, unpublish, unstar, update, +version, view +``` + +### Example of piping from `npm query` to other commands + +```bash +# list workspaces w/ peer deps +npm query ".workspace:has(.peer)" | npm ls + +# list outdated direct dev deps +npm query ":root > .dev:outdated" | npm outdated + +# install the same dev deps across all workspaces +npm query ":root > .dev" | npm install --workspaces + +# show audit details for dependencies with a specific vulnerability/CWE +npm query "*:cwe(1333)" | npm audit + +# show audit details for vulnerable deps that aren't ReDoS dev deps +npm query "*:vulnerable:not(.dev:cwe(1333))" | npm audit +``` + +## Prior Art + +- [`HTML`](https://html.spec.whatwg.org/) & [DOM](https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model) Specifications +- [`CSS`](https://www.w3.org/Style/CSS/specs.en.html), [Selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) & [Pseudo Class](https://developer.mozilla.org/en-US/docs/Web/CSS/Pseudo-classes) Specifications +- AST Selector Libraries/Parsers + - [`estree`](https://github.com/estree/estree) + - [`abstract-syntax-tree`](https://www.npmjs.com/package/abstract-syntax-tree) + - [`postcss-selector-parser`](https://www.npmjs.com/package/postcss-selector-parser) / [API](https://github.com/postcss/postcss-selector-parser/blob/master/API.md) + - [`css-selector-parser`](https://www.npmjs.com/package/css-selector-parser) +- [pnpm's `--filter`](https://pnpm.io/filtering) Flag +- [Gzemnid](https://github.com/nodejs/Gzemnid) Package Querying + +## F.A.Q. +- Q. Is there any such thing as a bare specifier? + - A. No. Unlike CSS Selectors, there's no `"element"`, or `"dependency"` in this context, equivalent. The selector syntax presuposes all entities are packages. +- Q. Should this syntax cover **all** possible queries of a dependency graph? + - A. No. This spec is meant to provide a sufficently mature mechanism/building block for answering the majority of questions end-users have about their depndencies (re. 80/20 rule applies here) + +## Unresolved Questions & Bikeshedding +- N/A