Skip to content

Latest commit

 

History

History
395 lines (297 loc) · 13.1 KB

0000-dependency-selector-syntax.md

File metadata and controls

395 lines (297 loc) · 13.1 KB
Raw

npm query Command & Dependency Selector Syntax

Summary

Introduce a new npm query commmand which exposes a new dependency selector syntax (informed by & respecting many aspects of the CSS Selectors 4 Spec).

Motivation

  • Standardize the shape of, & querying of, dependency graphs with a robust object model, metadata & selector syntax
  • Leverage existing, known language syntax & operators from CSS to make disparate package information broadly accessible
  • Unlock the ability to answer complex, multi-faceted questions about dependencies, their relationships & associative metadata
  • Consolidate redundant logic of similar query commands in npm (ex. npm fund, npm ls, npm outdated, npm audit ...)

Detailed Explanation

This RFC's spec & implementation should closely mimic the capabilities of existing CSS Selector specifications. Notably, we'll introduce limited net-new classes, states & syntax to ensure the widest adoption & understanding of paradigms. When deviating, we'll explicitely state why & how.

Implementation

  • Arborist's Node Class will have a new .querySelectorAll() method
    • this method will return a filtered, flattened dependency Arborist Node list based on a valid query selector
  • Introduce a new command, npm query, which will take a dependency selector & output a flattened dependency Node list (output is in json by default, but configurable)

Dependency Selector Syntax

Overview:

  • there is no "type" or "tag" selectors (ex. div, h1, a) as a dependency/target is the only type of Node that can be queried
  • the term "dependencies" is in reference to any Node found in the idealTree returned by Arborist

Selectors

  • * universal selector
  • #<name> dependency selector (equivalent to [name="..."])
  • #<name>@<version> (equivalent to [name=<name>]:semver(<version>))
  • , selector list delimiter
  • . class selector
  • : pseudo class selector
  • > direct decendent/child selector
  • ~ sibling selector

Pseudo Selectors

  • :not(<selector>)
  • :has(<selector>)
  • :where(<selector list>)
  • :is(<selector list>)
  • :root matches the root node/dependency
  • :scope matches node/dependency it was queried against
  • :empty when a dependency has no dependencies
  • :private when a dependency is private
  • :link when a dependency is linked
  • :deduped when a dependency has been deduped
  • :override when a dependency is an override
  • :extraneous when a dependency exists but is not defined as a dependency of any node
  • :invalid when a dependency version is out of its ancestors specified range
  • :missing when a dependency is not found on disk
  • :outdated when a dependency is not latest
  • :vulnerable when a dependency has a CVE
  • :semver(<spec>) matching a valid node-semver spec
  • :path(<path>) glob matching based on dependencies path relative to the project
  • :realpath(<path>) glob matching based on dependencies realpath
  • :type(<type>) based on currently recognized types

Attribute Selectors

The attribute selector evaluates the key/value pairs in package.json if they are Strings.

  • [] attribute selector (ie. existence of attribute)
  • [attribute=value] attribute value is equivalant...
  • [attribute~=value] attribute value contains word...
  • [attribute*=value] attribute value contains string...
  • [attribute|=value] attribute value is equal to or starts with...
  • [attribute^=value] attribute value begins with...
  • [attribute$=value] attribute value ends with...

Array & Object Attribute Selectors

The generic :attr() pseudo selector standardizes a pattern which can be used for attribute selection of Objects, Arrays or Arrays of Objects accessible via Arborist's Node.package metadata. This allows for iterative attribute selection beyond top-level String evaluation.

Arrays specifically use a special item keyword in place of a typical attribute name. Arrays also support exact value matching when a String is passed to the selector. See examples below:

Example of an Object:

/* return dependencies that have a `scripts.test` containing `"tap"` */
*:attr(:scripts([test~=tap]))

Example of an Array Attribute Selection:

/* return dependencies that have a keyword that begins with "react" */
*:attr(:keywords([item^="react"]))

Example of an Array matching directly to a value:

/* return dependencies that have the exact keyword "react" */
/* this is equivalent to `*:keywords([value="react"])` */
*:attr(:keywords("react"))

Example of an Array of Objects:

/* returns */
*:attr(:contributors([email="ruyadorno@github.com"]))

Classes

Classes are defined by the package relationships to their ancestors (ie. the dependency types that are defined in package.json). This approach is user-centric as the ecosystem has been taught to think about dependencies in these classifications first-and-foremost. Dependencies are allowed to have multiple classifications (ex. a workspace may also be a dev dependency & may also be bundled - a selector for that type of dependency would look like: *.workspace.dev.bundled).

  • .prod
  • .dev
  • .optional
  • .peer
  • .bundled
  • .workspace

Command

npm query "<selector>" (alias q)

Options:

  • query-output
    • Default: json
    • Type: json, list, explain, outdated, funding, audit, duplicates, file

Example Response Output

  • an array of dependency objects is returned which can contain multiple copies of the same package which may or may not have been linked or deduped
[
  {
    "name": "",
    "version": "",
    "description": "",
    "homepage": "",
    "bugs": {},
    "author": {},
    "license": {},
    "funding": {},
    "files": [],
    "main": "",
    "browser": "",
    "bin": {},
    "man": [],
    "directories": {},
    "repository": {},
    "scripts": {},
    "config": {},
    "dependencies": {},
    "devDependencies": {},
    "optionalDependencies": {},
    "bundledDependencies": {},
    "peerDependencies": {},
    "peerDependenciesMeta": {},
    "engines": {},
    "os": [],
    "cpu": [],
    "workspaces": {},
    "keywords": [],
    "ancenstry": "",
    "path": "",
    "realpath": "",
    "parent": "",
    "vulnerabilities": [],
    "cwe": []
    ...
  },
  …

Usage

npm query ":root > .workspace > *"  # get all workspace direct deps

Extended Example Queries & Use Cases

// all deps
*

// all direct deps
:root > *

// direct production deps
:root > .prod

// direct development deps
:root > .dev 

// any peer dep of a direct deps
:root > * > .peer  
  
// any workspace dep
.workspace

// all workspaces that depend on another workspace
.workspace > .workspace
    
// all workspaces that have peer deps
.workspace:has(*.peer)

// any dep named "lodash"
// equivalent to [name="lodash"]
#lodash 
  
// any deps named "lodash" & within semver range ^"1.2.3"
#lodash@^1.2.3
// equivalent to...
[name="lodash"]:semver(^1.2.3)

// get the hoisted node for a given semver range
#lodash@^1.2.3:not(:deduped)

// querying deps with a specific version
#lodash@2.1.5
// equivalent to...
[name="lodash"][version="2.1.5"]
  
// all deps living alongside vulnerable deps
*:vulnerable ~ *:not(:vulnerable)

// has any deps
*:has(*)

// has any vulnerable deps
*:has(*:vulnerable)

// deps with no other deps (ie. "leaf" nodes)
*:empty

// all vulnerable deps that aren't dev deps & that aren't vulnerable to CWE-1333
*:vulnerable:not(.dev:cwe(1333))
  
// manually querying git dependencies
*[repository^="github:"],
*[repository^="git:"],
*[repository^="https://github.com"],
*[repository^="http://github.com"],
*[repository^="https://github.com"],
*[repository^="+git:..."]
  
// querying for all git dependencies
*:type(git)

// find all references to "install" scripts
*[scripts=install], 
*[scripts=postinstall],
*[scripts=preinstall]

// get production dependencies that aren't also dev deps
.prod:not(.dev)
    
// get dependencies with specific licenses
*[license="MIT"], *[license="ISC"]

// find all packages that have @ruyadorno as a contributor
*:attr(:contributors([email=ruyadorno@github.com]))

Next Steps

Command Mapping to query-output

Previous commands with similar behaivours will now be able to utilize Aborist Node.querySelectorAll() under-the-hood & will fast-follow the npm query implementation.

npm list

npm list # equivalent to...
npm query ":root > *"

npm list --all # equivalent to...
npm query "*" --query-output list

npm list <pkg> # equivalent to...
npm query "#<pkg>" --query-output list

npm explain

npm explain <pkg> # equivalent to...
npm query "#<pkg>" --query-output explain

npm outdated

npm outdated # equivalent to...
npm query ":root > *:outdated" --query-output outdated

npm outdated --all # equivalent to...
npm query "*:outdated" --query-output outdated

npm fund

npm fund # equivalent to...
npm query ":root > *[funding]" --query-output funding

npm fund --all # equivalent to...
npm query "*[funding]" --query-output funding

npm fund <pkg> # equivalent to...
npm query "#<pkg>" --query-output funding

npm audit

npm audit # equivalent to...
npm query ":root > *:vulnerable" --query-output audit

npm audit --all # equivalent to...
npm query "*:vulnerable" --query-output audit

npm audit <pkg> # equivalent to...
npm query "#<pkg>" --query-output audit

npm find-dupes

npm find-dupes # equivalent to...
npm query "*:deduped" --query-output duplicates

npm view

npm view # equivalent to...
npm query ":root" --query-output view

npm view <pkg> # equivalent to...
npm query "#<pkg>" --query-output view

Commands could read from stdin

In a future RFC, & major version bump, npm could begin reading from stdin to chain commands together with a common understand of a dependency object. All of the below commands would add the ability to execute as if they were passed package specs (our current defulat representation of packages/dependencies).

audit, bugs, ci, config, deprecate, diff, dist-tag, docs, edit, exec, 
explain, explore, find-dupes, fund, get, install, install-ci-test, 
install-test, link, list, outdated, pack, pkg, prune, publish, rebuild, 
repo, restart, run-script, set, set-script, shrinkwrap, star, stars,
start, stop, team, test, token, uninstall, unpublish, unstar, update,
version, view

Example of piping from npm query to other commands

# list workspaces w/ peer deps
npm query ".workspace:has(.peer)" | npm ls

# list outdated direct dev deps
npm query ":root > .dev:outdated" | npm outdated

# install the same dev deps across all workspaces
npm query ":root > .dev" | npm install --workspaces

# show audit details for dependencies with a specific vulnerability/CWE
npm query "*:cwe(1333)" | npm audit

# show audit details for vulnerable deps that aren't ReDoS dev deps
npm query "*:vulnerable:not(.dev:cwe(1333))" | npm audit

Prior Art

F.A.Q.

  • Q. Is there any such thing as a bare specifier?
    • A. No. Unlike CSS Selectors, there's no "element", or "dependency" in this context, equivalent. The selector syntax presuposes all entities are packages.
  • Q. Should this syntax cover all possible queries of a dependency graph?
    • A. No. This spec is meant to provide a sufficently mature mechanism/building block for answering the majority of questions end-users have about their depndencies (re. 80/20 rule applies here)

Unresolved Questions & Bikeshedding

  • N/A