diff --git a/.changeset/silver-spies-begin.md b/.changeset/silver-spies-begin.md new file mode 100644 index 000000000..c28cb3af6 --- /dev/null +++ b/.changeset/silver-spies-begin.md @@ -0,0 +1,14 @@ +--- +"skott": minor +--- + +Allow unused files to be tracked and reported. From the CLI, `--showUnusedFiles` can be used to report unused files. From the API, a new `collectUnusedFiles` method is accessible through the graph API: + +```js +import skott from "skott"; + +const instance = await skott(); +const unusedFiles = instance.useGraph().collectUnusedFiles(); +``` + +This version also includes a fix for a bug related to `--trackBuiltinDependencies` and `--trackThirdPartyDependencies` that were not propagated anymore (since 0.34.0) when being provided from the CLI. \ No newline at end of file diff --git a/README.md b/README.md index 77d138d67..ad1e90f03 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,11 @@ ✅ Deeply detects **circular dependencies** in an efficient way, with the ability to provide a max depth for the search -✅ Many **builtin visualization modes** including a web application or terminal-based outputs such as file-tree or graph views. Visualization modes can be rendered using both the CLI and API. +✅ Detect **unused source code files**. Eliminate dead code by finding files not imported anywhere else in the graph. + +✅ Detect **unused npm third-party dependencies**. Note that all unused `devDependencies` are not guaranteed to be detected as `depcheck` [only provides analysis for set of supported libraries](https://github.com/depcheck/depcheck) (eslint, karma, mocha, etc). + +✅ Many **builtin visualization modes** including a web application or terminal-based outputs such as file-tree or graph views. Visualization modes can be rendered using both the CLI and programatically using the API. ✅ Builtin **watch mode** updating the graph when file changes are detected. It works with all display modes (webapp and all CLIs visualization modes). Support all options of file ignoring/filtering from skott. @@ -31,11 +35,9 @@ ✅ Works with any custom **dependency resolver** (useful for specific monorepos integration where module identifiers need to be mapped to a specific workspace package) -✅ Detect **unused npm third-party dependencies**. Note that all unused `devDependencies` are not guaranteed to be detected as `depcheck` [only provides analysis for set of supported libraries](https://github.com/depcheck/depcheck) (eslint, karma, mocha, etc). - ✅ Deeply **collect all dependencies of the project graph**, including third-party and builtin. -✅ Deep **parent and child dependencies traversals** using DFS and BFS algorithms. +✅ Graph API including deep **parent and child dependencies traversals** using DFS and BFS algorithms. ✅ Metadata collection per traversed node (file size, dependencies) @@ -158,8 +160,9 @@ _Dead code_ can be defined as a code literally having no impact on the applicati However, tree shaking is not an easy task and can mostly work with module systems using static-based imports/exports such as ECMAScript modules. To avoid removing code that appears to be used at runtime, module bundlers are being very precise about determining automatically chunks of code that can be safely removed. Module bundlers can also be helped by providing them manually clues about what can be safely removed e.g. `/*#__PURE__*/` for Webpack. -If you're not using tools implementing tree shaking, you will be able soon to use **skott**, which will bring up soon unused imports/exports warnings 🚀 +Also, bundling might not be possible or might not even be a target. In that context, it's even more important to care about dead code elimination. Dead code can harm cold start and have unwanted side-effects. +**skott** exposes information that can help identifying dead code and getting rid of it. Check documentation to get more information about identifying unused files and dependencies. ## Graph Management diff --git a/packages/skott/README.md b/packages/skott/README.md index 91a921111..954f7f306 100644 --- a/packages/skott/README.md +++ b/packages/skott/README.md @@ -157,6 +157,14 @@ Using this command, skott will deeply search for all ".ts" and ".tsx" files star $ skott --fileExtensions=.ts,.tsx ``` +**Finding unused files and dependencies:** + +```bash +$ skott --showUnusedFiles --showUnusedDependencies --trackThirdPartyDependencies +``` + +An important description of that feature is available below in the API section. + **skott** offers many ways to visualize the generated graph. **Embedded Web Application** @@ -246,6 +254,7 @@ const { traverseFiles, collectFilesDependencies, collectFilesDependingOn, + collectUnusedFiles, findLeaves, findCircularDependencies, hasCircularDependencies  @@ -258,6 +267,8 @@ const { const { useGraph } = await skott(); const { traverseFiles } = useGraph(); +const unusedFiles = collectUnusedFiles(); + // Starting from any node, walking the whole graph for(const file of traverseFiles()) { // SkottNode { } @@ -353,6 +364,23 @@ console.log(collectFilesDependencies("parent.js", CollectLevel.Shallow)); // logs [ SkottNode { id: "children.js" } ] ``` +### Find unused files + +skott provides a way to collect **unused** files. Files are marked as "unused" from a pure source code analysis standpoint, meaning that a given file is considered unused only if it is not importing any other file and there is no other file importing it. In the graph lingo, we refer to these nodes as **isolated nodes**. + +Note: having a file being marked as unused does not necessarily mean that this file is useless, but rather than skott didn't find any use of it when traversing the whole project graph. Sometimes files are being exported as a npm library entrypoint even though they are not used in the internals of that library (for instance `package.json#exports`), or sometimes files are being used by other tools being run from npm scripts or whatever else toolchain. + +Unlike `unused dependencies` shown below, `unused files` don't need further analysis or need additional context e.g. a manifest file (package.json for Node.js) to be determined. This is why they belong in the Graph API, as `unused files` are nothing but `isolated nodes` in the context of skott. + +```javascript +import skott from "skott"; + +const { useGraph } = await skott(); + +const unusedFiles = useGraph().collectUnusedFiles(); +// [ "index.js", "some-other-file.ts", "else.js" ] +``` + ### Find unused dependencies skott provides a way to walk through dependencies listed in the current working directory manifest (package.json) and compare them to what it founds and marked as "used" during the analysis. The "use" marking will be done when a third-party module appears to be imported in the source code that was walked. All the third-party dependencies that are not used in the traversed files will be returned as "unused". diff --git a/packages/skott/bin/cli.ts b/packages/skott/bin/cli.ts index ab221b0b0..be5d52841 100644 --- a/packages/skott/bin/cli.ts +++ b/packages/skott/bin/cli.ts @@ -154,6 +154,11 @@ cli "Search for unused third-party dependencies in the graph", false ) + .option( + "-uf, --showUnusedFiles", + "Search for unused files in the graph", + false + ) .option("-vb, --verbose", "Enable verbose mode. Display all the logs", false) .option( "-w, --cwd ", diff --git a/packages/skott/src/graph/api.ts b/packages/skott/src/graph/api.ts new file mode 100644 index 000000000..c4952fb87 --- /dev/null +++ b/packages/skott/src/graph/api.ts @@ -0,0 +1,144 @@ +import type { DiGraph } from "digraph-js"; + +import type { SkottConfig } from "../skott.js"; + +import type { SkottNode } from "./node.js"; + +export const CollectLevel = { + Deep: "deep", + Shallow: "shallow" +} as const; + +export type CollectLevelValues = + (typeof CollectLevel)[keyof typeof CollectLevel]; + +const skottToDiGraphTraversal = { + deepFirst: "dfs", + shallowFirst: "bfs" +} as const; + +export class GraphApi { + constructor( + private readonly graph: DiGraph>, + private readonly config: SkottConfig + ) { + this.getFileNode = this.getFileNode.bind(this); + this.getNodes = this.getNodes.bind(this); + this.traverseFiles = this.traverseFiles.bind(this); + this.collectFilesDependencies = this.collectFilesDependencies.bind(this); + this.collectFilesDependingOn = this.collectFilesDependingOn.bind(this); + this.collectUnusedFiles = this.collectUnusedFiles.bind(this); + this.hasCircularDependencies = this.hasCircularDependencies.bind(this); + this.findCircularDependencies = this.findCircularDependencies.bind(this); + this.findLeaves = this.findLeaves.bind(this); + } + + getFileNode(id: string): SkottNode { + return this.getNodes()[id]; + } + + getNodes() { + return this.graph.toDict(); + } + + *traverseFiles(options?: { + rootFile?: string; + moduleImportsCollection?: "deepFirst" | "shallowFirst"; + }): Generator, void, void> { + const rootNode = options?.rootFile; + const moduleImportsCollection = + options?.moduleImportsCollection ?? "shallowFirst"; + + const traversal = skottToDiGraphTraversal[moduleImportsCollection]; + + if (rootNode) { + return yield* this.graph.traverse({ + rootVertexId: rootNode, + traversal + }); + } + + return yield* this.graph.traverse({ + traversal + }); + } + + collectFilesDependencies( + rootFile: string, + collectLevel: CollectLevelValues + ): SkottNode[] { + if (collectLevel === CollectLevel.Shallow) { + return this.graph.getChildren(rootFile); + } + + const nodes = this.getNodes(); + const childrenIds = Array.from(this.graph.getDeepChildren(rootFile)); + const dependencies = []; + + for (const id of childrenIds) { + dependencies.push(nodes[id]); + } + + return dependencies; + } + + collectFilesDependingOn( + rootFile: string, + collectLevel: CollectLevelValues + ): SkottNode[] { + if (collectLevel === CollectLevel.Shallow) { + return this.graph.getParents(rootFile); + } + + const nodes = this.getNodes(); + const parentIds = Array.from(this.graph.getDeepParents(rootFile)); + const dependingOn = []; + + for (const id of parentIds) { + dependingOn.push(nodes[id]); + } + + return dependingOn; + } + + collectUnusedFiles(): Array { + const leaves = this.findLeaves(); + const unused = []; + + for (const leaf of leaves) { + const node = this.getFileNode(leaf); + const noNodesDependingOn = + this.collectFilesDependingOn(leaf, CollectLevel.Deep).length === 0; + if (noNodesDependingOn) { + unused.push(node.id); + } + } + + return unused; + } + + hasCircularDependencies(): boolean { + return this.graph.hasCycles({ + maxDepth: this.config.circularMaxDepth ?? Number.POSITIVE_INFINITY + }); + } + + findCircularDependencies(): Array> { + return this.graph.findCycles({ + maxDepth: this.config.circularMaxDepth ?? Number.POSITIVE_INFINITY + }); + } + + findLeaves(): Array { + const nodes = this.getNodes(); + const leaves = []; + + for (const node of Object.values(nodes)) { + if (node.adjacentTo.length === 0) { + leaves.push(node.id); + } + } + + return leaves; + } +} diff --git a/packages/skott/src/graph/traversal.ts b/packages/skott/src/graph/traversal.ts deleted file mode 100644 index 05498268c..000000000 --- a/packages/skott/src/graph/traversal.ts +++ /dev/null @@ -1,101 +0,0 @@ -import type { DiGraph } from "digraph-js"; - -import type { SkottConfig } from "../skott.js"; - -import type { SkottNode } from "./node.js"; - -export const CollectLevel = { - Deep: "deep", - Shallow: "shallow" -} as const; - -export type CollectLevelValues = - (typeof CollectLevel)[keyof typeof CollectLevel]; - -export interface TraversalApi { - getFileNode(id: string): SkottNode; - traverseFiles: (options?: { - rootFile?: string; - moduleImportsCollection?: "deepFirst" | "shallowFirst"; - }) => Generator, void, void>; - collectFilesDependencies: ( - rootFile: string, - level: CollectLevelValues - ) => SkottNode[]; - collectFilesDependingOn: ( - rootFile: string, - level: CollectLevelValues - ) => SkottNode[]; - findLeaves: () => string[]; - findCircularDependencies: () => string[][]; - hasCircularDependencies: () => boolean; -} - -const skottToDiGraphTraversal = { - deepFirst: "dfs", - shallowFirst: "bfs" -} as const; - -export function makeTraversalApi( - graph: DiGraph>, - config: SkottConfig -): TraversalApi { - const nodes = graph.toDict(); - - return { - getFileNode: (id) => nodes[id], - - *traverseFiles(options) { - const rootNode = options?.rootFile; - const moduleImportsCollection = - options?.moduleImportsCollection ?? "shallowFirst"; - - const traversal = skottToDiGraphTraversal[moduleImportsCollection]; - - if (rootNode) { - return yield* graph.traverse({ - rootVertexId: rootNode, - traversal - }); - } - - return yield* graph.traverse({ - traversal - }); - }, - - collectFilesDependencies: (rootFile, collectLevel) => { - if (collectLevel === CollectLevel.Shallow) { - return graph.getChildren(rootFile); - } - - return Array.from(graph.getDeepChildren(rootFile)).map((id) => nodes[id]); - }, - - collectFilesDependingOn: (rootFile, collectLevel) => { - if (collectLevel === CollectLevel.Shallow) { - return graph.getParents(rootFile); - } - - return Array.from(graph.getDeepParents(rootFile)).map((id) => nodes[id]); - }, - - hasCircularDependencies(): boolean { - return graph.hasCycles({ - maxDepth: config.circularMaxDepth ?? Number.POSITIVE_INFINITY - }); - }, - - findCircularDependencies(): SkottNode["id"][][] { - return graph.findCycles({ - maxDepth: config.circularMaxDepth ?? Number.POSITIVE_INFINITY - }); - }, - - findLeaves(): SkottNode["id"][] { - return Object.entries(nodes) - .filter(([_, node]) => node.adjacentTo.length === 0) - .map(([leafId]) => leafId); - } - }; -} diff --git a/packages/skott/src/rendering/config.ts b/packages/skott/src/rendering/config.ts new file mode 100644 index 000000000..8db8b2b1f --- /dev/null +++ b/packages/skott/src/rendering/config.ts @@ -0,0 +1,15 @@ +import kleur from "kleur"; + +import type { InputConfig, RuntimeConfig } from "../config.js"; +import { createRuntimeConfig } from "../instance.js"; + +export function toRuntimeConfigOrDie(input: InputConfig): RuntimeConfig { + try { + return createRuntimeConfig(input); + } catch (error) { + // @ts-expect-error + console.log(`\n ${kleur.bold().red(error.message)}`); + + return process.exit(1); + } +} diff --git a/packages/skott/src/rendering/terminal/api.ts b/packages/skott/src/rendering/terminal/api.ts index 4ed18d8b5..d3ce06f4e 100644 --- a/packages/skott/src/rendering/terminal/api.ts +++ b/packages/skott/src/rendering/terminal/api.ts @@ -1,7 +1,7 @@ import kleur from "kleur"; import type { InputConfig } from "../../config.js"; -import { createRuntimeConfig } from "../../instance.js"; +import { toRuntimeConfigOrDie } from "../config.js"; import { runTerminal } from "./runner.js"; import { makeSkottRunner } from "./runner.js"; @@ -23,7 +23,8 @@ export function renderTerminalApplication( apiConfig: InputConfig, options: TerminalConfig = defaultTerminalConfig ): Promise { - const runtimeConfig = createRuntimeConfig(apiConfig); + const runtimeConfig = toRuntimeConfigOrDie(apiConfig); + const terminalOptions: TerminalConfig = { watch: options.watch ?? defaultTerminalConfig.watch, displayMode: options.displayMode ?? defaultTerminalConfig.displayMode, @@ -33,9 +34,15 @@ export function renderTerminalApplication( defaultTerminalConfig.showCircularDependencies, showUnusedDependencies: options.showUnusedDependencies ?? - defaultTerminalConfig.showUnusedDependencies + defaultTerminalConfig.showUnusedDependencies, + showUnusedFiles: + options.showUnusedFiles ?? defaultTerminalConfig.showUnusedFiles }; - const isTerminalConfigValid = ensureNoIllegalTerminalConfig(terminalOptions); + + const isTerminalConfigValid = ensureNoIllegalTerminalConfig(terminalOptions, { + entrypoint: runtimeConfig.entrypoint, + trackThirdPartyDependencies: runtimeConfig.dependencyTracking.thirdParty + }); if (isTerminalConfigValid._tag === "Left") { console.log(`\n ${kleur.bold().red(isTerminalConfigValid.left)}`); diff --git a/packages/skott/src/rendering/terminal/internal.ts b/packages/skott/src/rendering/terminal/internal.ts index 5b38346e6..13d7e0f5d 100644 --- a/packages/skott/src/rendering/terminal/internal.ts +++ b/packages/skott/src/rendering/terminal/internal.ts @@ -1,7 +1,7 @@ import kleur from "kleur"; -import { createRuntimeConfig } from "../../instance.js"; import { kExpectedModuleExtensions } from "../../modules/resolvers/base-resolver.js"; +import { toRuntimeConfigOrDie } from "../config.js"; import { runTerminal } from "./runner.js"; import { makeSkottRunner } from "./runner.js"; @@ -23,6 +23,7 @@ export type CliParameterOptions = { manifest: string; showCircularDependencies: boolean; showUnusedDependencies: boolean; + showUnusedFiles: boolean; trackBuiltinDependencies: boolean; trackThirdPartyDependencies: boolean; trackTypeOnlyDependencies: boolean; @@ -35,20 +36,34 @@ export function runTerminalApplicationFromCLI( entrypoint: string | undefined, options: CliParameterOptions ): Promise { - const isTerminalConfigValid = ensureNoIllegalTerminalConfig(options); + const isTerminalConfigValid = ensureNoIllegalTerminalConfig(options, { + entrypoint, + trackThirdPartyDependencies: options.trackThirdPartyDependencies + }); if (isTerminalConfigValid._tag === "Left") { console.log(`\n ${kleur.bold().red(isTerminalConfigValid.left)}`); process.exit(1); } - const runtimeConfig = createRuntimeConfig({ - ...options, + const runtimeConfig = toRuntimeConfigOrDie({ entrypoint, ignorePatterns: options.ignorePattern, + dependencyTracking: { + thirdParty: options.trackThirdPartyDependencies, + builtin: options.trackBuiltinDependencies, + typeOnly: options.trackTypeOnlyDependencies + }, fileExtensions: options.fileExtensions .split(",") - .filter((ext) => kExpectedModuleExtensions.has(ext)) + .filter((ext) => kExpectedModuleExtensions.has(ext)), + circularMaxDepth: options.circularMaxDepth, + tsConfigPath: options.tsconfig, + manifestPath: options.manifest, + includeBaseDir: options.includeBaseDir, + cwd: options.cwd, + incremental: options.incremental, + verbose: options.verbose }); const runSkott = makeSkottRunner(runtimeConfig); diff --git a/packages/skott/src/rendering/terminal/runner.ts b/packages/skott/src/rendering/terminal/runner.ts index bc300d3dc..5a8b77800 100644 --- a/packages/skott/src/rendering/terminal/runner.ts +++ b/packages/skott/src/rendering/terminal/runner.ts @@ -206,6 +206,7 @@ export async function runTerminal( await new Promise((resolve) => { const depsReportComponent = new CliComponent(() => displayDependenciesReport(skottInstance, { + showUnusedFiles: terminalOptions.showUnusedFiles, showUnusedDependencies: terminalOptions.showUnusedDependencies, trackBuiltinDependencies: runtimeConfig.dependencyTracking.builtin, trackThirdPartyDependencies: diff --git a/packages/skott/src/rendering/terminal/terminal-config.ts b/packages/skott/src/rendering/terminal/terminal-config.ts index b88444f55..b994bda09 100644 --- a/packages/skott/src/rendering/terminal/terminal-config.ts +++ b/packages/skott/src/rendering/terminal/terminal-config.ts @@ -6,6 +6,7 @@ export interface TerminalConfig { displayMode: "raw" | "file-tree" | "graph" | "webapp"; showCircularDependencies: boolean; showUnusedDependencies: boolean; + showUnusedFiles: boolean; exitCodeOnCircularDependencies: number; } @@ -14,6 +15,7 @@ export const defaultTerminalConfig: TerminalConfig = { displayMode: "raw", showCircularDependencies: false, showUnusedDependencies: false, + showUnusedFiles: false, exitCodeOnCircularDependencies: 1 }; @@ -30,14 +32,30 @@ const terminalSchema = D.struct({ exitCodeOnCircularDependencies: D.number }); -export function ensureNoIllegalTerminalConfig(options: TerminalConfig) { - const result = terminalSchema.decode(options); +export function ensureNoIllegalTerminalConfig( + terminalConfig: TerminalConfig, + apiConfig: { + entrypoint: string | undefined; + trackThirdPartyDependencies: boolean; + } +) { + const result = terminalSchema.decode(terminalConfig); if (result._tag === "Left") { return Either.left( `Invalid terminal configuration: ${D.draw(result.left)}` ); } + + if ( + terminalConfig.showUnusedDependencies && + !apiConfig.trackThirdPartyDependencies + ) { + return Either.left( + "`--trackThirdPartyDependencies` must be provided when searching for unused dependencies." + ); + } + /** * Some `show*` params exist but are only relevant when using CLI-based. * `--showCircularDependencies` is already supported in the webapp even without @@ -45,25 +63,36 @@ export function ensureNoIllegalTerminalConfig(options: TerminalConfig) { * `--showUnusedDependencies` is not supported in the webapp yet but to enforce * consistency, we don't allow it to be used with `--displayMode=webapp`. */ - if (options.displayMode === "webapp") { - if (options.showCircularDependencies) { + if (terminalConfig.displayMode === "webapp") { + if (terminalConfig.showCircularDependencies) { return Either.left( "`--showCircularDependencies` can't be used when using `--displayMode=webapp`" ); } - if (options.showUnusedDependencies) { + if (terminalConfig.showUnusedDependencies) { return Either.left( "`--showUnusedDependencies` can't be used when using `--displayMode=webapp`" ); } + if (terminalConfig.showUnusedFiles) { + return Either.left( + "`--showUnusedFiles` can't be used when using `--displayMode=webapp`" + ); + } + } + + if (apiConfig.entrypoint && terminalConfig.showUnusedFiles) { + return Either.left( + "`--showUnusedFiles` can't be used when using providing an entrypoint." + ); } - if (options.watch) { + if (terminalConfig.watch) { if ( - options.displayMode !== "webapp" && - options.displayMode !== "graph" && - options.displayMode !== "raw" && - options.displayMode !== "file-tree" + terminalConfig.displayMode !== "webapp" && + terminalConfig.displayMode !== "graph" && + terminalConfig.displayMode !== "raw" && + terminalConfig.displayMode !== "file-tree" ) { return Either.left( "`--watch` needs either `raw`, `file-tree`, `graph` or `webapp` display mode" diff --git a/packages/skott/src/rendering/terminal/ui/console/dependencies.ts b/packages/skott/src/rendering/terminal/ui/console/dependencies.ts index b197abe35..a1e631383 100644 --- a/packages/skott/src/rendering/terminal/ui/console/dependencies.ts +++ b/packages/skott/src/rendering/terminal/ui/console/dependencies.ts @@ -32,7 +32,9 @@ export async function displayThirdPartyDependencies( ); } - console.log(`\n production third-party dependencies: \n`); + console.log( + "\n" + kleur.bold().white().underline("Third-party dependencies:") + "\n" + ); const sortedDependencies = [...thirdPartyRegistry.entries()].sort( ([a], [b]) => { @@ -60,15 +62,6 @@ export async function displayThirdPartyDependencies( const indents = makeIndents(1); if (thirdParty.length > 0) { console.log( - `${kleur.bold( - `\n Found ${kleur - .bold() - .red( - thirdParty.length - )} third-party dependencies (dev/prod) that ${kleur.yellow( - "might be unused" - )}` - )} (${kleur.bold().magenta(timeTook)}) \n`, `\n ${kleur .bold() .grey( @@ -78,11 +71,21 @@ export async function displayThirdPartyDependencies( " source files before removing them. You might also want to move them as 'devDependencies'" + " if that is the case." )} - ` + `, + `${kleur.bold( + `\n Found ${kleur + .bold() + .red( + thirdParty.length + )} third-party dependencies (dev/prod) that ${kleur.yellow( + "might be unused (check message just above)" + )}` + )} (${kleur.bold().magenta(timeTook)}) \n` ); for (const dep of thirdParty) { - console.log(`${indents} ${kleur.bold().red(dep)}`); + const d = `➡️ ${dep}`; + console.log(`${indents} ${kleur.bold().red(d)}`); } } else { console.log( @@ -123,7 +126,11 @@ export function displayBuiltinDependencies( return; } - console.log(`\n Builtin Node.js dependencies: \n`); + console.log( + "\n" + + kleur.bold().white().underline("Builtin Node.js dependencies:") + + "\n" + ); for (const [depName, depOccurrence] of builtinRegistry.entries()) { console.log( @@ -173,28 +180,11 @@ export async function displayDependenciesReport( skottInstance: SkottInstance, options: { showUnusedDependencies: boolean; + showUnusedFiles: boolean; trackThirdPartyDependencies: boolean; trackBuiltinDependencies: boolean; } ) { - if (options.showUnusedDependencies && !options.trackThirdPartyDependencies) { - console.log( - `\n ${kleur - .bold() - .yellow( - "Warning: `--trackThirdPartyDependencies` must be provided when searching for unused dependencies." - )}` - ); - - console.log( - `\n ${kleur - .bold() - .grey( - "Example: `skott --displayMode=raw --showUnusedDependencies --trackThirdPartyDependencies`" - )} \n` - ); - } - const { graph } = skottInstance.getStructure(); if (options.trackThirdPartyDependencies) { @@ -208,6 +198,10 @@ export async function displayDependenciesReport( if (options.trackBuiltinDependencies) { displayBuiltinDependencies(graph); } + + if (options.showUnusedFiles) { + displayUnusedFiles(skottInstance); + } } export function displayCircularDependencies( @@ -260,3 +254,27 @@ export function displayCircularDependencies( return circularDependencies; } + +export function displayUnusedFiles(skottInstance: SkottInstance) { + const unusedFiles = skottInstance.useGraph().collectUnusedFiles(); + + if (unusedFiles.length > 0) { + console.log("\n " + kleur.bold().white().underline(`Unused files found:`)); + + for (const unused of unusedFiles) { + console.log(kleur.bold().red(`\n ➡️ ${unused}`)); + } + + console.log( + kleur + .bold() + .grey( + `\n Warning: files are marked as unused based on source code analysis.` + + `\n They might be exported through package.json or used through other mechanisms not analyzed by skott.` + + `\n Please double check their potential use before removing them.` + ) + ); + } else { + console.log(kleur.bold().green(`\n 🧹 no unused files found`)); + } +} diff --git a/packages/skott/src/rendering/terminal/ui/renderers/webapp.ts b/packages/skott/src/rendering/terminal/ui/renderers/webapp.ts index 3b57668e0..f96b280ec 100644 --- a/packages/skott/src/rendering/terminal/ui/renderers/webapp.ts +++ b/packages/skott/src/rendering/terminal/ui/renderers/webapp.ts @@ -105,6 +105,20 @@ export function renderWebApplication(config: { response.end(JSON.stringify(cycles)); }); + app.get("/api/unused", async (_, response) => { + const unusedDependencies = + await getSkottInstance().findUnusedDependencies(); + const unusedFiles = getSkottInstance().useGraph().collectUnusedFiles(); + + response.setHeader("Content-Type", "application/json"); + response.end( + JSON.stringify({ + dependencies: unusedDependencies, + files: unusedFiles + }) + ); + }); + app.get("/api/analysis", (_, response) => { const structure = getSkottInstance().getStructure(); @@ -118,5 +132,16 @@ export function renderWebApplication(config: { ); }); + app.get("/api/meta", (_, response) => { + // When the webapp is registered from the CLI, the only default option is "group" + const meta = { + visualization: "group", + tracking: dependencyTracking + }; + + response.setHeader("Content-Type", "application/json"); + response.end(JSON.stringify(meta)); + }); + listen({ autoOpen: true }); } diff --git a/packages/skott/src/rendering/webapp/api.ts b/packages/skott/src/rendering/webapp/api.ts index 30c082ce2..49cef7e08 100644 --- a/packages/skott/src/rendering/webapp/api.ts +++ b/packages/skott/src/rendering/webapp/api.ts @@ -1,8 +1,9 @@ import EventEmitter from "events"; import type { InputConfig } from "../../config.js"; -import { createRuntimeConfig, runFromRuntimeConfig } from "../../instance.js"; +import { runFromRuntimeConfig } from "../../instance.js"; import type { SkottInstance, SkottStructure } from "../../skott.js"; +import { toRuntimeConfigOrDie } from "../config.js"; import { registerWatchMode } from "../watch-mode.js"; import { createHttpApp, resolveEntrypointPath } from "./internal.js"; @@ -44,7 +45,7 @@ export async function renderWebApplication( onOpenError?: (error: Error) => void; } ) { - const runtimeConfig = createRuntimeConfig(apiConfig); + const runtimeConfig = toRuntimeConfigOrDie(apiConfig); const entrypoint = resolveEntrypointPath({ entrypoint: runtimeConfig.entrypoint, includeBaseDir: runtimeConfig.includeBaseDir @@ -114,6 +115,19 @@ export async function renderWebApplication( response.end(JSON.stringify(cycles)); }); + app.get("/api/unused", async (_, response) => { + const unusedDependencies = await skottInstance.findUnusedDependencies(); + const unusedFiles = skottInstance.useGraph().collectUnusedFiles(); + + response.setHeader("Content-Type", "application/json"); + response.end( + JSON.stringify({ + dependencies: unusedDependencies, + files: unusedFiles + }) + ); + }); + app.get("/api/analysis", (_, response) => { const structure = skottInstance.getStructure(); @@ -128,7 +142,10 @@ export async function renderWebApplication( }); app.get("/api/meta", (_, response) => { - const meta = { visualization }; + const meta = { + visualization, + tracking: runtimeConfig.dependencyTracking + }; response.setHeader("Content-Type", "application/json"); response.end(JSON.stringify(meta)); @@ -237,7 +254,20 @@ export async function renderStandaloneWebApplication( response.end(JSON.stringify(cycles)); }); - app.get("/api/analysis", (_, response) => { + app.get("/api/unused", async (_, response) => { + const unusedDependencies = await skottInstance.findUnusedDependencies(); + const unusedFiles = skottInstance.useGraph().collectUnusedFiles(); + + response.setHeader("Content-Type", "application/json"); + response.end( + JSON.stringify({ + dependencies: unusedDependencies, + files: unusedFiles + }) + ); + }); + + app.get("/api/analysis", async (_, response) => { const structure = skottInstance.getStructure(); response.setHeader("Content-Type", "application/json"); diff --git a/packages/skott/src/skott.ts b/packages/skott/src/skott.ts index af7b01f64..759a9c285 100644 --- a/packages/skott/src/skott.ts +++ b/packages/skott/src/skott.ts @@ -12,8 +12,8 @@ import { import { FileReader } from "./filesystem/file-reader.js"; import type { FileWriter } from "./filesystem/file-writer.js"; import { toUnixPathLike } from "./filesystem/path.js"; +import { GraphApi } from "./graph/api.js"; import type { SkottNode } from "./graph/node.js"; -import { type TraversalApi, makeTraversalApi } from "./graph/traversal.js"; import { highlight, logFailureM, @@ -116,7 +116,7 @@ export interface ImplicitUnusedDependenciesOptions { } export interface SkottInstance { - useGraph: () => TraversalApi; + useGraph: () => GraphApi; getStructure: () => SkottStructure; getWorkspace: () => ManifestDependenciesByName; findUnusedDependencies: ( @@ -495,6 +495,14 @@ export class Skott { } } ): Promise { + if (!this.config.dependencyTracking.thirdParty) { + this.logger.info( + `Skipping finding unused third-party dependencies as they are not tracked by the analysis.` + ); + + return { thirdParty: [] }; + } + const manifestDependencies = await pipe( findManifestDependencies(this.#baseDir, this.config.manifestPath), Effect.provideService(FileReader, this.fileReader), @@ -767,7 +775,7 @@ export class Skott { } return { - useGraph: () => makeTraversalApi(this.#projectGraph, this.config), + useGraph: () => new GraphApi(this.#projectGraph, this.config), getStructure: this.makeProjectStructure.bind(this), getWorkspace: () => this.#workspaceConfiguration.manifests, findUnusedDependencies: this.findUnusedDependencies.bind(this) diff --git a/packages/skott/test/benchmark/result-node-18.x.json b/packages/skott/test/benchmark/result-node-18.x.json index 3454f26fe..a9d3d2d82 100644 --- a/packages/skott/test/benchmark/result-node-18.x.json +++ b/packages/skott/test/benchmark/result-node-18.x.json @@ -1,40 +1,40 @@ { "name": "skott_benchmark", - "done_at": "2024-06-15T06:46:42.994Z", - "git_commit_hash": "e5b398675c80867b776a3b4fb626b27a7c3b52a0", - "git_branch_reference": "support-multiple-ignore-patterns", + "done_at": "2024-06-15T16:09:05.376Z", + "git_commit_hash": "dd7e66babc49b71afc70142074f0fc1d09ce5db4", + "git_branch_reference": "feat-isolated-nodes", "results": [ { "name": "knex/knex", "rank": 1, - "totalTime": 7856.093021999986, - "min": 1543.816434999986, - "mean": 1571.2186043999973, - "max": 1598.8641500000085 + "totalTime": 7615.61580499995, + "min": 1499.4296109999996, + "mean": 1523.12316099999, + "max": 1545.6813479999837 }, { "name": "parcel-bundler/parcel", "rank": 3, - "totalTime": 28688.497830000008, - "min": 5693.05104999998, - "mean": 5737.699566000001, - "max": 5791.180710000015 + "totalTime": 28911.505858999968, + "min": 5636.564931000001, + "mean": 5782.301171799993, + "max": 5929.7201230000355 }, { "name": "Effect-TS/effect", "rank": 4, - "totalTime": 66600.87857700011, - "min": 13158.308263000043, - "mean": 13320.175715400022, - "max": 13496.371823000023 + "totalTime": 64159.772661999974, + "min": 12708.475534000027, + "mean": 12831.954532399996, + "max": 12930.35306699999 }, { "name": "gcanti/fp-ts", "rank": 2, - "totalTime": 22328.540177999937, - "min": 4433.088367999997, - "mean": 4465.708035599988, - "max": 4501.924900999991 + "totalTime": 21564.079151000013, + "min": 4288.3607329999795, + "mean": 4312.815830200003, + "max": 4341.06635800004 } ] } \ No newline at end of file diff --git a/packages/skott/test/benchmark/result-node-20.x.json b/packages/skott/test/benchmark/result-node-20.x.json index fb430c87a..4b1b59275 100644 --- a/packages/skott/test/benchmark/result-node-20.x.json +++ b/packages/skott/test/benchmark/result-node-20.x.json @@ -1,40 +1,40 @@ { "name": "skott_benchmark", - "done_at": "2024-06-15T06:46:25.854Z", - "git_commit_hash": "e5b398675c80867b776a3b4fb626b27a7c3b52a0", - "git_branch_reference": "support-multiple-ignore-patterns", + "done_at": "2024-06-15T16:09:04.355Z", + "git_commit_hash": "dd7e66babc49b71afc70142074f0fc1d09ce5db4", + "git_branch_reference": "feat-isolated-nodes", "results": [ { "name": "knex/knex", "rank": 1, - "totalTime": 7429.507001000004, - "min": 1433.0685569999987, - "mean": 1485.9014002000008, - "max": 1537.9539349999995 + "totalTime": 7505.696782000003, + "min": 1466.6123599999992, + "mean": 1501.1393564000005, + "max": 1548.2739910000018 }, { "name": "parcel-bundler/parcel", "rank": 3, - "totalTime": 28467.876101999987, - "min": 5594.911652999988, - "mean": 5693.575220399997, - "max": 5801.409022 + "totalTime": 28102.342516000004, + "min": 5546.889507, + "mean": 5620.468503200001, + "max": 5686.298540000003 }, { "name": "Effect-TS/effect", "rank": 4, - "totalTime": 62452.874924, - "min": 12322.135523000004, - "mean": 12490.574984800001, - "max": 12710.745521000004 + "totalTime": 63869.92417999997, + "min": 12599.422429999977, + "mean": 12773.984835999994, + "max": 12931.928272999998 }, { "name": "gcanti/fp-ts", "rank": 2, - "totalTime": 21699.805303, - "min": 4307.874993999983, - "mean": 4339.961060600001, - "max": 4359.28962700002 + "totalTime": 21470.55115699998, + "min": 4259.032535999984, + "mean": 4294.110231399996, + "max": 4345.170425999997 } ] } \ No newline at end of file diff --git a/packages/skott/test/integration/api/runner.spec.ts b/packages/skott/test/integration/api/runner.spec.ts index 1dc88a873..caad89f6d 100644 --- a/packages/skott/test/integration/api/runner.spec.ts +++ b/packages/skott/test/integration/api/runner.spec.ts @@ -46,26 +46,31 @@ describe("When running skott using all real dependencies", () => { ); }); - describe("groupBy", () => { - test("Should not allow `groupBy` to be a non-function", async () => { - await expect(async () => - skott({ - // @ts-expect-error - groupBy: "not-a-function" - }) - ).rejects.toThrow( - "`groupBy` must be a function or not provided at all" - ); + test("Should allow `groupBy` to be a function", async () => { + const skottInstance = await skott({ + groupBy: (_path) => "group", + ignorePatterns: [] }); - test("Should allow `groupBy` to be a function", async () => { - const skottInstance = await skott({ - groupBy: (_path) => "group", - ignorePatterns: [] - }); + expect(skottInstance).toBeDefined(); + }); + + test("Should not allow `groupBy` to be a non-function", async () => { + await expect(async () => + skott({ + // @ts-expect-error + groupBy: "not-a-function" + }) + ).rejects.toThrow("`groupBy` must be a function or not provided at all"); + }); - expect(skottInstance).toBeDefined(); + test("Should allow `groupBy` to be a function", async () => { + const skottInstance = await skott({ + groupBy: (_path) => "group", + ignorePatterns: [] }); + + expect(skottInstance).toBeDefined(); }); }); diff --git a/packages/skott/test/integration/cli/bootstrap.ts b/packages/skott/test/integration/cli/bootstrap.ts index 5fafc9564..160d34a58 100644 --- a/packages/skott/test/integration/cli/bootstrap.ts +++ b/packages/skott/test/integration/cli/bootstrap.ts @@ -55,7 +55,7 @@ export function runOneShotSkottCli( }); childProcess.on("exit", (code) => { - if (code === 0) { + if (code === 0 || stderr === "") { return resolve(Either.right(stdout)); } diff --git a/packages/skott/test/integration/cli/cli.spec.ts b/packages/skott/test/integration/cli/cli.spec.ts index 25df57d2d..c599eee5c 100644 --- a/packages/skott/test/integration/cli/cli.spec.ts +++ b/packages/skott/test/integration/cli/cli.spec.ts @@ -41,6 +41,82 @@ describe.sequential("When running skott cli", () => { expect(right.includes("Usage: cli")).toBeTruthy(); }); + describe("When providing configuration ending up in an illegal state", () => { + describe("When running `webapp` display mode", () => { + const showOptions = [ + "--showCircularDependencies", + "--showUnusedFiles", + "--showUnusedDependencies" + ]; + + test.each(showOptions)( + "Should not allow all `show*` options", + async (o) => { + const result = await runOneShotSkottCli( + ["--displayMode=webapp", "--trackThirdPartyDependencies", o], + AbortSignal.timeout(5000) + ); + + expect(Either.isRight(result)).toBeTruthy(); + + const right = Either.getOrThrow(result); + + expect(right).toContain( + // eslint-disable-next-line no-useless-concat + "`" + o + "`" + " can't be used when using `--displayMode=webapp`" + ); + } + ); + }); + + describe("When collecting unused files", () => { + test("Should fail when providing an entrypoint", async () => { + const result = await runOneShotSkottCli( + ["entrypoint.ts", "--displayMode=raw", "--showUnusedFiles"], + AbortSignal.timeout(5000) + ); + + expect(Either.isRight(result)).toBeTruthy(); + + const right = Either.getOrThrow(result); + + expect(right).toContain( + "`--showUnusedFiles` can't be used when using providing an entrypoint." + ); + }); + }); + + describe("When collecting unused dependencies", () => { + test("Should fail when not having third-party tracking enabled", async () => { + const result = await runOneShotSkottCli( + ["--displayMode=raw", "--showUnusedDependencies"], + AbortSignal.timeout(5000) + ); + + expect(Either.isRight(result)).toBeTruthy(); + + const right = Either.getOrThrow(result); + + expect(right).toContain( + "`--trackThirdPartyDependencies` must be provided when searching for unused dependencies." + ); + }); + + test("Should display unused dependencies when third-party tracking is enabled", async () => { + const result = await runOneShotSkottCli( + [ + "--displayMode=raw", + "--trackThirdPartyDependencies", + "--showUnusedDependencies" + ], + AbortSignal.timeout(5000) + ); + + expect(Either.isLeft(result)).toBeTruthy(); + }); + }); + }); + describe.sequential( "When using watch mode", () => { diff --git a/packages/skott/test/unit/ecmascript/graph.spec.ts b/packages/skott/test/unit/ecmascript/graph.spec.ts index 0a1390736..f93a33c5b 100644 --- a/packages/skott/test/unit/ecmascript/graph.spec.ts +++ b/packages/skott/test/unit/ecmascript/graph.spec.ts @@ -1,9 +1,10 @@ import * as memfs from "memfs"; import { describe, expect, it } from "vitest"; +import { InMemoryFileReader } from "../../../src/filesystem/fake/file-reader.js"; import { InMemoryFileWriter } from "../../../src/filesystem/fake/file-writer.js"; import { FileReader } from "../../../src/filesystem/file-reader.js"; -import { CollectLevel } from "../../../src/graph/traversal.js"; +import { CollectLevel } from "../../../src/graph/api.js"; import { FakeLogger } from "../../../src/logger.js"; import { EcmaScriptDependencyResolver } from "../../../src/modules/resolvers/ecmascript/resolver.js"; import { ModuleWalkerSelector } from "../../../src/modules/walkers/common.js"; @@ -169,6 +170,100 @@ describe("When building the project structure independently of JavaScript or Typ }); }); + describe("When searching for isolated nodes", () => { + describe("When isolated nodes don't have any dependencies", () => { + it("Should return all isolated nodes", async () => { + mountFakeFileSystem({ + "a.js": ` + import { b } from "./b.js"; + export const a = () => b(); + `, + "b.js": ` + export const b = { doSomething: () => bar() }; + `, + "isolated.js": ` + export const x = { doSomething: () => {} }; + ` + }); + + const skott = new Skott( + { + entrypoint: undefined, + incremental: false, + circularMaxDepth: Number.POSITIVE_INFINITY, + includeBaseDir: false, + dependencyTracking: { + thirdParty: false, + builtin: false, + typeOnly: true + }, + fileExtensions: [".js"], + tsConfigPath: "./tsconfig.json", + manifestPath: "./package.json", + dependencyResolvers: [new EcmaScriptDependencyResolver()] + }, + new InMemoryFileReader(), + new InMemoryFileWriter(), + new ModuleWalkerSelector(), + new FakeLogger() + ); + + const { collectUnusedFiles } = await skott + .initialize() + .then(({ useGraph }) => useGraph()); + + expect(collectUnusedFiles()).to.deep.equal(["isolated.js"]); + }); + }); + + describe("When isolated nodes have third-party dependencies or unresolved dependencies", () => { + it("Should return all isolated nodes", async () => { + mountFakeFileSystem({ + "a.js": ` + import { b } from "./b.js"; + export const a = () => b(); + `, + "b.js": ` + export const b = { doSomething: () => bar() }; + `, + "isolated.js": ` + import { Effect } from "effect"; + import notFound from "./not-found.js"; + export const x = { doSomething: () => {} }; + ` + }); + + const skott = new Skott( + { + entrypoint: undefined, + incremental: false, + circularMaxDepth: Number.POSITIVE_INFINITY, + includeBaseDir: false, + dependencyTracking: { + thirdParty: false, + builtin: false, + typeOnly: true + }, + fileExtensions: [".js"], + tsConfigPath: "./tsconfig.json", + manifestPath: "./package.json", + dependencyResolvers: [new EcmaScriptDependencyResolver()] + }, + new InMemoryFileReader(), + new InMemoryFileWriter(), + new ModuleWalkerSelector(), + new FakeLogger() + ); + + const { collectUnusedFiles } = await skott + .initialize() + .then(({ useGraph }) => useGraph()); + + expect(collectUnusedFiles()).to.deep.equal(["isolated.js"]); + }); + }); + }); + describe("Skott Node details", () => { describe("When there is only one file in the project", () => { describe("When the file is empty", () => { diff --git a/packages/skott/test/unit/runner.spec.ts b/packages/skott/test/unit/runner.spec.ts index 30f06ccef..445ccf283 100644 --- a/packages/skott/test/unit/runner.spec.ts +++ b/packages/skott/test/unit/runner.spec.ts @@ -95,7 +95,13 @@ describe("Skott analysis runner", () => { mountFakeFileSystem(fs); const skott = new Skott( - defaultConfig, + { + ...defaultConfig, + dependencyTracking: { + ...defaultConfig.dependencyTracking, + thirdParty: true + } + }, new InMemoryFileReader({ cwd, ignorePatterns: [] }), new InMemoryFileWriter(), new ModuleWalkerSelector(), diff --git a/packages/skott/test/unit/traversal.spec.ts b/packages/skott/test/unit/traversal.spec.ts index 9d2dc615b..afb382376 100644 --- a/packages/skott/test/unit/traversal.spec.ts +++ b/packages/skott/test/unit/traversal.spec.ts @@ -2,7 +2,7 @@ import { describe, expect, test } from "vitest"; import { InMemoryFileReader } from "../../src/filesystem/fake/file-reader.js"; import { InMemoryFileWriter } from "../../src/filesystem/fake/file-writer.js"; -import { CollectLevel } from "../../src/graph/traversal.js"; +import { CollectLevel } from "../../src/graph/api.js"; import { FakeLogger } from "../../src/logger.js"; import { ModuleWalkerSelector } from "../../src/modules/walkers/common.js"; import { defaultConfig, Skott } from "../../src/skott.js";