Skip to content

Commit

Permalink
fix(compiler): normalize paths on windows
Browse files Browse the repository at this point in the history
this patch ensures that paths are properly normalized by the compiler.
this ensures that regardless of the platform (operating system) that a
project is compiled on, paths are uniformly treated internally by
stencil. this has system-wide reaching effects - from the in-memory
filesystem, to configuration/output target validation, and file
generation.

previously, stencil's in-browser compilation support included a polyfill
for the following NodeJS `path` module functions: `join`, `normalize`,
`relative` & `resolve`. this polyfill did the following:
- it wrapped each of the aforementioned functions in a `normalizePath`
  function to convert Windows-style path separators (`\\`) to
  Unix/POSIX-style path separators (`/`)
- it overwrote the standard NodeJS `path` implementations for each of
  these functions.
as a result, calling `join` or any of the other three methods, even when
importing the method from `path` like below would result in the polyfill
being called:
```ts
import { join } from 'path'; // this imports the polyfilled `join`

// runs the native `path.join`, then normalizes the returned path
const filePath = join(part1, part2);
```

while this was 'nice' in that stencil engineers didn't need to think
about which implementation of `path` functions they were using, this
polyfill made some behavior of the compiler hard to understand.

the polyfills were removed in #4317 (b042d8b). this led to calls to the
aforementioned functions to call their original implementations, rather
than the wrapped implementations:
```ts
import { join } from 'path'; // imports Node's `join`

// run the native `path.join`, without any normalization
const filePath = join(part1, part2);
```

discrepencies arose where parts of the code would explicitly wrap a
call to `join()` (or one of its ilk) around a path normalization
function. this caused paths to not be uniformly normalized throughout
the codebase, leading to errors.

since the removal of in-browser compilation, additional pull requests
to fix path-related issues on windows have landed in the codebase:
- #4545 (cd58d9c)
- #4932 (b97dadc)

this commit builds on the previous commits by attempting to move
stencil's compiler completely over to polyfilled versions of the
mentioned `path` functions that are explicitly imported/called.

Some of the changes found herein were created/validated using Alice's
codemod branch, #4996
Co-authored-by: alicewriteswrongs <alicewriteswrongs@users.noreply.github.com>

STENCIL-975 Determine Scope of Path Polyfill Bug, Fix

Fixes: #4980
Fixes: #4961

Spun Off: #5036, #5032, #5029
  • Loading branch information
rwaskiewicz committed Nov 9, 2023
1 parent eaff964 commit d7d1f7b
Show file tree
Hide file tree
Showing 73 changed files with 132 additions and 123 deletions.
3 changes: 1 addition & 2 deletions src/compiler/app-core/app-es5-disabled.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { escapeHtml, generatePreamble } from '@utils';
import { join } from 'path';
import { escapeHtml, generatePreamble, join } from '@utils';

import type * as d from '../../declarations';

Expand Down
2 changes: 1 addition & 1 deletion src/compiler/app-core/app-polyfills.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from '@utils';

import type * as d from '../../declarations';

Expand Down
4 changes: 2 additions & 2 deletions src/compiler/build/compiler-ctx.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { noop, normalizePath } from '@utils';
import { basename, dirname, extname, join } from 'path';
import { join, noop, normalizePath } from '@utils';
import { basename, dirname, extname } from 'path';

import type * as d from '../../declarations';
import { buildEvents } from '../events';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/build/watch-build.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { isString } from '@utils';
import { dirname, resolve } from 'path';
import { isString, resolve } from '@utils';
import { dirname } from 'path';
import type ts from 'typescript';

import type * as d from '../../declarations';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/bundle/core-resolve-plugin.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { isRemoteUrl, normalizeFsPath, normalizePath } from '@utils';
import { dirname, join } from 'path';
import { isRemoteUrl, join, normalizeFsPath, normalizePath } from '@utils';
import { dirname } from 'path';
import type { Plugin } from 'rollup';

import type * as d from '../../declarations';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/bundle/dev-module.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { generatePreamble } from '@utils';
import { basename, dirname, join, relative } from 'path';
import { generatePreamble, join, relative } from '@utils';
import { basename, dirname } from 'path';
import { OutputOptions, rollup } from 'rollup';

import type * as d from '../../declarations';
Expand Down
3 changes: 2 additions & 1 deletion src/compiler/bundle/dev-node-module-resolve.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { basename, dirname, join, relative } from 'path';
import { join, relative } from '@utils';
import { basename, dirname } from 'path';
import { ResolveIdResult } from 'rollup';

import type * as d from '../../declarations';
Expand Down
3 changes: 1 addition & 2 deletions src/compiler/bundle/ext-transforms-plugin.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { hasError, isOutputTargetDistCollection, normalizeFsPath } from '@utils';
import { join, relative } from 'path';
import { hasError, isOutputTargetDistCollection, join, normalizeFsPath, relative } from '@utils';
import type { Plugin } from 'rollup';

import type * as d from '../../declarations';
Expand Down
3 changes: 1 addition & 2 deletions src/compiler/bundle/plugin-helper.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { buildError } from '@utils';
import { relative } from 'path';
import { buildError, relative } from '@utils';

import type * as d from '../../declarations';

Expand Down
2 changes: 1 addition & 1 deletion src/compiler/bundle/user-index-plugin.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from '@utils';
import type { Plugin } from 'rollup';

import type * as d from '../../declarations';
Expand Down
2 changes: 1 addition & 1 deletion src/compiler/cache.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from '@utils';

import type * as d from '../declarations';
import { InMemoryFileSystem } from './sys/in-memory-fs';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/config/config-utils.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { isBoolean } from '@utils';
import { isAbsolute, join } from 'path';
import { isBoolean, join } from '@utils';
import { isAbsolute } from 'path';

import type { ConfigFlags } from '../../cli/config-flags';
import type * as d from '../../declarations';
Expand Down
3 changes: 1 addition & 2 deletions src/compiler/config/outputs/validate-custom-element.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { COPY, DIST_TYPES, isBoolean, isOutputTargetDistCustomElements } from '@utils';
import { join } from 'path';
import { COPY, DIST_TYPES, isBoolean, isOutputTargetDistCustomElements, join } from '@utils';

import type {
OutputTarget,
Expand Down
4 changes: 3 additions & 1 deletion src/compiler/config/outputs/validate-dist.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,10 @@ import {
isBoolean,
isOutputTargetDist,
isString,
join,
resolve,
} from '@utils';
import { isAbsolute, join, resolve } from 'path';
import { isAbsolute } from 'path';

import type * as d from '../../../declarations';
import { getAbsolutePath } from '../config-utils';
Expand Down
3 changes: 2 additions & 1 deletion src/compiler/config/outputs/validate-docs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ import {
isOutputTargetDocsReadme,
isOutputTargetDocsVscode,
isString,
join,
} from '@utils';
import { isAbsolute, join } from 'path';
import { isAbsolute } from 'path';

import type * as d from '../../../declarations';
import { NOTE } from '../../docs/constants';
Expand Down
3 changes: 2 additions & 1 deletion src/compiler/config/outputs/validate-hydrate-script.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ import {
isOutputTargetHydrate,
isOutputTargetWww,
isString,
join,
} from '@utils';
import { isAbsolute, join } from 'path';
import { isAbsolute } from 'path';

import type * as d from '../../../declarations';

Expand Down
3 changes: 1 addition & 2 deletions src/compiler/config/outputs/validate-lazy.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { DIST_LAZY, isBoolean, isOutputTargetDistLazy } from '@utils';
import { join } from 'path';
import { DIST_LAZY, isBoolean, isOutputTargetDistLazy, join } from '@utils';

import type * as d from '../../../declarations';
import { getAbsolutePath } from '../config-utils';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/config/outputs/validate-stats.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { isOutputTargetStats, STATS } from '@utils';
import { isAbsolute, join } from 'path';
import { isOutputTargetStats, join, STATS } from '@utils';
import { isAbsolute } from 'path';

import type * as d from '../../../declarations';

Expand Down
3 changes: 2 additions & 1 deletion src/compiler/config/outputs/validate-www.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ import {
isOutputTargetDist,
isOutputTargetWww,
isString,
join,
WWW,
} from '@utils';
import { isAbsolute, join } from 'path';
import { isAbsolute } from 'path';

import type * as d from '../../../declarations';
import { getAbsolutePath } from '../config-utils';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/config/validate-dev-server.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { buildError, isBoolean, isNumber, isOutputTargetWww, isString, normalizePath } from '@utils';
import { isAbsolute, join } from 'path';
import { buildError, isBoolean, isNumber, isOutputTargetWww, isString, join, normalizePath } from '@utils';
import { isAbsolute } from 'path';

import type * as d from '../../declarations';

Expand Down
3 changes: 2 additions & 1 deletion src/compiler/config/validate-paths.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { isAbsolute, join } from 'path';
import { join } from '@utils';
import { isAbsolute } from 'path';

import type * as d from '../../declarations';

Expand Down
4 changes: 2 additions & 2 deletions src/compiler/config/validate-prerender.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { buildError, isString, normalizePath } from '@utils';
import { isAbsolute, join } from 'path';
import { buildError, isString, join, normalizePath } from '@utils';
import { isAbsolute } from 'path';

import type * as d from '../../declarations';

Expand Down
4 changes: 2 additions & 2 deletions src/compiler/config/validate-service-worker.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { isString } from '@utils';
import { isAbsolute, join } from 'path';
import { isString, join } from '@utils';
import { isAbsolute } from 'path';

import type * as d from '../../declarations';

Expand Down
4 changes: 2 additions & 2 deletions src/compiler/config/validate-testing.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { buildError, isOutputTargetDist, isOutputTargetWww, isString } from '@utils';
import { basename, dirname, isAbsolute, join } from 'path';
import { buildError, isOutputTargetDist, isOutputTargetWww, isString, join } from '@utils';
import { basename, dirname, isAbsolute } from 'path';

import type * as d from '../../declarations';
import { isLocalModule } from '../sys/resolve/resolve-utils';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/docs/generate-doc-data.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { flatOne, isOutputTargetDocsJson, normalizePath, sortBy, unique } from '@utils';
import { basename, dirname, join, relative } from 'path';
import { flatOne, isOutputTargetDocsJson, join, normalizePath, relative, sortBy, unique } from '@utils';
import { basename, dirname } from 'path';

import type * as d from '../../declarations';
import { JsonDocsValue } from '../../declarations';
Expand Down
3 changes: 1 addition & 2 deletions src/compiler/docs/json/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { isOutputTargetDocsJson } from '@utils';
import { join } from 'path';
import { isOutputTargetDocsJson, join } from '@utils';

import type * as d from '../../../declarations';

Expand Down
3 changes: 1 addition & 2 deletions src/compiler/docs/readme/markdown-dependencies.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { normalizePath } from '@utils';
import { relative } from 'path';
import { normalizePath, relative } from '@utils';

import type * as d from '../../../declarations';

Expand Down
2 changes: 1 addition & 1 deletion src/compiler/docs/readme/output-docs.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join, relative } from 'path';
import { join, relative } from '@utils';

import type * as d from '../../../declarations';
import { AUTO_GENERATE_COMMENT } from '../constants';
Expand Down
3 changes: 1 addition & 2 deletions src/compiler/docs/vscode/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { isOutputTargetDocsVscode } from '@utils';
import { join } from 'path';
import { isOutputTargetDocsVscode, join } from '@utils';

import type * as d from '../../../declarations';
import { getNameText } from '../generate-doc-data';
Expand Down
2 changes: 1 addition & 1 deletion src/compiler/html/add-script-attr.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from '@utils';

import type * as d from '../../declarations';
import { getAbsoluteBuildDir } from './html-utils';
Expand Down
2 changes: 1 addition & 1 deletion src/compiler/html/html-utils.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join, relative } from 'path';
import { join, relative } from '@utils';

import type * as d from '../../declarations';

Expand Down
2 changes: 1 addition & 1 deletion src/compiler/html/inject-module-preloads.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from '@utils';

import type * as d from '../../declarations';
import { getAbsoluteBuildDir } from './html-utils';
Expand Down
3 changes: 1 addition & 2 deletions src/compiler/html/inline-esm-import.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { isString } from '@utils';
import { join } from 'path';
import { isString, join } from '@utils';
import ts from 'typescript';

import type * as d from '../../declarations';
Expand Down
2 changes: 1 addition & 1 deletion src/compiler/html/inline-style-sheets.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from '@utils';

import type * as d from '../../declarations';

Expand Down
2 changes: 1 addition & 1 deletion src/compiler/html/update-global-styles-link.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from '@utils';

import type * as d from '../../declarations';
import { getAbsoluteBuildDir } from './html-utils';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/html/validate-manifest-json.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { buildError, buildJsonFileError, isOutputTargetWww } from '@utils';
import { dirname, join } from 'path';
import { buildError, buildJsonFileError, isOutputTargetWww, join } from '@utils';
import { dirname } from 'path';

import type * as d from '../../declarations';

Expand Down
4 changes: 2 additions & 2 deletions src/compiler/output-targets/copy/assets-copy-tasks.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { normalizePath } from '@utils';
import { dirname, join, relative } from 'path';
import { join, normalizePath, relative } from '@utils';
import { dirname } from 'path';

import type * as d from '../../../declarations';

Expand Down
3 changes: 2 additions & 1 deletion src/compiler/output-targets/copy/hashed-copy.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { dirname, extname, join } from 'path';
import { join } from '@utils';
import { dirname, extname } from 'path';

import type * as d from '../../../declarations';

Expand Down
3 changes: 2 additions & 1 deletion src/compiler/output-targets/copy/local-copy-tasks.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { isAbsolute, join } from 'path';
import { join } from '@utils';
import { isAbsolute } from 'path';

import type * as d from '../../../declarations';

Expand Down
3 changes: 1 addition & 2 deletions src/compiler/output-targets/copy/output-copy.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import { buildError, isGlob, isOutputTargetCopy, normalizePath } from '@utils';
import { buildError, isGlob, isOutputTargetCopy, join, normalizePath } from '@utils';
import minimatch from 'minimatch';
import { join } from 'path';

import type * as d from '../../../declarations';
import { canSkipAssetsCopy, getComponentAssetsCopyTasks } from './assets-copy-tasks';
Expand Down
2 changes: 1 addition & 1 deletion src/compiler/output-targets/dist-custom-elements/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ import {
hasError,
isOutputTargetDistCustomElements,
isString,
join,
rollupToStencilSourceMap,
} from '@utils';
import { join } from 'path';
import ts from 'typescript';

import type * as d from '../../../declarations';
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import { catchError, createOnWarnFn, generatePreamble, loadRollupDiagnostics } from '@utils';
import { catchError, createOnWarnFn, generatePreamble, join, loadRollupDiagnostics } from '@utils';
import MagicString from 'magic-string';
import { join } from 'path';
import { RollupOptions } from 'rollup';
import { rollup } from 'rollup';

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { basename, join } from 'path';
import { join } from '@utils';
import { basename } from 'path';
import type { RollupOutput } from 'rollup';

import type * as d from '../../../declarations';
Expand Down
3 changes: 1 addition & 2 deletions src/compiler/output-targets/dist-lazy/generate-cjs.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { generatePreamble, relativeImport } from '@utils';
import { join } from 'path';
import { generatePreamble, join, relativeImport } from '@utils';
import type { OutputOptions, RollupBuild } from 'rollup';

import type * as d from '../../../declarations';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ import {
formatComponentRuntimeMeta,
getSourceMappingUrlForEndOfFile,
hasDependency,
join,
rollupToStencilSourceMap,
stringifyRuntimeData,
} from '@utils';
import { join } from 'path';
import type { SourceMap as RollupSourceMap } from 'rollup';

import type * as d from '../../../declarations';
Expand Down
3 changes: 1 addition & 2 deletions src/compiler/output-targets/dist-lazy/generate-system.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { generatePreamble, relativeImport } from '@utils';
import { join } from 'path';
import { generatePreamble, join, relativeImport } from '@utils';
import type { OutputOptions, RollupBuild } from 'rollup';

import type * as d from '../../../declarations';
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { getSourceMappingUrlForEndOfFile } from '@utils';
import { join } from 'path';
import { getSourceMappingUrlForEndOfFile, join } from '@utils';

import type * as d from '../../../declarations';

Expand Down
4 changes: 2 additions & 2 deletions src/compiler/plugin/plugin.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { buildError, catchError, isFunction, isOutputTargetDocs, isString } from '@utils';
import { basename, relative } from 'path';
import { buildError, catchError, isFunction, isOutputTargetDocs, isString, relative } from '@utils';
import { basename } from 'path';

import type * as d from '../../declarations';
import { PluginCtx, PluginTransformResults } from '../../declarations';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/prerender/prerender-main.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { buildError, catchError, hasError, isOutputTargetWww, isString } from '@utils';
import { isAbsolute, join } from 'path';
import { buildError, catchError, hasError, isOutputTargetWww, isString, join } from '@utils';
import { isAbsolute } from 'path';

import type * as d from '../../declarations';
import { createHydrateBuildId } from '../../hydrate/runner/render-utils';
Expand Down
3 changes: 1 addition & 2 deletions src/compiler/prerender/prerender-optimize.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { catchError, flatOne, isString, unique } from '@utils';
import { join } from 'path';
import { catchError, flatOne, isString, join, unique } from '@utils';

import type * as d from '../../declarations';
import { injectModulePreloads } from '../html/inject-module-preloads';
Expand Down
4 changes: 2 additions & 2 deletions src/compiler/prerender/prerender-worker.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { catchError, isFunction, isPromise, isRootPath, normalizePath } from '@utils';
import { dirname, join } from 'path';
import { catchError, isFunction, isPromise, isRootPath, join, normalizePath } from '@utils';
import { dirname } from 'path';

import type * as d from '../../declarations';
import { crawlAnchorsForNextUrls } from './crawl-urls';
Expand Down
Loading

0 comments on commit d7d1f7b

Please sign in to comment.