Error: EPERM: operation not permitted, scandir 'C:/$Recycle.Bin/S-1-5-18' when running lingui extract

[revskill10]

Describe the bug
Running lingui extract results in error

Error: EPERM: operation not permitted, scandir 'C:/$Recycle.Bin/S-1-5-18'

To Reproduce
Use next-js example, and run lingui extract

Expected behavior
The command should run successfully.

• jsLingui version 3.0
• Babel version @babel/core@7.7.7
• Your Babel config (e.g. .babelrc):

{
    "presets": ["next/babel"],
    "plugins": [
      "macros",
      "jsx-control-statements",
      [
        "import",
        {
          "libraryName": "@geist-ui/react",
          "libraryDirectory": "esm"
        }
      ]
    ]
  }
  

[tricoder42]

Hi @revskill10, this seems to be a problem with your configuration rather than with Lingui. The error clearly states that you don't have permission to read from C:/$Recycle.Bin/S-1-5-18.

• What's LinguiJS your configuration?
• Do you have read permissions to folder C:/$Recycle.Bin/S-1-5-18?
• Why do you extract messages while inside a recycle bin?

[revskill10]

@tricoder42 I just followed next-js example. The configuration is the same.
I don't know why lingui extract touch the recycle bin there.

[tricoder42]

Do you run lingui extract or yarn lingui extract? Where's located your repository? Is it inside C:/$Recycle.Bin/S-1-5-18? Do you have read permissions to access that folder?

[MaxEvan]

I'm having the same issue on Windows 10. It seems to be only since we upgraded to version 3. Isn't it weird though that lingui is scanning the Recycling Bin when the default folder should be where the config file or the package.json is?

Here is my lingui.config.js

const config = {
  catalogs: [
    {
      path: '<rootDir>/locale/{locale}/messages',
      include: ['<rootDir>'],
      exclude: ['**/node_modules/**', '**/.next'],
    },
  ],
  compileNamespace: 'cjs',
  fallbackLocale: 'en',
  sourceLocale: 'en',
  locales: ['en', 'fr'],
  format: 'po',
};

module.exports = config;

When I console.log then entries that are sent to glob I get this

{
  entries: [
    '$Recycle.Bin',
    '$WinREAgent',
    'Documents and Settings',
    'DumpStack.log.tmp',
    'hiberfil.sys',
    'Intel',
    'pagefile.sys',
    'PerfLogs',
    'Program Files',
    'Program Files (x86)',
    'ProgramData',
    'Recovery',
    'swapfile.sys',
    'System Volume Information',
    'Users',
    'Windows'
  ]
}

Something's gotta be wrong with the way Lingui resolves the paths or something right?

[tricoder42]

Thanks @MaxEvan, that console.log output is helpful. Unfortunately I don't access to the win machine. Where did you get these entries?

I suspect the bug is somewhere in getCatalogs function:

js-lingui/packages/cli/src/api/catalog.ts

Lines 307 to 392 in aff1d09

	export function getCatalogs(config: LinguiConfig) {
	const catalogsConfig = config.catalogs
	const catalogs = []
	catalogsConfig.forEach((catalog) => {
	// Validate that `catalogPath` doesn't end with trailing slash
	if (catalog.path.endsWith(PATHSEP)) {
	const extension = getFormat(config.format).catalogExtension
	const correctPath = catalog.path.slice(0, -1)
	const examplePath =
	correctPath.replace(
	LOCALE,
	// Show example using one of configured locales (if any)
	(config.locales || [])[0] || "en"
	) + extension
	throw new Error(
	// prettier-ignore
	`Remove trailing slash from "${catalog.path}". Catalog path isn't a directory,` +
	` but translation file without extension. For example, catalog path "${correctPath}"` +
	` results in translation file "${examplePath}".`
	)
	}
	const include = ensureArray(catalog.include).map(normalizeRelativePath)
	const exclude = ensureArray(catalog.exclude).map(normalizeRelativePath)
	// catalog.path without {name} pattern -> always refers to a single catalog
	if (!catalog.path.includes(NAME)) {
	// Validate that sourcePaths doesn't use {name} pattern either
	const invalidSource = include.find((path) => path.includes(NAME))
	if (invalidSource !== undefined) {
	throw new Error(
	`Catalog with path "${catalog.path}" doesn't have a {name} pattern` +
	` in it, but one of source directories uses it: "${invalidSource}".` +
	` Either add {name} pattern to "${catalog.path}" or remove it` +
	` from all source directories.`
	)
	}
	// catalog name is the last directory of catalog.path.
	// If the last part is {locale}, then catalog doesn't have an explicit name
	const name = (function () {
	const _name = catalog.path.split(PATHSEP).slice(-1)[0]
	return _name !== LOCALE ? _name : null
	})()
	catalogs.push(
	new Catalog(
	{
	name,
	path: normalizeRelativePath(catalog.path),
	include,
	exclude,
	},
	config
	)
	)
	return
	}
	const patterns = include.map((path) => path.replace(NAME, "*"))
	const candidates = glob.sync(
	patterns.length > 1 ? `{${patterns.join(",")}` : patterns[0],
	{
	ignore: exclude,
	}
	)
	candidates.forEach((catalogDir) => {
	const name = path.basename(catalogDir)
	catalogs.push(
	new Catalog(
	{
	name,
	path: normalizeRelativePath(catalog.path.replace(NAME, name)),
	include: include.map((path) => path.replace(NAME, name)),
	exclude: exclude.map((path) => path.replace(NAME, name)),
	},
	config
	)
	)
	})
	})
	return catalogs
	}

Another option is

js-lingui/packages/cli/src/api/catalog.ts

Lines 282 to 289 in aff1d09

	get sourcePaths() {
	const includeGlobs = this.include.map(
	(includePath) => `${includePath}${PATHSEP}**${PATHSEP}*.*`
	)
	const patterns =
	includeGlobs.length > 1 ? `{${includeGlobs.join("|")}` : includeGlobs[0]
	return glob.sync(patterns, { ignore: this.exclude })
	}

Could you please print the patterns from the line 280? (in compiled bundle, not the source one. You shoudl find it under node_modules/@lingui/cli/api/catalog.js) That might give us a hint what's going on. This and maybe the output of glob.sync the line below. Thanks!

[MaxEvan]

I tried to log the things you asked for at line 280 but it didn't work because the code is breaking in the try catch at line 143. Here is the error message:

C:\....\node_modules\@lingui\cli\api\catalog.js:143
        throw e;
        ^
Error: EPERM: operation not permitted, scandir 'C:/$Recycle.Bin/S-1-5-18'
    at Object.readdirSync (fs.js:1021:3)
    at GlobSync._readdir (C:\....\node_modules\glob\sync.js:290:41)
    at GlobSync._readdirInGlobStar (C:\....\node_modules\glob\sync.js:269:20)
    at GlobSync._readdir (C:\....\node_modules\glob\sync.js:278:17)
    at GlobSync._processReaddir (C:\....\node_modules\glob\sync.js:137:22)
    at GlobSync._process (C:\....\node_modules\glob\sync.js:132:10)
    at GlobSync._processGlobStar (C:\....\node_modules\glob\sync.js:382:10)
    at GlobSync._process (C:\....\node_modules\glob\sync.js:130:10)
    at GlobSync._processGlobStar (C:\....\node_modules\glob\sync.js:385:10)
    at GlobSync._process (C:\....\node_modules\glob\sync.js:130:10)

So it seems that it's trying to scan everything from C:\ and upwards somehow instead of just scanning starting from my root dir.

[tricoder42]

Interesting. What if you try console.log(this.includes) as a first statement in collect method (line 106)? Also console.log(tmpDir) few lines below (109) would be helpful.

It's weird that there's no output at all. readdirSync and glob.sync are all called after the console.log(patterns) I asked you before.

[MaxEvan]

console.log(this.includes) as a first statement in collect method => undefined
console.log(tmpDir) in collect method => C:\Users\Maxime\AppData\Local\Temp\lingui-6348

[tricoder42]

Released in 3.0.1. Consider donating using OpenCollective to support development and maintanence of this project 👍

[Snaptags]

I'm still having this issue using 3.1.0 :-(

Windows seems unable to resolve the default rootDir. Using this instead works:

  rootDir: "./src",

[semoal]

I'll take a look on this since I downloaded parallels yesterday to fix other issue

[tricoder42]

Thank you @Snaptags for debugging, but it's still rather weird that CLI searches the root of C:\ instead of the src path.

[Snaptags]

Can the shell version have an influence here? I'm using PowerShell Core 7.0.3

[semoal]

Can the shell version have an influence here? I'm using PowerShell Core 7.0.3

I've tried with next-js example running: npx lingui extract or directly using yarn extract (with extract: lingui extract) on package.json, and couldn't replicate the issue.

Powershell 5.1 and the standard windows terminal both worked. If you can provide me some extra details to replicate the issue I take a further look.

[Snaptags]

I have the same behavior with cmd.exe, too. But I noticed that the catalog config is not the problem, but the default rootDir entry. Edited my first comment, because it was making false accusations :-)

rootDir: "./",
or
rootDir: ".",
both do not resolve to the current working directory, but to "/", causing the sourcePaths getter to traverse the drive's root.

But
rootDir: "./src",
works.

[Snaptags]

And this is not limited to Windows. If I run it in the WSL bash I get

That is extract is still starting from / even though rootDir is ".'

[Snaptags]

@semoal could you perhaps post your lingui.config.js?

My current one is

module.exports = {
  catalogs: [
    {
      path: "<rootDir>/locale/{locale}/messages",
      include: ["<rootDir>"],
      exclude: ["**/node_modules/**"],
    },
  ],
  compileNamespace: "cjs",
  extractBabelOptions: {},
  fallbackLocales: {},
  format: "po",
  locales: ["de", "en", "en-x-dev"],
  orderBy: "messageId",
  pseudoLocale: "",
  rootDir: ".",
  runtimeConfigModule: ["@lingui/core", "i18n"],
  sourceLocale: "en-x-dev",
};

[semoal]

@semoal could you perhaps post your lingui.config.js?

My current one is

module.exports = {
  catalogs: [
    {
      path: "<rootDir>/locale/{locale}/messages",
      include: ["<rootDir>"],
      exclude: ["**/node_modules/**"],
    },
  ],
  compileNamespace: "cjs",
  extractBabelOptions: {},
  fallbackLocales: {},
  format: "po",
  locales: ["de", "en"],
  orderBy: "messageId",
  pseudoLocale: "",
  rootDir: ".",
  runtimeConfigModule: ["@lingui/core", "i18n"],
  sourceLocale: "en-x-dev",
};

With your config also fails for me on Windows, because rootDir "." goes to Windows homespace. Just removing it, works perfect on Powershell and normal cmd

[Snaptags]

WITHOUT rootDir: ".", it is using the current working directory as expected 👍

But if I copy the default config from https://lingui.js.org/ref/conf.html I'd still expect it to work.
And I'd expect "." to resolve to the current working directory, too :-)

[semoal]

WITHOUT rootDir: ".", it is using the current working directory as expected 👍

But if I copy the default config from lingui.js.org/ref/conf.html I'd still expect it to work.
And I'd expect "." to resolve to the current working directory, too :-)

Yes, absolutely, i forgot to add i'm looking to fix that asap.

[Snaptags]

I'd appreciate that. Unfortunately I could not narrow down the problem. But I'm happy to help if you need additional debugging output, etc.

[semoal]

I'd appreciate that. Unfortunately I could not narrow down the problem. But I'm happy to help if you need additional debugging output, etc.

Markus, just to confirm that with this change is fixed.

Can you change your /node_modules/@lingui/cli/api/catalog.js, line +- 549, is the normalizeRelativePath func. Change to this, an let me know if the issue is fixed.

function normalizeRelativePath(sourcePath) {
  if (sourcePath === "/") {
    return (0, _normalizePath.default)(_path.default.relative(process.cwd(), sourcePath));
  }

  if (_path.default.isAbsolute(sourcePath)) {
    // absolute path
    return (0, _normalizePath.default)(sourcePath, false);
  }

  var isDir = _fsExtra.default.existsSync(sourcePath) && _fsExtra.default.lstatSync(sourcePath).isDirectory();

  return (0, _normalizePath.default)(_path.default.relative(process.cwd(), sourcePath)) + (isDir ? PATHSEP : "");
}

[Snaptags]

After applying your change it "succeeds" on Windows 10, but still fails in WSL Ubuntu, now reporting

…/node_modules/glob/sync.js:341
        throw er
        ^
Error: EACCES: permission denied, scandir '/etc/polkit-1/localauthority'
    at Object.readdirSync (fs.js:790:3)

On Windows it does not extract any texts though. I think the returned paths still are pointing to the file system root:

(left: Ubuntu / right: Windows)

[Snaptags]

On the other hand: I think normalizeRelativePath is supposed to return relative paths! If I change rootDir back to ./src I get:

i.e. the "logical" way to deal with "." would be to just return an empty string. But then I get even more exceptions from the glob tool…

[semoal]

On the other hand: I think normalizeRelativePath is supposed to return relative paths! If I change rootDir back to ./src I get:

i.e. the "logical" way to deal with "." would be to just return an empty string. But then I get even more exceptions from the glob tool…

Yes I'll refactor that function tomorrow and we'll release a new patch versión at afternon

[semoal]

@Snaptags give a chance to 3.2.1 just published

[Snaptags]

Thanks for fixing this, using 3.2.1 it works as expected. (Powershell Core and Ubuntu Bash)

I'd still recommend using rootDir: "./src" — Lingui will check a bunch of very large files (e.g. in ./build or ./dist) otherwise, making things really slow 😄

[semoal]

Thanks for fixing this, using 3.2.1 it works as expected. (Powershell Core and Ubuntu Bash)

I'd still recommend using rootDir: "./src" — Lingui will check a bunch of very large files (e.g. in ./build or ./dist) otherwise, making things really slow 😄

Or use the exclude pattern is also an option, glad to help Markus :)