Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

worker: support case-insensitive process.env usage on Windows in worker threads #48976

Closed
wants to merge 3 commits into from

Conversation

daeyeon
Copy link
Member

@daeyeon daeyeon commented Jul 31, 2023

Fixes: #48955

By default, a Worker thread uses a copy of parent thread's process.env, which is currently case-sensitive. It can be a problem on Windows using case-insensitive env keys, as reported issue.

This fixes the issue by using a case-insensitive unordered_map for the copied process.env on Windows.

Signed-off-by: Daeyeon Jeong daeyeon.dev@gmail.com

This patch fixes the reported issue by using a case-insensitive
`unordered_map` for the copied `process.env` on Windows.

Signed-off-by: Daeyeon Jeong <daeyeon.dev@gmail.com>
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Jul 31, 2023
Copy link
Contributor

@StefanStojanovic StefanStojanovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if CI passes.

@daeyeon daeyeon added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 31, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 31, 2023
@nodejs-github-bot

This comment was marked as outdated.

src/node_env_var.cc Outdated Show resolved Hide resolved
Signed-off-by: Daeyeon Jeong <daeyeon.dev@gmail.com>
@daeyeon daeyeon added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 1, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 1, 2023
@nodejs-github-bot

This comment was marked as outdated.

Copy link
Member

@bnoordhuis bnoordhuis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorta -.5 on this pull request. Yes, real environment variables are case-insensitive on Windows but deliberately mimicking that behavior for fake environment variables feels kind of yuck.

I expect it'll end up confusing people coding on Windows but deploying on Linux.

src/node_env_var.cc Outdated Show resolved Hide resolved
…indows

Signed-off-by: Daeyeon Jeong <daeyeon.dev@gmail.com>
@daeyeon
Copy link
Member Author

daeyeon commented Aug 3, 2023

I understand the concern. Any mimicking could always break where we least expect it. And I think this issue also come from such a mimicking based on copied environment variables.

In this case, do you think it would be better to document the limitation of the copy? Alternatively, if you have any other suggestions, I would be glad to hear them.

@daeyeon daeyeon added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 3, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 3, 2023
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot
Copy link
Collaborator

@bnoordhuis
Copy link
Member

do you think it would be better to document the limitation of the copy?

You may want to get opinions from more people but that would be my suggestion, yes.

@daeyeon
Copy link
Member Author

daeyeon commented Aug 3, 2023

I'm actually leaning towards that way too now. Will open another PR.

// Test for https://github.com/nodejs/node/issues/48955.

if (!common.isWindows) {
common.skip('this test is Windows-specific.');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering if it would make sense to modify this test for non-windows also to validate if the case-sensitivity is honoured.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it makes sense. That could be like:

const { isWindows } = require('../common');

if (isMainThread) {
  process.env.barkey = 'foovalue';

  strictEqual(process.env.BARKEY, isWindows ? process.env.barkey : undefined);

  const worker = new Worker(__filename);
  worker.once('message', (data) => {
    strictEqual(data.BARKEY, isWindows ? data.barkey : undefined);
  });
} else {
  parentPort.postMessage({
    barkey: process.env.barkey,
    BARKEY: process.env.BARKEY,
  });
}

I would follow up on this PR based on feedback regarding the document fix. Thanks.

@daeyeon
Copy link
Member Author

daeyeon commented Aug 6, 2023

Closing in favor of #49008

@pimterry
Copy link
Member

pimterry commented Aug 7, 2023

I'm sorta -.5 on this pull request. Yes, real environment variables are case-insensitive on Windows but deliberately mimicking that behavior for fake environment variables feels kind of yuck.

I expect it'll end up confusing people coding on Windows but deploying on Linux.

@bnoordhuis @daeyeon As a counterpoint: this current behaviour causes problems that are very confusing when coding on Linux but deploying on Windows (and I have hit issues related to this specifically in production).

Similarly, it will cause confusion when just coding on Windows normally, regardless of deployments, when correct & worker-compatible code inexplicably works differently just because it was moved from the main thread into a worker.

Is it possible to reconsider closing this?

A motivating example:

  • Launch a Node process, with both HTTP_PROXY and http_proxy env vars set to the same value (useful for Linux, where these are different values, and in quite a few weird cases apps will prefer one or the other).
  • On Linux, reading HTTP_PROXY or http_proxy in the Node process will always work, as expected - great.
  • On Windows, due to case insensitivity, those collapse into a single environment variable.
  • On Windows in the main thread, reading both variables works just the same as on Linux - great.
  • On Windows within a worker thread, only the http_proxy lowercase env var is now set and the uppercase env var has mysteriously disappeared, due to this issue. Even though the application was launched with HTTP_PROXY set explicitly, that value is gone in the worker thread - very bad.

Effectively unsetting environment variables unexpectedly could cause all sorts of serious problems, and is definitely confusing, even if it's documented. I wouldn't be surprised if the inconsistency more generally caused other weird edge cases.

It would be much better to fix this so env vars on Windows always work the same everywhere, rather than just documenting the behaviour as broken in workers.

@daeyeon
Copy link
Member Author

daeyeon commented Aug 8, 2023

@leumasme @pimterry Apologies, I assumed there were no objections to addressing issue by the documentation. I should have pinged before closing it. Reopened the issue instead of this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

In worker threads, process.env is case-sensitive on windows
7 participants