Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unset user-defined locale variables #304

Merged
merged 1 commit into from
Apr 18, 2023

Conversation

smortex
Copy link
Contributor

@smortex smortex commented Apr 11, 2023

Some commands used by the package module can localize the output to
match the user preferences. When matching the output string with a
Regexp to extract information, the pattern fails to be found and the
information is not correctly extracted:

  {
    "status": "install ok installed",
    "version": "7.24.0-1bullseye",
    "latest": ""
  }

In the above example, the shell on the target system has a fr_FR.UTF-8
locale, apt-cache policy puppet-agent returned:

puppet-agent:
  Installé : 7.24.0-1bullseye
  Candidat : 7.24.0-1bullseye
 Table de version :
 *** 7.24.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
        100 /var/lib/dpkg/status
     7.23.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.21.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.20.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.19.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.18.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.17.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.16.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.15.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.14.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.13.1-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.12.1-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.12.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.11.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.10.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.9.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages

and grep 'Candidate:' found no match. Because the available version
is "" and is different from the installed one, it is reported with the
latest key.

Because we rely on mathing localized output, we must make sure to unset
variables that the shell we are running all might have set.

@smortex smortex requested a review from a team as a code owner April 11, 2023 18:41
files/common.sh Outdated
@@ -36,6 +36,8 @@ fi
_tmp="$(mktemp)"
exec 2>>"$_tmp"

unset LANG LC_ALL
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting, what is LANG afterwards? Could we explicitly set LC_ALL=C?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LANG is unset after that 🙃.

I was not sure about forcing LC_ALL to C or C.UTF-8, then recalled that older FreeBSD versions do not support C.UTF-8 and that if the user used something else that UTF-8 with extended ASCII forcing UTF-8 would still break it. But maybe we can do better: we only need to change LC_MESSAGES to have messages in english, and for LC_*, the value is:

  • The value of LC_ALL if set (highest priority);
  • The value of LC_* if set;
  • The value of LANG (lowest priority).

So, we can:

  1. Copy LC_ALL in LANG if LANG is unset;
  2. Unset LC_ALL
  3. Set LC_MESSAGES to C
export LANG=${LANG:-$LC_ALL}
unset LC_ALL
export LC_MESSAGES=C

This should preserve a correct behavior for users of accented chars.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I adjusted the PR accordingly. I guess no packages have a name with accented char so this would not have caused trouble in the tasks of this module, but something makes me think this code should be part of Bolt itself and be run before running any shell scripts on remote systems…

Some commands used by the package module can localize the output to
match the user preferences.  When matching the output string with a
Regexp to extract information, the pattern fails to be found and the
information is not correctly extracted:

```json
  {
    "status": "install ok installed",
    "version": "7.24.0-1bullseye",
    "latest": ""
  }
```

In the above example, the shell on the target system has a fr_FR.UTF-8
locale, `apt-cache policy puppet-agent` returned:

```
puppet-agent:
  Installé : 7.24.0-1bullseye
  Candidat : 7.24.0-1bullseye
 Table de version :
 *** 7.24.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
        100 /var/lib/dpkg/status
     7.23.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.21.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.20.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.19.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.18.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.17.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.16.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.15.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.14.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.13.1-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.12.1-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.12.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.11.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.10.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
     7.9.0-1bullseye 500
        500 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages
```
and `grep 'Candidate:'` found no match.  Because the available version
is `""` and is different from the installed one, it is reported with the
`latest` key.

Because we rely on mathing localized output, we must make sure to unset
variables that the shell we are running all might have set.
@donoghuc donoghuc merged commit 79f046f into puppetlabs:main Apr 18, 2023
@smortex smortex deleted the be-locale-agnostic branch April 19, 2023 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants