Linuxbrew error parsing formula.jws.json started this morning: Error: "\xE2" on US-ASCII

[dljsjr]

Update

I can see that this is actually fixed on the master branch. I guess the bug can close when it's released to stable?

brew doctor output

$ brew doctor
Error: "\xE2" on US-ASCII
/home/linuxbrew/.linuxbrew/Cellar/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'String#encode'
/home/linuxbrew/.linuxbrew/Cellar/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'JSON::Ext::Parser.parse'
/home/linuxbrew/.linuxbrew/Cellar/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'JSON.parse'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api.rb:98:in 'Homebrew::API.fetch_json_api_file'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api/formula.rb:50:in 'Homebrew::API::Formula.download_and_cache_data!'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api/formula.rb:72:in 'Homebrew::API::Formula.all_formulae'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:852:in 'Formulary::FromAPILoader.try_new'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:861:in 'block (2 levels) in Homebrew::Diagnostic::Checks#check_deleted_formula'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:859:in 'Array#any?'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:859:in 'block in Homebrew::Diagnostic::Checks#check_deleted_formula'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:851:in 'Array#each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:851:in 'Enumerable#filter_map'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:851:in 'Homebrew::Diagnostic::Checks#check_deleted_formula'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/doctor.rb:59:in 'block in Homebrew::Cmd::Doctor#run'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/doctor.rb:52:in 'Array#each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/doctor.rb:52:in 'Homebrew::Cmd::Doctor#run'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:95:in '<main>'
Please report this issue:
  https://docs.brew.sh/Troubleshooting


Obviously this doesn't say "Your system is ready to brew", but I think this is a genuine bug, so there you go.

Verification

• My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update twice and am still able to reproduce my issue.
• This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

$ brew config
HOMEBREW_VERSION: 4.5.0
ORIGIN: https://github.com/Homebrew/brew
HEAD: 3332d3331b56e0aff675d3816d8ebfe564075299
Last commit: 2 days ago
Branch: stable
Core tap JSON: 01 May 16:47 UTC
HOMEBREW_PREFIX: /home/linuxbrew/.linuxbrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: nvim
HOMEBREW_FORCE_API_AUTO_UPDATE: set
HOMEBREW_MAKE_JOBS: 20
Homebrew Ruby: 3.4.3 => /home/linuxbrew/.linuxbrew/Cellar/ruby/3.4.3/bin/ruby
CPU: 20-core 64-bit alderlake
Clang: 20.1.3
Git: 2.49.0 => /home/linuxbrew/.linuxbrew/bin/git
Curl: 7.81.0 => /bin/curl
Kernel: Linux 5.15.0-138-generic x86_64 GNU/Linux
OS: Ubuntu 22.04.5 LTS (jammy)
Host glibc: 2.35
/usr/bin/gcc: 10.5.0
/usr/bin/ruby: N/A
Error: "\xE2" on US-ASCII
/home/linuxbrew/.linuxbrew/Cellar/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'String#encode'
/home/linuxbrew/.linuxbrew/Cellar/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'JSON::Ext::Parser.parse'
/home/linuxbrew/.linuxbrew/Cellar/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'JSON.parse'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api.rb:99:in 'Homebrew::API.fetch_json_api_file'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api/formula.rb:50:in 'Homebrew::API::Formula.download_and_cache_data!'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api/formula.rb:72:in 'Homebrew::API::Formula.all_formulae'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:852:in 'Formulary::FromAPILoader.try_new'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:1133:in 'block in Formulary.loader_for'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:1132:in 'Array#each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:1132:in 'Formulary.loader_for'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:926:in 'Formulary.factory'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/system_config.rb:31:in 'SystemConfig.formula_linked_version'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/system_config.rb:53:in 'block in SystemConfig.dump_verbose_config'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/system_config.rb:52:in 'Array#each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/system_config.rb:52:in 'SystemConfig.dump_verbose_config'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/config.rb:21:in 'Homebrew::Cmd::Config#run'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:95:in '<main>'
Please report this issue:
  https://docs.brew.sh/Troubleshooting

What were you trying to do (and why)?

Just doing a normal brew update this morning. Almost all brew operations that require parsing the JSON are failing.

Here's some additional information and checks I performed before opening the issue:

1. Still happens if I rm -rf ~/.caches/Homebrew/api and have brew re-download the JSON
2. Everything is fine if I use HOMEBREW_NO_INSTALL_FROM_API, as would be expected
3. I have checked that both LANG and LC_ALL are set to en_US.UTF-8, and they are exported:

$ echo $LC_ALL
en_US.UTF-8

$ set --show LC_ALL
$LC_ALL: set in global scope, exported, with 1 elements
$LC_ALL[1]: |en_US.UTF-8|

$ echo $LANG
en_US.UTF-8

$ set --show LANG
$LANG: set in global scope, exported, with 1 elements
$LANG[1]: |en_US.UTF-8|
$LANG: originally inherited as |en_US.UTF-8|

I'm not sure if this is a regression in the JSON file, in Linuxbrew, or a weird regression on my Linux system itself.

The crash goes away if I apply the attached patch:

homebrew.patch

What happened (include all command output)?

Everything is bailing at that JSON parsing operation.

What did you expect to happen?

It shouldn't bail? lol.

Step-by-step reproduction instructions (by running brew commands)

Run any brew command while using the API for formulas

[Bo98]

It'll be interesting to know why this code:

brew/Library/Homebrew/brew.sh

Lines 435 to 446 in bbac5be

	if ! command -v locale >/dev/null
	then
	export LC_ALL=C
	elif [[ "$(locale charmap)" != "UTF-8" ]]
	then
	locales="$(locale -a)"
	c_utf_regex='\bC\.(utf8|UTF-8)\b'
	en_us_regex='\ben_US\.(utf8|UTF-8)\b'
	utf_regex='\b[a-z][a-z]_[A-Z][A-Z]\.(utf8|UTF-8)\b'
	if [[ ${locales} =~ ${c_utf_regex} || ${locales} =~ ${en_us_regex} || ${locales} =~ ${utf_regex} ]]
	then
	export LC_ALL="${BASH_REMATCH[0]}"

doesn't work for you. Although I've fixed the bug in a different way (and yes we should probably tag 4.5.1 with that fix), it'll be good to know why our LC_ALL setting didn't work here.

[dljsjr]

@Bo98 I wonder if it's because this was on Linux? My macOS homebrew installation didn't break in this way, and the block you shared is guarded with a test of -n ${HOMEBREW_MACOS}.

[Bo98]

The upper part is, yes. Though this part is in else part.

Something I also missed is this:

Homebrew Ruby: 3.4.3 => /home/linuxbrew/.linuxbrew/Cellar/ruby/3.4.3/bin/ruby

Something to try is HOMEBREW_FORCE_VENDOR_RUBY=1 and see if the same thing happens.

[dljsjr]

The upper part is, yes. Though this part is in else part.

🤦 should probably be wearing my glasses. Doubly interesting is that not only are LANG and LC_ALL already exported in my environment, locale charmap returns UTF-8.

I'll try out HOMEBREW_FORCE_VENDOR_RUBY and see what happens, just for kicks and giggles.

[dljsjr]

Hah! Sure enough:

[JackBauer69]

I just ran into the same issue running rm -rf "$(brew --cache)". Ran that and went to update && upgrade and hit the exact same error.
System is running Rocky Linux 9.5, following the exact same flow as shown in dljsjr's terminal screenshot fixed it.

Thank You!

[Bo98]

4.5.1 has a mitigation for this so this should work better now.