/script gets stuck downloading

[ja-gooding]

Bug summary

Weechat v3.3 gets stuck on downloading list of scripts on macOS Monterey with Apple M1.

Steps to reproduce

1. Install weechat with Homebrew.sh brew install weechat
2. Run weechat in terminal weechat
3. In the weechat buffer enter /script
4. After receiving the warning about needing to enable scripts, enable it.
5. /set script.scripts.download_enabled on
6. In the weechat buffer enter /script

Current behavior

After entering the /script command weechat responds with script: downloading list of scripts... and nothing else ever happens, re-running the command produces the same output each time.

I have checked ~/.cache/weechat/script and plugins.xml.gz is present. If I delete the file and re-run the /script command it will re-download the file, but still hangs.

Expected behavior

Drop you into the interactive /script buffer after running the command.

Suggested solutions

I found a work around by doing the following:

1. create a .weechat directory in the home folder
2. mkdir ~/.weechat
3. set the WEECHAT_HOME environment variable
4. export WEECHAT_HOME=~/.weechat
5. Run weechat as normal, and the /script command will now work

Additional information

I don't know how to debug this further, or find the root cause. I have found the work around as described above, but it would be nice if it worked out of the box.

This also happens if built from source, so I don't believe it is an issue with Homebrew itself.

It doesn't seem to matter what variation of the commands you do if you don't explicitly set the WEECHAT_HOME directory. Meaning if you enable downloads first, then run /script, close, restart, etc.

---

• WeeChat version: 3.3
• OS, distribution and version: macOS Monterey Apple M1
• Terminal: Default apple "Terminal" version 2.12 (443), also on "kitty" version 0.23.1
• Terminal multiplexer (screen/tmux/…/none): none

[flashcode]

Hi,

It could be a problem with XDG directories then, but I'm not able to reproduce on my side.

Two things to test, could you please:

• run /debug dirs and show me the output?
• run weechat -t from command line (which runs WeeChat in a single temporary directory) and check there if the download is OK or not?

Thanks.

[flashcode]

Note: when I say I can not reproduce, I tested on Linux/Debian as I've no way to test on macOS.

[flashcode]

Another question: you said file plugins.xml.gz is present, is it non-empty (ie size > 0)?

[ja-gooding]

The output of /debug dirs:

Running weechat -t does not fix it, it still gets stuck downloading the plugins.xml.gz file.

The file is present, however it is is empty, size 0. I have been unable to reproduce the original workaround for some reason. Manually downloading the plugins.xml.gz file per the faq allows me to access the /script buffer, but the scripts don't install, the files are created, but are size zero.

[flashcode]

Weird, then it looks like an issue with the file, does the file has proper write permissions for your own user?
Is there an error displayed when you try to install the script?
Can you try to reproduce on another machine (same or different OS)?

[ja-gooding]

I can try to reproduce on another Mac next week.

[ja-gooding]

Also, you said you have no way of testing on a Mac, shouldn't GitHub actions allow you test this on macOS? Do more tests need to be written for it, or is it not testable?

[flashcode]

There are automatic tests on macOS (10.15 and 11) but I think there's no way to connect interactively on such machines to test.
And currently there's no test of the download of scripts in automatic tests.

[tomoakley]

I am able to reproduce this on a MacBook Pro, Apple M1 Pro processor, macOS Monterey. Exactly the same as @ja-gooding describes; plugins.xml.gz is downloaded to ~/.cache/weechat/scripts, but is empty/0 KB, and the output in Weechat hangs atscript: downloading list of scripts.... However, for me, it also doesn't work with the workaround. I installed weechat with brew install weechat.

python --version is 2.7.18, in Weechat running /python version gives 3.10.1.

Output of /debug dirs:

Directories:                                                                                                                                                       
  home:                                                                                                                                                            
    config: /Users/toakley/.weechat                                                                                                                         
    data: /Users/toakley/.local/share/weechat                                                                                                                      
    cache: /Users/toakley/.cache/weechat                                                                                                                           
    runtime: /Users/toakley/.cache/weechat                                                                                                                         
  lib: /opt/homebrew/Cellar/weechat/3.3_1/lib/weechat                                                                                                              
  lib (extra): -                                                                                                                                                   
  share: /opt/homebrew/Cellar/weechat/3.3_1/share/weechat                                                                                                          
  locale: /opt/homebrew/Cellar/weechat/3.3_1/share/locale                                                                                                          

[flashcode]

For people having this issue, please try to set the environment variable before running WeeChat, like mentioned in this comment: rails/rails#38560 (comment)

You can also check this page for extra info: http://sealiesoftware.com/blog/archive/2017/6/5/Objective-C_and_fork_in_macOS_1013.html

[flashcode]

I'm not sure if something can/should be fixed on WeeChat (not sure something illegal is made).
I could make some tests if someone gives me a temporary access to a macOS system.

[flashcode]

By the way I'll try to display an explicit error when trying to download the file with /script in case of error in the forked process.

[flashcode]

I added the workaround in the FAQ: https://weechat.org/files/doc/devel/weechat_faq.en.html#scripts_update (commit: 1cedb78).

[totallypredictable]

I can confirm on my end that the issue exists.

1. Uninstall weechat with brew uninstall --force weechat
2. Install it again with brew install weechat
3. Enable script downloading with /set script.scripts.download_enabled on
4. run /script update
5. gets stuck at script: downloading list of scripts...

I also tried to set the environment variable with export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES before running weechat, to no avail.
I also tried the workaround described in the commit 1cedb78 but again, no luck.
My machine is MacBook Pro 2019 intel.
The output of /debug dirs is

Directories:
                      │17:58:01      weechat     |   home:
                      │17:58:01      weechat     |     config: /Users/nashjr/.config/weechat
                      │17:58:01      weechat     |     data: /Users/nashjr/.local/share/weechat
                      │17:58:01      weechat     |     cache: /Users/nashjr/.cache/weechat
                      │17:58:01      weechat     |     runtime: /Users/nashjr/.cache/weechat
                      │17:58:01      weechat     |   lib: /usr/local/Cellar/weechat/3.4/lib/weechat
                      │17:58:01      weechat     |   lib (extra): -
                      │17:58:01      weechat     |   share: /usr/local/Cellar/weechat/3.4/share/weechat
                      │17:58:01      weechat     |   locale: /usr/local/Cellar/weechat/3.4/share/locale

The output of python --version is Python 2.7.18 and the output of /python version is python: 3.10.1

[tomoakley]

Coming back to this to see if there's any updates. When I set export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES and start weechat and run /script the interactive buffer starts. However I'm unable to install anything and running /python load <plugin>.py (e.g vimode.py) which is located in ~./weechat/python it just says python: script "vimode.py" not found.

EDIT: fixed (I think) by ensuring I followed instructions re setting correctly WEECHAT_HOME

[flashcode]

So with export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES, you managed to download list of scripts and to install a script?

Unfortunately, I can not help further as I don't have access to any macOS to test.
If someone can give me a temporary access, I can make some tests.

[hexadecagram]

MacPorts user here, having almost the same exact experience on Intel hardware. Was /script ever working for you? It does seem to be a Monterey issue. I was on Big Sur up until about a week ago. Once I upgraded and rebuilt from source, I got this behavior.

[GilOliveira]

Homebrew user here. Just wanted to add that I'm also experiencing this behaviour on my Intel Macbook Pro with Monterey 12.3.1 and none of the fixes proposed here worked. Unfortunately I can't speak for previous macOS versions as I only installed Weechat on Monterey, but the issue is definitely there.

[hexadecagram]

I just tried setting both WEECHAT_HOME and OBJC_DISABLE_INITIALIZE_FORK_SAFETY, both exported and not (i.e., issuing OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES weechat to the shell).

The latter does the trick for me. I have 2 Macs, both on intel, both running 12.3.1. On both machines, I am able to install, contrary to what @tomoakley has reported. Loading scripts was never an issue for me. Tom, do you have both environment variables set? Would you mind experimenting with having either one or the other set?

@flashcode are you still looking for testing access?

[luigi370]

I would love to have this fixed. :D

[tomoakley]

The latter does the trick for me. I have 2 Macs, both on intel, both running 12.3.1. On both machines, I am able to install, contrary to what @tomoakley has reported. Loading scripts was never an issue for me. Tom, do you have both environment variables set? Would you mind experimenting with having either one or the other set?

Hi @hexadecagram, sorry for delay in replying. I've tried with WEECHAT_HOME and OBJC_DISABLE_INITIALIZE_FORK_SAFETY, both exported and non-exported. The WEECHAT_HOME needs exporting (i.e export WEECHAT_HOME=~/.weechat) so it can find my installed script, but for the OBJC variable I can run OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES weechat (i.e without exporting) and it can download scripts; without that it cannot. Hope that helps.

[evgenykushnarenko]

It's work for me:

1. mkdir ~/.weechat
2. add to ~/.zshrc export WEECHAT_HOME=~/.weechat and export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

After that /script working.
MacBook M1 Ventura 13.0.1

[lamons]

I am still having the issue after trying @evgenykushnarenko's method. I can manually download the script list file to ~/.local/cache/weechat/script and have it work temporary, but when weechat try to download the file again it will stop working and the file would be replaced by an empty one.

[weechatter]

try: /set script.scripts.cache_expire -1

or empty the filename in: /set script.scripts.url

[hexadecagram]

The latter does the trick for me. I have 2 Macs, both on intel, both running 12.3.1. On both machines, I am able to install, contrary to what @tomoakley has reported. Loading scripts was never an issue for me. Tom, do you have both environment variables set? Would you mind experimenting with having either one or the other set?

Hi @hexadecagram, sorry for delay in replying. I've tried with WEECHAT_HOME and OBJC_DISABLE_INITIALIZE_FORK_SAFETY, both exported and non-exported. The WEECHAT_HOME needs exporting (i.e export WEECHAT_HOME=~/.weechat) so it can find my installed script, but for the OBJC variable I can run OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES weechat (i.e without exporting) and it can download scripts; without that it cannot. Hope that helps.

I haven't had to load any scripts in some time but I just tried running OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES weechat, which started WeeChat. I entered the /script command and the program displayed a list of scripts!

FWIW, this also works as two separate shell commands. The first sets the environment variable (which doesn't need to be exported) and the second actually runs WeeChat.

$ OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
$ weechat

Looking through the manpage, I see this:

-d, --dir
Force a single directory for all WeeChat files (directory is created if not found). Four directories can be given, separated by colons (in this order: config, data, cache, runtime). If this option is not given, the environment variable WEECHAT_HOME is used (if not empty).

It isn't necessary to set WEECHAT_HOME in my case. All that is required is OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES. And just based on WEECHAT_HOME's stated purpose, I do not think setting it would have any effect one way or another. If it does, it's a side-effect which is a distinct issue from this.

[justinnichols]

It's work for me:

1. `mkdir ~/.weechat`

2. add to ~/.zshrc `export WEECHAT_HOME=~/.weechat` and `export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES`

After that /script working. MacBook M1 Ventura 13.0.1

This indeed worked for me. So something about the configuration being in .config/weechat is the issue.

[Hritik14]

@flashcode can we reopen this issue ?

Above listed solutions did not work for me. Here's how I'm invoking weechat

; mkdir /tmp/weechat
; WEECHAT_HOME=/tmp/weechat OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES weechat

It still gets hung at script: downloading list of scripts... when invoking the /script command.

/debug dirs

Directories:
  home:
    config: /tmp/weechat
    data: /tmp/weechat
    cache: /tmp/weechat
    runtime: /tmp/weechat
  lib: /opt/homebrew/Cellar/weechat/3.8/lib/weechat
  lib (extra): -
  share: /opt/homebrew/Cellar/weechat/3.8/share/weechat
  locale: /opt/homebrew/Cellar/weechat/3.8/share/locale

; ll /tmp/weechat/script/plugins.xml.gz
-rw-r--r--  1 neo  wheel  0 Sep  6 18:36 /tmp/weechat/script/plugins.xml.gz

I'm running this on:

ProductName:		macOS
ProductVersion:		13.4
BuildVersion:		22F66

[Hritik14]

Meanwhile, I'm installing a script by directly downloading from https://weechat.org/scripts/ and placing under $WEECHAT_HOME/perl for perl scripts

[flashcode]

Reopening the issue.

I'm experimenting the add of a new hook to transfer URL, by using threads instead of a process (via function hook_process_hashtable), this should fix indirectly this issue, at least for the download of scripts (there could still be problems with processes on macOS).

And as a bonus, using a thread is lighter than launching a new process.

I'll post here the link to a new branch for tests, when it's ready.

[flashcode]

Hi,

I pushed a test branch called "hook-url": https://github.com/weechat/weechat/tree/hook-url

It adds a new API function "hook_url" that is using a thread to download the files.
It is used in the script plugin to download scripts and list of scripts.
It should fix the download issue on macOS.

In case it doesn't work, there's also a way to debug hook_url: execute command /debug url and then /script update, it will show URL options and output (including error) from hook_url.

Please try to compile and run this version, and you don't need to set OBJC_DISABLE_INITIALIZE_FORK_SAFETY (please don't set it to be sure it works fine without this).

If it's working fine, I'll add this function in scripting API and in the documentation.

[flashcode]

Hi macOS users,

I'm still waiting some positive feedback before merging this to master.
It would be nice to have some weeks to test this before the next release.
Thanks for your help!

[hrvstr]

I just compiled the new hook-url branch on macOS 12.6.8 (21G725) and everything seems to work fine again. I can list and install scripts with the /scripts command.

In confirmed that that the environment variable is not set by running env | grep OBJC_DISABLE_INITIALIZE_FORK_SAFETY in my fish shell.

Thanks for the great work!

[flashcode]

Thanks for the tests, this has been merged into master branch.

[transcendr]

Hi I'm on MacOS (monterey) with intel chip. I Just installed weechat via homebrew and I still have this issue, but /debug url throws an error, so I don't have the latest version.

that said, I used the following and then started weechat and the script list loaded:

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

How do i upgrade to the latest version on homebrew and I will give you another mac test.

[flashcode]

@transcendr: for now it's available in the master branch of the repository, I don't know how you can install this from homebrew.

You could try to compile it yourself if you want to give a try (be careful, you must never downgrade config files, so if you test 4.1.0-dev, better either test in a separate WeeChat directory or backup your config files before, if you plan to come back to any older release).