Failing to build nightly on MacOS due to "librsync.h file not found" #4042

MrBobot · 2021-09-18T15:10:32Z

Hi

This morning building kitty from nightly has stopped working on my MacOS BigSur 11.6, something that has not failed me for a year or so since i started building from source.

Excuse my poorly developed script for building from nightly:
#!/bin/bash

cd ~/nightly/kitty
git fetch origin
if ! git diff origin master --quiet
then
git pull
make app
# python3 setup.py linux-package
rm -rf /Applications/kitty.app
cp -R kitty.app /Applications
rm -rf /usr/local/lib/kitty
# cp -R linux-package/ /usr/local
fi

▶ make app
python3 setup.py kitty.app
CC: clang (12, 0)
:1:10: fatal error: 'librsync.h' file not found
#include <librsync.h>
^~~~~~~~~~~~
1 error generated.
The librsync library is required
make: *** [app] Error 1

I am not sure if this has something todo with my updates of python or some other Homebrew process.
I also notice some failures in your Actions related to the same file and MacOS

Thank you

kovidgoyal · 2021-09-18T15:16:18Z

You need to install librsync

MrBobot · 2021-09-18T15:43:12Z

Thanks, that fixed it, should the list on https://sw.kovidgoyal.net/kitty/build/ be updated to include librsync ?

kovidgoyal · 2021-09-18T15:47:16Z

It already is in master. The documentation lists the deps for the
released version.

MrBobot · 2021-09-18T15:52:14Z

Excellent! should have looked there first considering im building nightly ; )
I've made a small $10 via paypal , thanks for your support

kovidgoyal · 2021-09-18T15:59:50Z

Thanks :) Oh and note that nowadays there are nightly builds of kitty
available so you dont have to build them yourself.

9f4f227

bhanuone · 2021-10-05T11:06:09Z

I've installed librsync using homebrew but still getting the same error

python3 setup.py kitty.app 
CC: clang (13, 0)
/var/folders/xd/nn_2kn217b58dmsxnq73s8hh0000gn/T/kitty-test-compile-240u31bt/source.c:1:10: fatal error: 'librsync.h' file not found
#include <librsync.h>
         ^~~~~~~~~~~~
1 error generated.
The librsync library is required
make: *** [app] Error 1

kovidgoyal · 2021-10-05T15:03:43Z

librsync.h needs to be somewhere in a standard include directory on your system. Or use the --extra-include-dirs argument to setup.py to specify the path to the folder it is placed in.

bhanuone · 2021-10-06T04:01:56Z

@kovidgoyal I've included the librsync headers. But failing like this. FYI, I am trying to build kitty in Mac book Pro M1 Chip

python3 setup.py kitty.app 
CC: clang (13, 0)
ld: library not found for -lrsync
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The librsync library is required
make: *** [app] Error 1

kovidgoyal · 2021-10-06T04:35:53Z

next you need to include librsync.dylib in some standard location.

bhanuone · 2021-10-06T06:17:37Z

Thanks @kovidgoyal . I was able to build successfully using LDFLAGS=-L/opt/homebrew/lib make app

xiangpeng2008 · 2021-10-28T01:52:16Z

When I do LDFLAGS=-L/opt/homebrew/lib python3 setup.py --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include, I got

MacBook-Pro /Users/xiangpeng/Documents/kitty $ LDFLAGS=-L/opt/homebrew/lib python3 setup.py  --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include
CC: ['clang'] (13, 0)
Traceback (most recent call last):
  File "setup.py", line 1439, in <module>
    main()
  File "setup.py", line 1400, in main
    build(args)
  File "setup.py", line 821, in build
    kitty_env(), 'kitty/fast_data_types', args.compilation_database, sources, headers
  File "setup.py", line 412, in kitty_env
    at_least_version('harfbuzz', 1, 5)
  File "setup.py", line 122, in at_least_version
    if subprocess.run([PKGCONFIG, package, '--atleast-version=' + q]
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'pkg-config'

page-down · 2021-10-28T02:26:12Z

The pkg-config executable is not found in $PATH. Did you forget to install pkg-config?
https://formulae.brew.sh/formula/pkg-config

xiangpeng2008 · 2021-10-28T07:22:19Z

The pkg-config executable is not found in $PATH. Did you forget to install pkg-config?
https://formulae.brew.sh/formula/pkg-config

Thanks sir for your help, will take a try tonight.

xiangpeng2008 · 2021-10-29T02:13:41Z

after installing

pkg-config harfbuzz lcms2

with home-brew and source a python3.9 virtual environment, remove full kitty folder and redo git clone

LDFLAGS=-L/opt/homebrew/lib python3 setup.py  --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include

works !

tennyson-mccalla · 2021-11-23T17:33:15Z

I must still be doing something wrong here. I run the following in the kitty directory:

LDFLAGS=-L/opt/homebrew/lib python3 setup.py kitty.app --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers/

This seems to work successfully as a kitty.app file gets made:

CC: ['clang'] (13, 0)
[63/63] Compiling kitty/gl-wrapper.c ... done
[6/6] Linking kittens/choose/subseq_matcher ... done
kitty.app successfully built!

But actually trying to run the kitty.app file by double clicking on the icon brings "kitty quit unexpectedly" with the following details:

Translated Report (Full Report Below)

Process: kitty [43327]
Path: /Users/USER/*/kitty.app/Contents/MacOS/kitty
Identifier: net.kovidgoyal.kitty
Version: 0.23.1 (0.23.1)
Code Type: ARM-64 (Native)
Parent Process: launchd [1]
User ID: 501

Date/Time: 2021-11-23 12:22:53.7084 -0500
OS Version: macOS 12.0.1 (21A559)
Report Version: 12
Anonymous UUID: 1C430EA6-B493-4EA4-ABE3-D674194116DA

Sleep/Wake UUID: D6A72BA0-B7B9-4F54-9475-965C6607EF80

Time Awake Since Boot: 61000 seconds
Time Since Wake: 9968 seconds

System Integrity Protection: enabled

Crashed Thread: 0

Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY

Termination Reason: Namespace DYLD, Code 1 Library missing
Library not loaded: @rpath/Python3.framework/Versions/3.8/Python3
Referenced from: /Users/USER/*/kitty.app/Contents/MacOS/kitty
Reason: tried: '/Library/Frameworks/Python3.framework/Versions/3.8/Python3' (no such file), '/System/Library/Frameworks/Python3.framework/Versions/3.8/Python3' (no such file)
(terminated at launch; ignore backtrace)

For some reason it looks like it's looking for /Library/Frameworks/Python3.framework/Versions/3.8/Python3 and /System/Library/Frameworks/Python3.framework/Versions/3.8/Python3 (which don't exist) instead of /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Python3.

EDIT: Got it working. I just used the homebrew installed Python3 instead of the "native" one.

LDFLAGS=-L/opt/homebrew/lib /opt/homebrew/Cellar/python@3.9/3.9.9/bin/python3 setup.py kitty.app --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include -I"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers/"

I'm leaving this all here in case anyone else needs it.

George9000 · 2021-12-15T16:56:37Z

Failing with a -stack-size error on the linking step.

macOS 12.1
arm64-apple-darwin21.2.0
XCode 13.2
Apple clang version 13.0.0 (clang-1300.0.29.30)

Using pyenv

LDFLAGS=-L/opt/homebrew/lib ~/.pyenv/versions/3.9.8/bin/python3 setup.py kitty.app --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include -I"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers/"

This fails at the linking step.

Linking kittens/choose/subseq_matcher ...
clang -Wextra -Wfloat-conversion -Wno-missing-field-initializers -Wall -Wstrict-prototypes -std=c11 -O3 -fwrapv -fstack-protector-strong -pipe -fvisibility=hidden -D_FORTIFY_SOURCE=2 -flto -I/opt/homebrew/Cellar/librsync/2.3.2/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers/ -DKITTY_HAS_RS_SIG_ARGS -Ikitty -I/Users/foo/.pyenv/versions/3.9.8/include/python3.9 -Wall -O3 -shared -L/opt/homebrew/lib -flto build/subseq_matcher-charsets.c.o build/subseq_matcher-main.c.o build/subseq_matcher-output.c.o build/subseq_matcher-score.c.o build/subseq_matcher-unix_compat.c.o -ldl -framework CoreFoundation -L/Users/foo/.pyenv/versions/3.9.8/lib -lpython3.9 -Wl,-stack_size,1000000 -framework CoreFoundation -o build/kittens/choose/subseq_matcher.so
ld: -stack_size option can only be used when linking a main executable
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Not sure where in the build the stack size option is being passed in subseq_matcher

kovidgoyal · 2021-12-15T17:00:59Z

It's not. That comes from a broken python installation, probably
anaconda. Search this issue tracker for stack_size

George9000 · 2021-12-15T17:27:56Z

Thanks so much. Found this in the issues. I just searched setup.py for LINKFORSHARED and made the change suggested in that post.
libs += (sysconfig.get_config_var('LINKFORSHARED') or '').replace('-Wl,-stack_size,1000000', '').split()

Afterwards,

LDFLAGS=-L/opt/homebrew/lib ~/.pyenv/versions/3.9.8/bin/python3 setup.py kitty.app --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include

built and linked without trouble.

However.... upon launching kitty.app, I got a segfault. So, perhaps that quick fix was not the answer.

arminbw · 2022-02-13T11:50:02Z

LDFLAGS=-L/opt/homebrew/lib /opt/homebrew/Cellar/python@3.9/3.9.9/bin/python3 setup.py kitty.app --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include -I"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers/"

EDIT: Got it working. I just used the homebrew installed Python3 instead of the "native" one.
LDFLAGS=-L/opt/homebrew/lib /opt/homebrew/Cellar/python@3.9/3.9.9/bin/python3 setup.py kitty.app --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include -I"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers/"
I'm leaving this all here in case anyone else needs it.

Thank you for leaving this here. It helped me. My slightly updated command (python 3.9.10 changed some paths) looked liked this:
LDFLAGS=-L/opt/homebrew/lib /opt/homebrew/Cellar/python@3.9/3.9.10/bin/python3 setup.py kitty.app --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.2/include -I"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9.10/Frameworks/Python.framework/Headers"

v4ld3rr4m4 · 2022-03-26T14:43:09Z

Somebody install kitty in Clear Linux The error about librsync.h also appear But i dont Know what blunder install that include that lib.

manuranga · 2022-06-14T13:48:32Z

librsync was installed at a different location for me, check using brew info librsync.

LDFLAGS=-L/usr/local/Cellar/librsync/2.3.2/lib python3 setup.py --extra-include-dirs /usr/local/Cellar/librsync/2.3.2/include worked for me.

joshuataylor · 2023-02-26T12:50:12Z

Just to provide more context (sorry for the bump):

If you're using MacOS, you can grab the runtime and build times dependencies via Homebrew. It's okay if you have some of these already installed, it won't reinstall.

# You should double check everything is upgraded first.
brew update
brew upgrade

# These are both runtime and buildtime.
brew install librsync harfbuzz zlib libpng librsync openssl go gcc

Now you're going to need to add LDFLAGS as a prefix, as we're going to need to add the librsync directory.

Homebrew has changed where it stores files on Apple Silicon (M1/M2, etc) vs Intel, but there is a super handy way to ensure you're using the correct prefix and the latest versions.

LDFLAGS=-L$(brew --cellar librsync)/$(brew list --versions librsync | tr ' ' '\n' | tail -1)/lib python3 setup.py --extra-include-dirs $(brew --cellar librsync)/$(brew list --versions librsync | tr ' ' '\n' | tail -1)/include

Note this can be a little slow as it has to invoke Homebrew, so a faster step might be to just do:

echo $(brew --cellar librsync)/$(brew list --versions librsync | tr ' ' '\n' | tail -1)

So for Apple Silicon for example I get this path:

/opt/homebrew/Cellar/librsync/2.3.4

So my command now becomes:

LDFLAGS=-L/opt/homebrew/Cellar/librsync/2.3.4/lib python3 setup.py --extra-include-dirs /opt/homebrew/Cellar/librsync/2.3.4/include

Python

Now as another note, the Python version may be something you don't want to use (ie if installing via homebrew, or using multiple system versions, etc).

You can use a tool like pyenv to create a locked version you use in that directory, so you can always use 3.10.10 if you wanted. I also HIGHLY recommend using virtualenvs, so you don't fight with system pip versions.

pyenv steps (optional, but recommended):

export PYTHON_CONFIGURE_OPTS="--enable-optimizations --with-lto"
This enables lto and optimisations which makes Python a fair bit faster.
pyenv install 3.11.2
Or whatever version is latest at the time you're reading this.
pyenv local 3.10.10
which python
Should show /Users/josh/.pyenv/shims/python
python --version
Should show Python 3.11.2

Virtualenv steps

python3 -m venv .venv
source .venv/bin/activate

You're now in the shell for this. You can also just use the python3 binary if you want (``./.venv/bin/activate`).

So to build the app files we would need to do this, instead of make app:

LDFLAGS=-L$(brew --cellar librsync)/$(brew list --versions librsync | tr ' ' '\n' | tail -1)/lib python3 setup.py kitty.app --extra-include-dirs $(brew --cellar librsync)/$(brew list --versions librsync | tr ' ' '\n' | tail -1)/include

yairchu · 2023-06-05T10:14:01Z

So to build the app files we would need to do this, instead of make app:

LDFLAGS=-L$(brew --cellar librsync)/$(brew list --versions librsync | tr ' ' '\n' | tail -1)/lib python3 setup.py kitty.app --extra-include-dirs $(brew --cellar librsync)/$(brew list --versions librsync | tr ' ' '\n' | tail -1)/include

Thanks, it worked for me!

Wouldn't it make sense for kitty to build according to instructions without custom flags? Or for the instructions to be augmented to make the build work on Apple Silicon?

MrBobot added the bug label Sep 18, 2021

kovidgoyal closed this as completed Sep 18, 2021

ghost mentioned this issue Jul 17, 2023

ENORMOUS performance regression in 0.29.0 & 0.29.1 #6479

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing to build nightly on MacOS due to "librsync.h file not found" #4042

Failing to build nightly on MacOS due to "librsync.h file not found" #4042

MrBobot commented Sep 18, 2021

kovidgoyal commented Sep 18, 2021

MrBobot commented Sep 18, 2021

kovidgoyal commented Sep 18, 2021

MrBobot commented Sep 18, 2021

kovidgoyal commented Sep 18, 2021

bhanuone commented Oct 5, 2021

kovidgoyal commented Oct 5, 2021

bhanuone commented Oct 6, 2021

kovidgoyal commented Oct 6, 2021

bhanuone commented Oct 6, 2021

xiangpeng2008 commented Oct 28, 2021

page-down commented Oct 28, 2021

xiangpeng2008 commented Oct 28, 2021

xiangpeng2008 commented Oct 29, 2021 •

edited

Loading

tennyson-mccalla commented Nov 23, 2021 •

edited

Loading

Translated Report (Full Report Below)

George9000 commented Dec 15, 2021

kovidgoyal commented Dec 15, 2021

George9000 commented Dec 15, 2021 •

edited

Loading

arminbw commented Feb 13, 2022

v4ld3rr4m4 commented Mar 26, 2022 •

edited

Loading

manuranga commented Jun 14, 2022

joshuataylor commented Feb 26, 2023

yairchu commented Jun 5, 2023

Failing to build nightly on MacOS due to "librsync.h file not found" #4042

Failing to build nightly on MacOS due to "librsync.h file not found" #4042

Comments

MrBobot commented Sep 18, 2021

kovidgoyal commented Sep 18, 2021

MrBobot commented Sep 18, 2021

kovidgoyal commented Sep 18, 2021

MrBobot commented Sep 18, 2021

kovidgoyal commented Sep 18, 2021

bhanuone commented Oct 5, 2021

kovidgoyal commented Oct 5, 2021

bhanuone commented Oct 6, 2021

kovidgoyal commented Oct 6, 2021

bhanuone commented Oct 6, 2021

xiangpeng2008 commented Oct 28, 2021

page-down commented Oct 28, 2021

xiangpeng2008 commented Oct 28, 2021

xiangpeng2008 commented Oct 29, 2021 • edited Loading

tennyson-mccalla commented Nov 23, 2021 • edited Loading

Translated Report (Full Report Below)

George9000 commented Dec 15, 2021

kovidgoyal commented Dec 15, 2021

George9000 commented Dec 15, 2021 • edited Loading

arminbw commented Feb 13, 2022

v4ld3rr4m4 commented Mar 26, 2022 • edited Loading

manuranga commented Jun 14, 2022

joshuataylor commented Feb 26, 2023

Python

pyenv steps (optional, but recommended):

Virtualenv steps

yairchu commented Jun 5, 2023

xiangpeng2008 commented Oct 29, 2021 •

edited

Loading

tennyson-mccalla commented Nov 23, 2021 •

edited

Loading

George9000 commented Dec 15, 2021 •

edited

Loading

v4ld3rr4m4 commented Mar 26, 2022 •

edited

Loading