Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workaround for passing givaro library to linker on OSX #38020

Merged
merged 1 commit into from
May 25, 2024

Conversation

vbraun
Copy link
Member

@vbraun vbraun commented May 17, 2024

Just hack the missing -lgivaro into the linker command, see also linbox-team/fflas-ffpack#391 for the real bug

Closes #38002

@mkoeppe
Copy link
Contributor

mkoeppe commented May 17, 2024

This works well, thanks

@vbraun vbraun force-pushed the fix-macos-link-fflas-ffpack branch from 5756422 to 8354962 Compare May 17, 2024 20:11
@vbraun
Copy link
Member Author

vbraun commented May 17, 2024

Linbox then fails to build on macos because const_cast<size_t&>(ld) should have been const_cast<size_t&>(_ld), I guess this is elided in gcc 14 so the error is not detected there. I've adapted the linbox patch.

Copy link

Documentation preview for this PR (built with commit 8354962; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

@mkoeppe
Copy link
Contributor

mkoeppe commented May 17, 2024

Can confirm that this fixes this error.

I'm then running into

[linbox-1.7.0] [spkg-install] ld: file not found: @rpath/libquadmath.0.dylib for architecture x86_64
[linbox-1.7.0] [spkg-install] clang: error: linker command failed with exit code 1 (use -v to see invocation)

this could be a misconfiguration related to gfortran though, I'll try again in a clean worktree

@mkoeppe
Copy link
Contributor

mkoeppe commented May 17, 2024

That was it. linbox builds now.

vbraun pushed a commit to vbraun/sage that referenced this pull request May 18, 2024
Just hack the missing -lgivaro into the linker command, see also
linbox-team/fflas-ffpack#391 for the real bug

Closes sagemath#38002

URL: sagemath#38020
Reported by: Volker Braun
Reviewer(s):
@kwankyu
Copy link
Collaborator

kwankyu commented May 18, 2024

Can confirm that this fixes this error.

I'm then running into

[linbox-1.7.0] [spkg-install] ld: file not found: @rpath/libquadmath.0.dylib for architecture x86_64
[linbox-1.7.0] [spkg-install] clang: error: linker command failed with exit code 1 (use -v to see invocation)

I am getting the same error.

this could be a misconfiguration related to gfortran though, I'll try again in a clean worktree

Do you mean something else than make distclean?

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

Just make distclean

@kwankyu
Copy link
Collaborator

kwankyu commented May 18, 2024

I did it, but still get the error...

@kwankyu
Copy link
Collaborator

kwankyu commented May 18, 2024

For me,

Checking whether SageMath should install SPKG gfortran...
checking the version of gfortran... 14
configure: will use system package and not install SPKG gfortran

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

Here I'm using this:

gfortran --version
GNU Fortran (Homebrew GCC 13.2.0) 13.2.0

I've started tests at https://github.com/mkoeppe/sage/actions/runs/9136322984

@kwankyu
Copy link
Collaborator

kwankyu commented May 18, 2024

Here

$ gfortran --version
GNU Fortran (Homebrew GCC 14.1.0) 14.1.0

I am installing gfortran spkg (gfortran 12.2) to see if this helps...

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

that will probably not build; you'll need #38008

@kwankyu
Copy link
Collaborator

kwankyu commented May 18, 2024

It worked:

...
Finished installing linbox-1.7.0
1m2.13s real 42.99s user 16.48s sys  

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

Error reproduced in local-macos (13, xcode_15.0, homebrew-macos-usrlocal-standard)](https://github.com/mkoeppe/sage/actions/runs/9136322984/job/25124971687#logs)

 [linbox-1.7.0]   [spkg-install] libtool: link: g++ -std=gnu++11 -std=gnu++11 -dynamiclib -Wl,-flat_namespace -Wl,-undefined -Wl,suppress -o .libs/liblinbox.0.dylib .libs/liblinbox.0.dylib-master.o  -L/Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib -lntl -lmpfr -liml -lflint  -O2 -march=native -pthread -g -O2 -Wl,-rpath -Wl,/Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib -Wl,-ld_classic -Wl,-rpath -Wl,/Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib   -pthread -install_name  /Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib/liblinbox.0.dylib -compatibility_version 1 -current_version 1.0
  [linbox-1.7.0]   [spkg-install] ld: warning: -undefined suppress is deprecated
  [linbox-1.7.0]   [spkg-install] ld: warning: duplicate -rpath '/Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib' ignored
  [linbox-1.7.0]   [spkg-install] ld: file not found: @rpath/libquadmath.0.dylib for architecture x86_64
  [linbox-1.7.0]   [spkg-install] clang: error: linker command failed with exit code 1 (use -v to see invocation)
  [linbox-1.7.0]   [spkg-install] make[6]: *** [liblinbox.la] Error 1

Likewise local-macos (latest, homebrew-macos-opthomebrew-standard)

@kwankyu
Copy link
Collaborator

kwankyu commented May 18, 2024

... but eventually failed to build sage:

[sagelib-10.4.beta6] [spkg-install]     gcc -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -L/Users/kwankyu/GitHub/sage-dev/local/lib -Wl,-rpath,/Users/kwankyu/GitHub/sage-dev/local/lib -L/Users/kwankyu/GitHub/sage-dev/local/lib -Wl,-rpath,/Users/kwankyu/GitHub/sage-dev/local/lib -g -O2 build/temp.macosx-12.0-x86_64-cpython-312/sage/schemes/elliptic_curves/period_lattice_region.o -o build/lib.macosx-12.0-x86_64-cpython-312/sage/schemes/elliptic_curves/period_lattice_region.cpython-312-darwin.so
[sagelib-10.4.beta6] [spkg-install]     ld: warning: -undefined dynamic_lookup may not work with chained fixups
[sagelib-10.4.beta6] [spkg-install]     24 warnings generated.
[sagelib-10.4.beta6] [spkg-install]     gcc -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -L/Users/kwankyu/GitHub/sage-dev/local/lib -Wl,-rpath,/Users/kwankyu/GitHub/sage-dev/local/lib -L/Users/kwankyu/GitHub/sage-dev/local/lib -Wl,-rpath,/Users/kwankyu/GitHub/sage-dev/local/lib -g -O2 build/temp.macosx-12.0-x86_64-cpython-312/sage/stats/time_series.o -o build/lib.macosx-12.0-x86_64-cpython-312/sage/stats/time_series.cpython-312-darwin.so -lpari
[sagelib-10.4.beta6] [spkg-install]     ld: warning: -undefined dynamic_lookup may not work with chained fixups
[sagelib-10.4.beta6] [spkg-install]     error: command '/usr/bin/gcc' failed with exit code 1
[sagelib-10.4.beta6] [spkg-install]     error: subprocess-exited-with-error
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     × python setup.py develop did not run successfully.
[sagelib-10.4.beta6] [spkg-install]     │ exit code: 1
[sagelib-10.4.beta6] [spkg-install]     ╰─> See above for output.
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     note: This error originates from a subprocess, and is likely not a problem with pip.
[sagelib-10.4.beta6] [spkg-install]     full command: /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.12/bin/python3 -c '
[sagelib-10.4.beta6] [spkg-install]     exec(compile('"'"''"'"''"'"'
[sagelib-10.4.beta6] [spkg-install]     # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
[sagelib-10.4.beta6] [spkg-install]     #
[sagelib-10.4.beta6] [spkg-install]     # - It imports setuptools before invoking setup.py, to enable projects that directly
[sagelib-10.4.beta6] [spkg-install]     #   import from `distutils.core` to work with newer packaging standards.
[sagelib-10.4.beta6] [spkg-install]     # - It provides a clear error message when setuptools is not installed.
[sagelib-10.4.beta6] [spkg-install]     # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
[sagelib-10.4.beta6] [spkg-install]     #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
[sagelib-10.4.beta6] [spkg-install]     #     manifest_maker: standard file '"'"'-c'"'"' not found".
[sagelib-10.4.beta6] [spkg-install]     # - It generates a shim setup.py, for handling setup.cfg-only projects.
[sagelib-10.4.beta6] [spkg-install]     import os, sys, tokenize
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     try:
[sagelib-10.4.beta6] [spkg-install]         import setuptools
[sagelib-10.4.beta6] [spkg-install]     except ImportError as error:
[sagelib-10.4.beta6] [spkg-install]         print(
[sagelib-10.4.beta6] [spkg-install]             "ERROR: Can not execute `setup.py` since setuptools is not available in "
[sagelib-10.4.beta6] [spkg-install]             "the build environment.",
[sagelib-10.4.beta6] [spkg-install]             file=sys.stderr,
[sagelib-10.4.beta6] [spkg-install]         )
[sagelib-10.4.beta6] [spkg-install]         sys.exit(1)
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     __file__ = %r
[sagelib-10.4.beta6] [spkg-install]     sys.argv[0] = __file__
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     if os.path.exists(__file__):
[sagelib-10.4.beta6] [spkg-install]         filename = __file__
[sagelib-10.4.beta6] [spkg-install]         with tokenize.open(__file__) as f:
[sagelib-10.4.beta6] [spkg-install]             setup_py_code = f.read()
[sagelib-10.4.beta6] [spkg-install]     else:
[sagelib-10.4.beta6] [spkg-install]         filename = "<auto-generated setuptools caller>"
[sagelib-10.4.beta6] [spkg-install]         setup_py_code = "from setuptools import setup; setup()"
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     exec(compile(setup_py_code, filename, "exec"))
[sagelib-10.4.beta6] [spkg-install]     '"'"''"'"''"'"' % ('"'"'/Users/kwankyu/GitHub/sage-dev/src/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' --no-user-cfg develop --no-deps
[sagelib-10.4.beta6] [spkg-install]     cwd: /Users/kwankyu/GitHub/sage-dev/src/
[sagelib-10.4.beta6] [spkg-install] error: subprocess-exited-with-error
[sagelib-10.4.beta6] [spkg-install] 
[sagelib-10.4.beta6] [spkg-install] × python setup.py develop did not run successfully.
[sagelib-10.4.beta6] [spkg-install] │ exit code: 1
[sagelib-10.4.beta6] [spkg-install] ╰─> See above for output.
[sagelib-10.4.beta6] [spkg-install] 
[sagelib-10.4.beta6] [spkg-install] note: This error originates from a subprocess, and is likely not a problem with pip.
[sagelib-10.4.beta6] [spkg-install] ********************************************************************************
[sagelib-10.4.beta6] [spkg-install] Error installing sagelib-10.4.beta6
[sagelib-10.4.beta6] [spkg-install] ********************************************************************************
[sagelib-10.4.beta6] [spkg-install] 1m17.87s real 3m3.21s user 19.39s sys
[sagelib-10.4.beta6] ************************************************************************
[sagelib-10.4.beta6] Error installing package sagelib-10.4.beta6

I don't know if this is related though...

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

The real error is likely much further up in the sagelib*.log

@kwankyu
Copy link
Collaborator

kwankyu commented May 18, 2024

Right. There are lots of errors like:

[sagelib-10.4.beta6] [spkg-install]     /usr/local/include/flint/padic.h:22:10: fatal error: 'padic_types.h' file not found
[sagelib-10.4.beta6] [spkg-install]     #include "padic_types.h"
[sagelib-10.4.beta6] [spkg-install]              ^~~~~~~~~~~~~~~
[sagelib-10.4.beta6] [spkg-install]     4 warnings and 1 error generated.

@kwankyu
Copy link
Collaborator

kwankyu commented May 18, 2024

Building again from scratch...with this PR.

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

There are lots of errors like:

[sagelib-10.4.beta6] [spkg-install]     /usr/local/include/flint/padic.h:22:10: fatal error: 'padic_types.h' file not found
[sagelib-10.4.beta6] [spkg-install]     #include "padic_types.h"
[sagelib-10.4.beta6] [spkg-install]              ^~~~~~~~~~~~~~~
[sagelib-10.4.beta6] [spkg-install]     4 warnings and 1 error generated.

I also see this one in CI: stage-2 / local-macos (12, homebrew-macos-usrlocal-standard)

@kwankyu
Copy link
Collaborator

kwankyu commented May 18, 2024

Building again from scratch...with this PR.

Then it stopped at the gfortran problem. So I did make gfortran to install gfortran 12.2 to avoid using the system gfortran 14.1

Then it stopped at "padic_types.h" problem. I did make flint. I guess that somehow sagelib had started to build before flint was built.

Then it is now building the doc. So the building seems to be going to finish well.

vbraun pushed a commit to vbraun/sage that referenced this pull request May 18, 2024
Just hack the missing -lgivaro into the linker command, see also
linbox-team/fflas-ffpack#391 for the real bug

Closes sagemath#38002

URL: sagemath#38020
Reported by: Volker Braun
Reviewer(s):
@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

Homebrew's flint package (both version 3.0.1 and 3.1.0) appears to be broken

$ ls /usr/local/opt/flint/include/flint/padic*  
/usr/local/opt/flint/include/flint/padic.h      
/usr/local/opt/flint/include/flint/padic_mat.h  
/usr/local/opt/flint/include/flint/padic_poly.h
$ grep padic_typ /usr/local/opt/flint/include/flint/padic*  
/usr/local/opt/flint/include/flint/padic.h:#include "padic_types.h"
/usr/local/opt/flint/include/flint/padic_mat.h:#include "padic_types.h"
/usr/local/opt/flint/include/flint/padic_poly.h:#include "padic_types.h"

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

I'm then running into

[linbox-1.7.0] [spkg-install] ld: file not found: @rpath/libquadmath.0.dylib for architecture x86_64
[linbox-1.7.0] [spkg-install] clang: error: linker command failed with exit code 1 (use -v to see invocation)

Also this one can be fixed by running autoreconf on the linbox source tree, same as #38002 (comment)

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

I regenerated fflas-ffpack release tarballs at https://github.com/mkoeppe/fflas-ffpack/releases using ubuntu-jammy and ubuntu-24.04. But these regenerated tarballs do not work either -- because Debian patches libtool to death: https://sources.debian.org/patches/libtool/2.4.7-7/link_all_deplibs.patch/

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

This one, https://github.com/mkoeppe/fflas-ffpack/releases/tag/v2.5.0%2Bsage-2024-05-18b, regenerated using upstream GNU libtool 2.4.7, works.

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2024

Follow-up PR that uses these two fixed tarballs: #38025.

@dcoudert
Copy link
Contributor

Building again from scratch...with this PR.

Then it stopped at the gfortran problem. So I did make gfortran to install gfortran 12.2 to avoid using the system gfortran 14.1

Then it stopped at "padic_types.h" problem. I did make flint. I guess that somehow sagelib had started to build before flint was built.

Then it is now building the doc. So the building seems to be going to finish well.

Almost the same for me with this PR: make build failed, then make gfortran, then make build -j4 failed, then make build succeed. I have not tried without this PR.

@kwankyu
Copy link
Collaborator

kwankyu commented May 19, 2024

Homebrew's flint package (both version 3.0.1 and 3.1.0) appears to be broken

$ ls /usr/local/opt/flint/include/flint/padic*  
/usr/local/opt/flint/include/flint/padic.h      
/usr/local/opt/flint/include/flint/padic_mat.h  
/usr/local/opt/flint/include/flint/padic_poly.h
$ grep padic_typ /usr/local/opt/flint/include/flint/padic*  
/usr/local/opt/flint/include/flint/padic.h:#include "padic_types.h"
/usr/local/opt/flint/include/flint/padic_mat.h:#include "padic_types.h"
/usr/local/opt/flint/include/flint/padic_poly.h:#include "padic_types.h"

Ah, I see. Thanks.

vbraun pushed a commit to vbraun/sage that referenced this pull request May 24, 2024
… FLINT package on homebrew

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

Fixes
sagemath#38020 (comment)

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38021
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee, Matthias Köppe
vbraun pushed a commit to vbraun/sage that referenced this pull request May 24, 2024
…s with working libtool

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

These fix the build on macOS, as discussed in sagemath#38020.

Test run:
https://github.com/mkoeppe/sage/actions/runs/9142716213/job/25138580652

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->

- Depends on sagemath#38020 (merged and partially reverted here)
- Depends on sagemath#37726 (merged here to resolve merge conflict)
    
URL: sagemath#38025
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee, Matthias Köppe
@vbraun vbraun merged commit 10c4e78 into sagemath:develop May 25, 2024
17 of 32 checks passed
@mkoeppe mkoeppe added this to the sage-10.4 milestone May 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

macOS: linbox-1.7.0 fails to build
4 participants