Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

autotools toolchain #755

Open
jsharpe opened this issue Aug 6, 2021 · 14 comments
Open

autotools toolchain #755

jsharpe opened this issue Aug 6, 2021 · 14 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@jsharpe
Copy link
Collaborator

jsharpe commented Aug 6, 2021

We should provide an autotools toolchain providing the common GNU tools, m4, autoconf and automake so that the configure_make rule can hermetically consume these components without relying on the host environment having working or new enough versions of these components.

@github-actions
Copy link

github-actions bot commented Feb 2, 2022

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_foreign_cc!

@uhthomas
Copy link
Contributor

Hi @jsharpe. What is the status of this issue? Is there any guidance on how best to achieve this?

Thanks :)

@jsharpe
Copy link
Collaborator Author

jsharpe commented Apr 19, 2022

@uhthomas there are toolchain definitions for each of the tools but the rules do not yet make use of them.
The main issue with completing this is that autoconf is not relocatable in its build - it generates many files with hardcoded paths and so to build a hermetic toolchain of all the autotools components would require heavy patching of autoconf. How far I got is in examples/third_party/autotools but I don't currently have much time available to look at this.

I did experiment with hacking the configure script to pass in /proc/self/cwd as part of the install prefix to attempt to fix this but IIRC I ran into some limitations with resolving symlinks in this approach.

My current advise on this is to execute your build within a container with a known deployed toolchain to make this as hermetic as possible.

PRs are welcome though to move towards this goal!

@uhthomas
Copy link
Contributor

Ah, I see, thank you.

Does the example you worked on also support autoreconf? I'd be really interested to help make this better.

@jsharpe
Copy link
Collaborator Author

jsharpe commented Apr 19, 2022

I don't think we currently have any examples that are using autoreconf. If you have a small package that does use that it'd be useful to get that added to test the autoreconf functionality.

@uhthomas
Copy link
Contributor

Not necessarily a "small" example, but as far as I understand capnproto is using autoreconf. Trying to get this working with rules_foreign_cc at the moment and struggling a bit.

https://github.com/capnproto/capnproto/tree/281aacc18003ae103b8a04ea83d7b6f814c97b8b/c%2B%2B

@jsharpe
Copy link
Collaborator Author

jsharpe commented Apr 19, 2022

Its also possible that running autoreconf should be the default behaviour - Debian recommend this: https://wiki.debian.org/Autoreconf#Autoreconfing_packages_-_A_Maintainers.27_Guide.

I guess that the only issue is that the configure_make rules aren't necessarily explicitly an autotools project. Maybe we should have an autotools specific rule to distinguish between autotools specific configure_make projects and more generic ones that just follow that pattern e.g. openssl.

@uhthomas
Copy link
Contributor

Sounds reasonable to me. Do you have a recommended path forward for this? As mentioned before, I'd be happy to help where possible. Though my knowledge of the C/C++ toolchains aren't too strong.

@jsharpe
Copy link
Collaborator Author

jsharpe commented Apr 19, 2022

As far as I'm aware all the code paths should already be in place for autoreconf so likely what I'm suggesting is just a macro around the existing rule with some appropriate defaults set

@uhthomas
Copy link
Contributor

Cool!

For what it's worth I tried building with the autotools example and didn't have too much luck.

+ autoreconf
configure.ac:70: warning: The macro `AC_LANG_C' is obsolete.
configure.ac:70: You should run autoupdate.
./lib/autoconf/c.m4:72: AC_LANG_C is expanded from...
m4/acx_pthread.m4:66: ACX_PTHREAD is expanded from...
lib/m4sugar/m4sh.m4:594: AS_CASE is expanded from...
configure.ac:70: the top level
configure.ac:70: warning: The macro `AC_TRY_LINK' is obsolete.
configure.ac:70: You should run autoupdate.
./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from...
m4/acx_pthread.m4:66: ACX_PTHREAD is expanded from...
lib/m4sugar/m4sh.m4:594: AS_CASE is expanded from...
configure.ac:70: the top level
configure.ac:70: warning: The macro `AC_TRY_LINK' is obsolete.
configure.ac:70: You should run autoupdate.
./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from...
lib/m4sugar/m4sh.m4:692: _AS_IF_ELSE is expanded from...
lib/m4sugar/m4sh.m4:699: AS_IF is expanded from...
./lib/autoconf/general.m4:2894: _AC_LINK_IFELSE is expanded from...
./lib/autoconf/general.m4:2911: AC_LINK_IFELSE is expanded from...
./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from...
m4/acx_pthread.m4:66: ACX_PTHREAD is expanded from...
lib/m4sugar/m4sh.m4:594: AS_CASE is expanded from...
configure.ac:70: the top level
configure.ac:6: error: required directory ./build-aux does not exist
configure.ac:65: error: required file 'build-aux/compile' not found
configure.ac:65:   'automake --add-missing' can install 'compile'
configure.ac:70: error: required file 'build-aux/config.guess' not found
configure.ac:70:   'automake --add-missing' can install 'config.guess'
configure.ac:70: error: required file 'build-aux/config.sub' not found
autoreconf: error: automake failed with exit status: 1

@github-actions
Copy link

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_foreign_cc!

@gozzarda
Copy link

@jsharpe sorry for the bump, but I was wondering if this was considered solved at this point? What is the current idiomatic way to build an autotools based dependency?

@jsharpe
Copy link
Collaborator Author

jsharpe commented Oct 27, 2022

@gozzarda the rules rely on the tools being installed on the host machine; I'd started trying to get bazel to build the tools from source but got stuck on building a relocatable version of autoconf; it hardcodes too many paths internally so I got stuck there with getting a hermetic autotools toolchain built.

Any contributions in fixing this are gladly received as I have very limited time (if any!) to work on rules_foreign_cc at the moment.

@jsharpe jsharpe added enhancement New feature or request help wanted Extra attention is needed and removed Can Close? labels Oct 27, 2022
@gozzarda
Copy link

@jsharpe Thanks for the update. That makes more sense to me. Sounds like builds should be working if you have the dependencies installed but not hermetic?

I too am pretty short on time lately, but have been looking into this because it could benefit a work project, so I may be able to convince work to allocate some time for me to look into this, but don't hold your breath.

For those looking for a combined solution in the meantime, do you still recommend running in a container? I assume a Nix shell would suffice, though the problems you encountered sound like they may also cause Nix headaches so I will have to check how autoconf behaves in Nix.

Thank you for taking what little time you have to respond to this thread. It is appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants