Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add C++ support (GCC and Clang) on all platforms #104

Closed
kcgen opened this issue Nov 20, 2019 · 33 comments
Closed

Add C++ support (GCC and Clang) on all platforms #104

kcgen opened this issue Nov 20, 2019 · 33 comments

Comments

@kcgen
Copy link

kcgen commented Nov 20, 2019

Tool information

  • Tool names: defacto C++ compilers: GCC and Clang

  • Add or update? Add

  • Desired versions: Latest version available in standard package repositories. Currently these are:

    Compiler Version OS Package Manager(s)
    GCC 9 all apt, Brew, MacPorts, and MSYS2
    Clang 8 Ubuntu-19.04 apt
    Clang 9 macOS and Windows Brew and Chocolatey
  • Approximate size: Clang depends on GCC (that's why it often makes sense to simply install both; in addition to their technical differences). Typically the combo is a couple GB on Windows and smaller on macOS and Linux.

  • Brief description of packages: These C++ compilers are used to build literally every C++ autotools/cmake project out there (tens of thousands of open-source applications and tools; perhaps more on GitHub).

    The main goal of this request is to allow the vast majority of open source C/C++ packages to be ./configure'd and built with either compiler (assuming the user pulls in any package-specific dependencies) "out of the box".

  • Homepages: GCC - https://gcc.gnu.org/, Clang - https://clang.llvm.org/

Virtual environments affected

  • ☑️ macOS 10.15
  • ☑️Ubuntu 16.04 LTS
  • ☑️ Ubuntu 18.04 LTS
  • ☑️ Windows Server 2016 R2
  • ☑️ Windows Server 2019

Can they be installed during the build?
Yes. I'm currently installing the following in my workflow:

Compilers OS Package Manager Time
GCC and Clang Ubuntu apt < 60s
GCC macOS Brew and MacPorts between 3 min and 5 min
GCC and Clang Windows MSYS2's pacman between 8 min and 12 min

Compiling my application takes less time than installing these.

Because of this excessive time and resource usage, I made use of GitHub's new cache feature available in CI workflows, however, the cache limit is (currently) too small to save most of these development stacks.

Because it seems absurd that GitHub's cache size cannot accommodate C++ "hello world", I flagged
it here to which GitHub staff member @chrispat suggested that I shouldn't be caching C++ dependencies and instead they should be available in the baseline VMs.

Are you willing to submit a PR?
Yes

@kcgen kcgen changed the title Add C++ compilers (GCC and Clang) on all platforms Add C++ support (GCC and Clang) on all platforms Nov 20, 2019
@stevencpp
Copy link

I would prefer to have clang-9 installed. Most people should be using the latest release. We could also have both the default clang (version 8) and clang-9 installed.

@kcgen
Copy link
Author

kcgen commented Nov 20, 2019

@stevencpp , yes, I would prefer that the latest compiler (as determined by the package manager), be installed by default. Can you let me know on which OS clang-9 is available via the package manager? Thank you; I will update my issue.

@stevencpp
Copy link

On Ubuntu I use this to install it as clang-9 (takes about a minute):

wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key 2>/dev/null | sudo apt-key add -
sudo add-apt-repository 'deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main' -y
sudo apt-get update -q
sudo apt-get install -y clang-9 lld-9 libc++-9-dev libc++abi-9-dev clang-tools-9

and on windows I use this (takes 25 seconds):

curl -fsSL -o LLVM9.exe https://releases.llvm.org/9.0.0/LLVM-9.0.0-win64.exe
7z x LLVM9.exe -y -o"C:/Program Files/LLVM"

I haven't tried setting it up on macOS yet.

@kcgen
Copy link
Author

kcgen commented Nov 20, 2019

Interesting; even though Clang-9 is not in Ubuntu's 19.04 baseline APT repo, nor in macOS's Brew or MacPorts repos, nor in MSYS2's pacman repo -- I do see that these are official releases from LLVM.org.

I would prefer to use latest version per the repos due to the level of integration they provide (ie: pacman under msys2 will pull in dependencies and allow most autotools packages to build - where as a standalone exe installer might be totally disconnected from the msys2 environment). I realize the trade-off here is having to wait a bit longer for the lastest version to be supported by the various operating system repositories.

The main goal of this request is to allow the vast majority of open source C/C++ packages to be ./configure'd and built with either compiler (assuming the user pulls in any package-specific dependencies), as well as support the creation or modifications of similar autotools/cmake type C++ projects "out of the box".

I've amended the version line in the request with a mix of this goal combined with your suggestion; thanks @stevencpp!

@stevencpp
Copy link

stevencpp commented Nov 20, 2019

Chocolatey https://chocolatey.org/packages/llvm and scoop https://github.com/ScoopInstaller/Main/blob/master/bucket/llvm.json have 9.0.0.
https://packages.msys2.org/base/mingw-w64-clang says pacman should have 9.0.0 as well, but I'd rather not use msys for clang. GCC may need msys/cygwin, but for clang I'm guessing most people use it without msys on windows (especially now that Visual Studio comes with clang-cl support built in). The official binary package includes the whole toolchain so it doesn't depend on GCC and cmake works fine with it, no need to depend on msys.
Also https://formulae.brew.sh/formula/llvm suggests brew on macOS should have 9 by default as well ?

@kcgen
Copy link
Author

kcgen commented Nov 20, 2019

@stevencpp, you've got more experience with Clang under windows than myself; thanks for all of this! I've updated the version block.

I left out mentioning that Scoop or MSYS2 could be used to install Clang 9 because it seems like Chocolatey is the prefered approach (want to keep this to the "best" recommended installations only). Does it read OK now?

@stevencpp
Copy link

Seems alright.

There's a proposed llvm-toolchain-9 package for apt https://launchpad.net/ubuntu/+source/llvm-toolchain-9/1:9-2~ubuntu18.04.1 so that might land as well soon ?

@zbeekman
Copy link

If you're going to install the latest stable GCC it would be really nice to include GFortran along with C/C++. On some package managers (Homebrew) you get gfortran and g++ with gcc (e.g., brew install gcc) but on others (at least linux) you need to apt-get install gfortran-<version>, possibly after adding the toolchain-r/test ppa.

This was referenced Jan 29, 2020
@ivan-nosar ivan-nosar self-assigned this Jan 30, 2020
@al-cheb al-cheb self-assigned this Jan 30, 2020
@ivan-nosar
Copy link
Contributor

Good day!
We have added gcc, gfortran and clang tools on Windows and Ubuntu virtual environments in #369

@kcgen
Copy link
Author

kcgen commented Feb 6, 2020

Good day!
We have added gcc, gfortran and clang tools on Windows and Ubuntu virtual environments in #369

Thank you @ivan-nosar ! This will dramatically reduce our workflow times.

@al-cheb al-cheb added the awaiting-deployment Code complete; awaiting deployment and/or deployment in progress label Feb 10, 2020
@al-cheb
Copy link
Contributor

al-cheb commented Feb 10, 2020

Hello, All

We have added GCC and Clang for Ubuntu in #369 and MacOs. We are going to add Windows support in scope of MSYS2 feature, after #355 PR will be merged. It should start rolling out early next week, once the rollout start it shows up as a pre-release here: https://github.com/actions/virtual-environments/releases

After that it takes about 6-7 days to reach all accounts (assuming no delays).

@kcgen
Copy link
Author

kcgen commented Feb 10, 2020

@al-cheb , excellent! Will monitor the release status and am looking forward to putting this to use.

@miketimofeev
Copy link
Contributor

@krcroft FYI ubuntu and macOS images have been rolled out.

@kcgen
Copy link
Author

kcgen commented Feb 20, 2020

@krcroft FYI ubuntu and macOS images have been rolled out.

Confirmed in the latest CI workflows.

macOS:

2020-02-20T20:42:54.7165320Z Warning: gcc 9.2.0_3 is already installed and up-to-date

Ubuntu:

2020-02-20T20:42:42.1321351Z clang-8 is already the newest version

2020-02-20T20:42:40.7763100Z build-essential is already the newest version (12.4ubuntu1).
2020-02-20T20:42:40.7768160Z libtool is already the newest version (2.4.6-2).
2020-02-20T20:42:40.7768382Z libtool set to manually installed.
2020-02-20T20:42:40.7770189Z g++-9 is already the newest version (9.2.1-17ubuntu1~18.04.1)

Our package install phases have dropped from mutliple minutes to sub-60s for macOS and roughly 15s in the case of Linux! Huge win for our (personal) time and GitHub's power usage :-). Thank you @miketimofeev .

@NilsIrl
Copy link
Contributor

NilsIrl commented Feb 22, 2020

Would it be possible to have clang-tools as well (especially clang-format for our use case)?

@dg0yt
Copy link

dg0yt commented Feb 27, 2020

Adding LLVM's clang-8 packages seems to have broken Ubuntu 18.04's clang-8 dependency chains. I have trouble with previously working tasks, such as apt-get install clang-tidy-8

The following packages have unmet dependencies:
 clang-tidy-8 : Depends: libllvm8 (= 1:8-3~ubuntu18.04.2) but 1:8.0.1+svn369350-1~exp1~20200114191400.80 is to be installed
                Depends: clang-tools-8 but it is not going to be installed

or apt-get install libclang-8-dev:

The following packages have unmet dependencies:
 libclang-8-dev : Depends: libclang1-8 (= 1:8-3~ubuntu18.04.2) but 1:8.0.1+svn369350-1~exp1~20200114191400.80 is to be installed
                  Depends: libclang-common-8-dev (= 1:8-3~ubuntu18.04.2) but 1:8.0.1+svn369350-1~exp1~20200114191400.80 is to be installed

@al-cheb
Copy link
Contributor

al-cheb commented Feb 27, 2020

Hello, @dg0yt
Could you please try?

sudo apt-get remove -y libllvm8
sudo apt-get install clang-tidy-8

@dg0yt
Copy link

dg0yt commented Feb 27, 2020

@al-cheb The proposed code works for clang-tidy-8. I will test the other case later.
The removal of libllvm8 took away the following 10 packages:

  clang-8 clangd-8 libclang-common-8-dev libclang1-8 liblldb-8 libllvm8 lld-8
  llvm-8 llvm-8-dev llvm-8-runtime

@al-cheb
Copy link
Contributor

al-cheb commented Feb 27, 2020

@dg0yt,
As another workaround using llvm-toolchain-bionic-8 repo:

wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
REPO_NAME="deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main"
sudo add-apt-repository "${REPO_NAME}"
sudo apt-get update 
sudo apt-get install -t llvm-toolchain-bionic-8 clang-tidy-8

@oovm
Copy link

oovm commented Mar 7, 2020

Failed only on windos beacuse no LIBCLANG_PATH found: https://github.com/GalAster/carbon-rs/runs/492035660

Is there any way to save this?

'Unable to find libclang: "couldn\'t find any valid shared libraries matching: [\'clang.dll\', \'libclang.dll\']
set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found

@splix
Copy link

splix commented Mar 8, 2020

@GalAster it seems you need to install LLVM for Windows, otherwise I had the same error for Rust.

Fixed with the following step:

 - name: Instal LLVM
   run: choco install -y llvm
   if: runner.os == 'Windows'

@adwhit
Copy link

adwhit commented Mar 28, 2020

I found scoop is faster than choco. I have the following:

      - name: Install LLVM (Windows)
        if: runner.os == 'Windows'
        shell: powershell
        run: |
          Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')
          scoop install llvm --global

Note you'll maybe need to pass in

        env:
          LIBCLANG_PATH: "C:\\ProgramData\\scoop\\apps\\llvm\\current\\bin"

to subsequent actions.

@Lewiscowles1986
Copy link

does anyone here know how to force clang to build for mac?

@al-cheb
Copy link
Contributor

al-cheb commented May 13, 2020

@Lewiscowles1986, Could you please take a look - #658 (comment)

@Lewiscowles1986
Copy link

Got it. I fixed last night by overriding CXX and CC environment variables. Thank you.

@aminya
Copy link

aminya commented May 21, 2020

Clang is installed on windows-latest, but it is not on the PATH. Any solution? I even tried installing it using choco, but I got the same error.
https://github.com/aminya/include-what-you-use/runs/697682319?check_suite_focus=true#step:5:5

Is there an llvm-setup action? Installing multiple LLVM versions on different operating
systems is hard.

@maxim-lobanov maxim-lobanov removed the awaiting-deployment Code complete; awaiting deployment and/or deployment in progress label Jun 17, 2020
@Darleev Darleev self-assigned this Jun 17, 2020
@Darleev
Copy link
Contributor

Darleev commented Jun 22, 2020

@aminya Hello,
Currently, you can find clang/clang++ and other binaries, which possible absent in current MinGW installation, by paths:

x64
C:\msys64\mingw64\bin
x32
C:\msys64\mingw32\bin

In order to add them into the system PATH, please follow the steps below:

GitHub Actions:
echo "::add-path::C:\msys64\mingw64\bin"

Azure DevOps
echo '##vso[task.prependpath]C:\msys64\mingw64\bin'

In case of any questions, do not hesitate to contact us.

@Darleev
Copy link
Contributor

Darleev commented Jun 25, 2020

In case of additional questions, do not hesitate to contact us, we will be glad to assist you further.

I'm going to close the issue.

@Darleev Darleev closed this as completed Jun 25, 2020
@sbeyer
Copy link

sbeyer commented Jul 2, 2020

In case of additional questions, do not hesitate to contact us, we will be glad to assist you further.

It's great that a lot of clang tools are available on windows-2019, however, clang-tidy is missing. Is there a reason for that? Is it going to be added?

@maxim-lobanov
Copy link
Contributor

@sbeyer , in scope of this request, only base tools were installed. Feel free to create the separate request and we will consider adding clang-tidy

@advancedwebdeveloper
Copy link

I am curious if there any plans to support Clang 11.0 9 (and corresponding version of lld) ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests