Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add gitlab CI #2200

Merged
merged 7 commits into from
Sep 19, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
112 changes: 112 additions & 0 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
variables:
# Commit of ghc/ci-images repository from which to pull Docker images
DOCKER_REV: "853f348f9caf38b08740b280296fbd34e09abb3a"

CABAL_INSTALL_VERSION: 3.4.0.0

.default_matrix: &default_matrix
matrix:
- GHC_VERSION: 8.8.4
CABAL_PROJECT: cabal.project
- GHC_VERSION: 8.10.7
CABAL_PROJECT: cabal.project
- GHC_VERSION: 9.0.1
CABAL_PROJECT: cabal-ghc901.project

.m1_matrix: &m1_matrix
matrix:
- GHC_VERSION: 8.10.7
CABAL_PROJECT: cabal.project

.arm_matrix: &arm_matrix
matrix:
- GHC_VERSION: 8.10.7
CABAL_PROJECT: cabal.project


workflow:
rules:
- if: $CI_COMMIT_TAG
when: always
- when: never

.build:
script:
- bash .gitlab/ci.sh
artifacts:
expire_in: 2 week
paths:
- out

build-aarch64-linux-deb10:
extends: .build
tags:
- aarch64-linux
image: "registry.gitlab.haskell.org/ghc/ci-images/aarch64-linux-deb10:$DOCKER_REV"
parallel: *arm_matrix

build-armv7-linux-deb10:
extends: .build
tags:
- armv7-linux
image: "registry.gitlab.haskell.org/ghc/ci-images/armv7-linux-deb10:$DOCKER_REV"
parallel: *arm_matrix

build-x86_64-linux:
extends: .build
tags:
- x86_64-linux
image: "registry.gitlab.haskell.org/ghc/ci-images/x86_64-linux-deb9:$DOCKER_REV"
parallel: *default_matrix

build-x86_64-linux-alpine:
extends: .build
tags:
- x86_64-linux
image: "registry.gitlab.haskell.org/ghc/ci-images/x86_64-linux-alpine3_12:$DOCKER_REV"
parallel: *default_matrix

build-x86_64-freebsd:
extends: .build
tags:
- x86_64-freebsd13
parallel: *default_matrix

build-x86_64-darwin:
extends: .build
tags:
- x86_64-darwin
parallel: *default_matrix

build-aarch64-darwin:
tags:
- aarch64-darwin-m1
script: |
set -Eeuo pipefail
function runInNixShell() {
time nix-shell $CI_PROJECT_DIR/.gitlab/shell-aarch64-darwin.nix \
-I nixpkgs=https://github.com/angerman/nixpkgs/archive/75f7281738b.tar.gz \
--argstr system "aarch64-darwin" \
--pure \
--keep CI_PROJECT_DIR \
--keep MACOSX_DEPLOYMENT_TARGET \
--keep GHC_VERSION \
--keep CABAL_PROJECT \
--keep CABAL_INSTALL_VERSION \
--run "$1" 2>&1
}
runInNixShell "cabal update && mkdir vendored && cd vendored && cabal unpack network-3.1.2.1 && cd network-3.1.2.1 && autoreconf -fi" 2>&1
runInNixShell "./.gitlab/ci.sh" 2>&1
variables:
MACOSX_DEPLOYMENT_TARGET: "10.7"
parallel: *m1_matrix
artifacts:
expire_in: 2 week
paths:
- out

build-x86_64-windows:
extends: .build
tags:
- new-x86_64-windows
parallel: *default_matrix
51 changes: 51 additions & 0 deletions .gitlab/ci.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#!/usr/bin/env bash

set -Eeuxo pipefail

source "$CI_PROJECT_DIR/.gitlab/common.sh"


export GHCUP_INSTALL_BASE_PREFIX="$CI_PROJECT_DIR/toolchain"
export CABAL_DIR="$CI_PROJECT_DIR/cabal"

case "$(uname)" in
MSYS_*|MINGW*)
export CABAL_DIR="$(cygpath -w "$CABAL_DIR")"
GHCUP_BINDIR="${GHCUP_INSTALL_BASE_PREFIX}/ghcup/bin"
;;
*)
GHCUP_BINDIR="${GHCUP_INSTALL_BASE_PREFIX}/.ghcup/bin"
;;
esac

mkdir -p "$CABAL_DIR"
mkdir -p "$GHCUP_BINDIR"
export PATH="$GHCUP_BINDIR:$PATH"

export BOOTSTRAP_HASKELL_NONINTERACTIVE=1
export BOOTSTRAP_HASKELL_GHC_VERSION="$GHC_VERSION"
export BOOTSTRAP_HASKELL_CABAL_VERSION="$CABAL_INSTALL_VERSION"
export BOOTSTRAP_HASKELL_VERBOSE=1
export BOOTSTRAP_HASKELL_ADJUST_CABAL_CONFIG=yes

curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh

# some alpines need workaround
if ghc --info | grep -q integer-simple ; then
echo -e 'package blaze-textual\n flags: +integer-simple' >> cabal.project.local
fi

run cabal v2-install exe:haskell-language-server exe:haskell-language-server-wrapper \
-O2 \
-w "ghc-$GHC_VERSION" \
--project-file "$CABAL_PROJECT" \
--installdir="$CI_PROJECT_DIR/out" \
--install-method=copy \
--overwrite-policy=always \
--enable-executable-static \
--disable-profiling \
--disable-tests \
--enable-split-sections \
--enable-executable-stripping

cp dist-newstyle/cache/plan.json "$CI_PROJECT_DIR/out/plan.json"
49 changes: 49 additions & 0 deletions .gitlab/common.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Common bash utilities
# ----------------------

# Colors
BLACK="0;30"
GRAY="1;30"
RED="0;31"
LT_RED="1;31"
BROWN="0;33"
LT_BROWN="1;33"
GREEN="0;32"
LT_GREEN="1;32"
BLUE="0;34"
LT_BLUE="1;34"
PURPLE="0;35"
LT_PURPLE="1;35"
CYAN="0;36"
LT_CYAN="1;36"
WHITE="1;37"
LT_GRAY="0;37"

# GitLab Pipelines log section delimiters
# https://gitlab.com/gitlab-org/gitlab-foss/issues/14664
start_section() {
name="$1"
echo -e "section_start:$(date +%s):$name\015\033[0K"
}

end_section() {
name="$1"
echo -e "section_end:$(date +%s):$name\015\033[0K"
}

echo_color() {
local color="$1"
local msg="$2"
echo -e "\033[${color}m${msg}\033[0m"
}

error() { echo_color "${RED}" "$1"; }
warn() { echo_color "${LT_BROWN}" "$1"; }
info() { echo_color "${LT_BLUE}" "$1"; }

fail() { error "error: $1"; exit 1; }

function run() {
info "Running $*..."
"$@" || ( error "$* failed"; return 1; )
}
90 changes: 90 additions & 0 deletions .gitlab/shell-aarch64-darwin.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
{ system ? "aarch64-darwin"
#, nixpkgs ? fetchTarball https://github.com/angerman/nixpkgs/archive/257cb120334.tar.gz #apple-silicon.tar.gz
, pkgs ? import <nixpkgs> { inherit system; }
, compiler ? if system == "aarch64-darwin" then "ghc8103Binary" else "ghc8103"
}: pkgs.mkShell {
# this prevents nix from trying to write the env-vars file.
# we can't really, as NIX_BUILD_TOP/env-vars is not set.
noDumpEnvVars=1;

# stop polluting LDFLAGS with -liconv
dontAddExtraLibs = true;

# we need to inject ncurses into --with-curses-libraries.
# the real fix is to teach terminfo to use libcurses on macOS.
# CONFIGURE_ARGS = "--with-intree-gmp --with-curses-libraries=${pkgs.ncurses.out}/lib";
CONFIGURE_ARGS = "--with-intree-gmp --with-curses-libraries=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib --with-iconv-includes=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include --with-iconv-libraries=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib SH=/bin/bash";

# magic speedup pony :facepalm:
#
# nix has the ugly habbit of duplicating ld flags more than necessary. This
# somewhat consolidates this.
shellHook = ''
export NIX_LDFLAGS=$(for a in $NIX_LDFLAGS; do echo $a; done |sort|uniq|xargs)
export NIX_LDFLAGS_FOR_TARGET=$(for a in $NIX_LDFLAGS_FOR_TARGET; do echo $a; done |sort|uniq|xargs)
export NIX_LDFLAGS_FOR_TARGET=$(comm -3 <(for l in $NIX_LDFLAGS_FOR_TARGET; do echo $l; done) <(for l in $NIX_LDFLAGS; do echo $l; done))


# Impurity hack for GHC releases.
#################################
# We don't want binary releases to depend on nix, thus we'll need to make sure we don't leak in references.
# GHC externally depends only on iconv and curses. However we can't force a specific curses library for
# the terminfo package, as such we'll need to make sure we only look in the system path for the curses library
# and not pick up the tinfo from the nix provided ncurses package.
#
# We also need to force us to use the systems COREFOUNDATION, not the one that nix builds. Again this is impure,
# but it will allow us to have proper binary distributions.
#
# do not use nixpkgs provided core foundation
export NIX_COREFOUNDATION_RPATH=/System/Library/Frameworks
# drop curses from the LDFLAGS, we really want the system ones, not the nix ones.
export NIX_LDFLAGS=$(for lib in $NIX_LDFLAGS; do case "$lib" in *curses*);; *) echo -n "$lib ";; esac; done;)
export NIX_CFLAGS_COMPILE+=" -Wno-nullability-completeness -Wno-availability -Wno-expansion-to-defined -Wno-builtin-requires-header -Wno-unused-command-line-argument"

# unconditionally add the MacOSX.sdk and TargetConditional.h
export NIX_CFLAGS_COMPILE+=" -isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include"
export NIX_LDFLAGS="-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib $NIX_LDFLAGS"

'';

nativeBuildInputs = (with pkgs; [
# This needs to come *before* ghc,
# otherwise we migth end up with the clang from
# the bootstrap GHC in PATH with higher priority.
clang_11
llvm_11

haskell.compiler.${compiler}
haskell.packages.${compiler}.cabal-install
haskell.packages.${compiler}.alex
haskell.packages.${compiler}.happy # _1_19_12 is needed for older GHCs.

automake
autoconf
m4

gmp
zlib.out
zlib.dev
glibcLocales
# locale doesn't build yet :-/
# locale

git

python3
# python3Full
# python3Packages.sphinx
perl

which
wget
curl
file

xz
xlibs.lndir

cacert ])
++ (with pkgs.darwin.apple_sdk.frameworks; [ Foundation Security ]);
}
2 changes: 2 additions & 0 deletions cabal.project
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ packages:
./plugins/hls-ormolu-plugin
./plugins/hls-call-hierarchy-plugin

optional-packages: vendored/*/*.cabal

Comment on lines +31 to +32
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's this for?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being able to use vendored packages. If they don't exist, nothing happens. If they exist, they get picked up.

This is required because the network package is broken on M1 and needs a autoreconf (but cabal is unable to do so for you). So we do it manually and put it into this directory.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it probably deserves a comment "Used by the M1 build script"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm it is a somewhat standard way to include potential optional packages in any situation but a comment with an actual use case would be fine

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's (evidently) not obvious to the most active maintainers of this project, it definitely needs a comment for the rest of us poor mortals :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment could even point to the script where it's used!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know, github has a feature to add code suggestions ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hasufell i hope you dont mind i've added a pair of comments directly in your branch

tests: true

package *
Expand Down