Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] run r-package Linux jobs in containers #5638

Merged
merged 39 commits into from
Jan 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
7cd6512
[ci] run r-package Linux jobs in containers
jameslamb Dec 16, 2022
3d87bd3
comment out CI
jameslamb Dec 16, 2022
d206222
remove extra dollar sign
jameslamb Dec 16, 2022
841577b
install git before checkout
jameslamb Dec 16, 2022
6653141
add certificates
jameslamb Dec 16, 2022
acbef20
install more stuff
jameslamb Dec 16, 2022
d67aa05
actually run tests
jameslamb Dec 16, 2022
ce5f897
use the IN_UBUNTU_LATEST_CONTAINER setup.sh stuff
jameslamb Dec 16, 2022
5a1df8c
fail earlier
jameslamb Dec 16, 2022
217fa7a
try changing apt repo
jameslamb Dec 16, 2022
3a6b7cf
add key-management stuff
jameslamb Dec 16, 2022
866072a
more packaging things
jameslamb Dec 16, 2022
327dfe0
fix locale
jameslamb Dec 16, 2022
9783583
testing
jameslamb Dec 16, 2022
1200bae
ensure we get the expected version of R
jameslamb Dec 16, 2022
951fd2b
ensure locale is set successfully
jameslamb Dec 20, 2022
dbfa42d
run old R on Ubuntu 18.04
jameslamb Dec 20, 2022
20ede2f
re-generate locale
jameslamb Dec 20, 2022
265931a
install newest version of git
jameslamb Dec 20, 2022
c59fe90
install add-apt-repository
jameslamb Dec 20, 2022
cbc4d32
move more stuff up prior to third-party actions
jameslamb Dec 20, 2022
587f070
re-organize
jameslamb Dec 21, 2022
be480a8
try to fix locale stuff
jameslamb Dec 21, 2022
c0d5106
set locale environment variables
jameslamb Dec 21, 2022
f11827d
add automake to get 'aclocal'
jameslamb Dec 21, 2022
5d96abc
test updating to newest cmake version
jameslamb Dec 21, 2022
fe82bac
install newest cmake
jameslamb Dec 21, 2022
d69bd00
skip license
jameslamb Dec 21, 2022
604f413
check R version, re-enable macOS jobs
jameslamb Dec 21, 2022
fa7fa85
*sighs in YAML*
jameslamb Dec 21, 2022
1009411
fix checks
jameslamb Dec 21, 2022
e256f8c
trust source dir
jameslamb Dec 21, 2022
6902822
revert encoding stuff that is now on master
jameslamb Dec 28, 2022
3c4e96e
env var
jameslamb Dec 28, 2022
4ba0cfb
merge master
jameslamb Dec 28, 2022
e6ba86a
restore CI jobs
jameslamb Dec 28, 2022
598a7d6
restore Windows R jobs
jameslamb Dec 28, 2022
5eeafa0
only install newer CMake on Ubuntu 18.04 builds
jameslamb Dec 28, 2022
4f2d268
Merge branch 'master' into ci/r-package-containers
jameslamb Dec 29, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 37 additions & 5 deletions .ci/test_r_package.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ if [[ "${R_MAJOR_VERSION}" == "3" ]]; then
elif [[ "${R_MAJOR_VERSION}" == "4" ]]; then
export R_MAC_VERSION=4.2.2
export R_MAC_PKG_URL=${CRAN_MIRROR}/bin/macosx/base/R-${R_MAC_VERSION}.pkg
export R_LINUX_VERSION="4.2.2-1.2004.0"
export R_APT_REPO="focal-cran40/"
export R_LINUX_VERSION="4.2.2-1.2204.0"
export R_APT_REPO="jammy-cran40/"
else
echo "Unrecognized R version: ${R_VERSION}"
exit -1
Expand All @@ -38,14 +38,15 @@ fi
if [[ $OS_NAME == "linux" ]]; then
sudo apt-key adv \
--keyserver keyserver.ubuntu.com \
--recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
--recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9 || exit -1
sudo add-apt-repository \
"deb ${CRAN_MIRROR}/bin/linux/ubuntu ${R_APT_REPO}"
"deb ${CRAN_MIRROR}/bin/linux/ubuntu ${R_APT_REPO}" || exit -1
sudo apt-get update
sudo apt-get install \
--no-install-recommends \
-y --allow-downgrades \
-y \
devscripts \
r-base-core=${R_LINUX_VERSION} \
r-base-dev=${R_LINUX_VERSION} \
texinfo \
texlive-latex-extra \
Expand All @@ -60,8 +61,18 @@ if [[ $OS_NAME == "linux" ]]; then
--no-install-recommends \
-y \
autoconf=$(cat R-package/AUTOCONF_UBUNTU_VERSION) \
automake \
|| exit -1
fi
if [[ $INSTALL_CMAKE_FROM_RELEASES == "true" ]]; then
curl -O -L \
https://github.com/Kitware/CMake/releases/download/v3.25.1/cmake-3.25.1-linux-x86_64.sh \
|| exit -1

sudo mkdir /opt/cmake || exit -1
sudo sh cmake-3.25.1-linux-x86_64.sh --skip-license --prefix=/opt/cmake || exit -1
sudo ln -s /opt/cmake/bin/cmake /usr/local/bin/cmake || exit -1
fi
fi

# Installing R precompiled for Mac OS 10.11 or higher
Expand Down Expand Up @@ -211,6 +222,27 @@ if [[ $check_succeeded == "no" ]]; then
exit -1
fi

used_correct_r_version=$(
cat $LOG_FILE_NAME \
| grep --count "using R version ${R_VERSION}"
)
if [[ $used_correct_r_version -ne 1 ]]; then
echo "Unexpected R version was used. Expected '${R_VERSION}'."
exit -1
fi

if [[ $R_BUILD_TYPE == "cmake" ]]; then
passed_correct_r_version_to_cmake=$(
cat $BUILD_LOG_FILE \
| grep --count "R version passed into FindLibR.cmake: ${R_VERSION}"
)
if [[ $used_correct_r_version -ne 1 ]]; then
echo "Unexpected R version was passed into cmake. Expected '${R_VERSION}'."
exit -1
fi
fi


if grep -q -E "NOTE|WARNING|ERROR" "$LOG_FILE_NAME"; then
echo "NOTEs, WARNINGs, or ERRORs have been found by R CMD check"
exit -1
Expand Down
64 changes: 56 additions & 8 deletions .github/workflows/r_package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:
test:
name: ${{ matrix.task }} (${{ matrix.os }}, ${{ matrix.compiler }}, R ${{ matrix.r_version }}, ${{ matrix.build_type }})
runs-on: ${{ matrix.os }}
container: ${{ matrix.container }}
timeout-minutes: 60
strategy:
fail-fast: false
Expand All @@ -38,72 +39,84 @@ jobs:
################
# CMake builds #
################
- os: ubuntu-22.04
- os: ubuntu-latest
task: r-package
compiler: gcc
r_version: 3.6
build_type: cmake
- os: ubuntu-22.04
container: 'ubuntu:18.04'
- os: ubuntu-latest
task: r-package
compiler: gcc
r_version: 4.2
build_type: cmake
- os: ubuntu-22.04
container: 'ubuntu:22.04'
- os: ubuntu-latest
task: r-package
compiler: clang
r_version: 3.6
build_type: cmake
- os: ubuntu-22.04
container: 'ubuntu:18.04'
- os: ubuntu-latest
task: r-package
compiler: clang
r_version: 4.2
build_type: cmake
container: 'ubuntu:22.04'
- os: macOS-latest
task: r-package
compiler: gcc
r_version: 3.6
build_type: cmake
container: null
- os: macOS-latest
task: r-package
compiler: gcc
r_version: 4.2
build_type: cmake
container: null
- os: macOS-latest
task: r-package
compiler: clang
r_version: 3.6
build_type: cmake
container: null
- os: macOS-latest
task: r-package
compiler: clang
r_version: 4.2
build_type: cmake
container: null
- os: windows-latest
task: r-package
compiler: MINGW
toolchain: MINGW
r_version: 3.6
build_type: cmake
container: null
- os: windows-latest
task: r-package
compiler: MINGW
toolchain: MSYS
r_version: 4.2
build_type: cmake
container: null
# Visual Studio 2019
- os: windows-2019
task: r-package
compiler: MSVC
toolchain: MSVC
r_version: 3.6
build_type: cmake
container: null
# Visual Studio 2022
- os: windows-2022
task: r-package
compiler: MSVC
toolchain: MSVC
r_version: 4.2
build_type: cmake
container: null
###############
# CRAN builds #
###############
Expand All @@ -113,35 +126,64 @@ jobs:
toolchain: MINGW
r_version: 3.6
build_type: cran
container: null
- os: windows-latest
task: r-package
compiler: MINGW
toolchain: MSYS
r_version: 4.2
build_type: cran
- os: ubuntu-22.04
container: null
- os: ubuntu-latest
task: r-package
compiler: gcc
r_version: 4.2
build_type: cran
container: 'ubuntu:22.04'
- os: macOS-latest
task: r-package
compiler: clang
r_version: 4.2
build_type: cran
container: null
################
# Other checks #
################
- os: ubuntu-22.04
- os: ubuntu-latest
task: r-rchk
compiler: gcc
r_version: 4.2
build_type: cran
container: 'ubuntu:22.04'
steps:
- name: Prevent conversion of line endings on Windows
if: startsWith(matrix.os, 'windows')
shell: pwsh
run: git config --global core.autocrlf false
- name: Install packages used by third-party actions
if: startsWith(matrix.os, 'ubuntu')
shell: bash
run: |
apt-get update -y
apt-get install --no-install-recommends -y \
ca-certificates \
dirmngr \
gpg \
gpg-agent \
software-properties-common \
sudo
# install newest version of git
# ref:
# - https://unix.stackexchange.com/a/170831/550004
# - https://git-scm.com/download/linux
add-apt-repository ppa:git-core/ppa -y
apt-get update -y
apt-get install --no-install-recommends -y \
git
- name: Trust git cloning LightGBM
if: startsWith(matrix.os, 'ubuntu')
run: |
git config --global --add safe.directory "${GITHUB_WORKSPACE}"
- name: Checkout repository
uses: actions/checkout@v3
with:
Expand All @@ -156,16 +198,22 @@ jobs:
CTAN_MIRROR: https://ctan.math.illinois.edu/systems/win32/miktex
TINYTEX_INSTALLER: TinyTeX
- name: Setup and run tests on Linux and macOS
if: matrix.os == 'macOS-latest' || matrix.os == 'ubuntu-22.04'
if: matrix.os == 'macOS-latest' || matrix.os == 'ubuntu-latest'
shell: bash
run: |
export TASK="${{ matrix.task }}"
export COMPILER="${{ matrix.compiler }}"
export GITHUB_ACTIONS="true"
if [[ "${{ matrix.os }}" == "macOS-latest" ]]; then
export OS_NAME="macos"
elif [[ "${{ matrix.os }}" == "ubuntu-22.04" ]]; then
elif [[ "${{ matrix.os }}" == "ubuntu-latest" ]]; then
export OS_NAME="linux"
export IN_UBUNTU_BASE_CONTAINER="true"
# the default version of cmake provided on Ubuntu 18.04 (v3.10.2), is not supported by LightGBM
# see https://github.com/microsoft/LightGBM/issues/5642
if [[ "${{ matrix.container }}" == "ubuntu:18.04" ]]; then
export INSTALL_CMAKE_FROM_RELEASES="true"
fi
fi
export BUILD_DIRECTORY="$GITHUB_WORKSPACE"
export R_VERSION="${{ matrix.r_version }}"
Expand Down