ARROW-18401: [R] Failing test on test-r-rhub-ubuntu-gcc-release-latest

[paleolimbot]

No description provided.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-18401

[github-actions]

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

[paleolimbot]

@github-actions crossbow submit test-r-rhub-ubuntu-gcc-release-latest

[github-actions]

Wrong oauth personal access token
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/3651840905

[paleolimbot]

@github-actions crossbow submit test-r-rhub-ubuntu-gcc-release-latest

[github-actions]

Wrong oauth personal access token
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/3651882824

[paleolimbot]

@github-actions crossbow submit test-r-rhub-ubuntu-gcc-release-latest

[github-actions]

Wrong oauth personal access token
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/3659993751

[paleolimbot]

@github-actions crossbow submit test-r-rhub-ubuntu-gcc-release-latest

[github-actions]

Wrong oauth personal access token
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/3675749754

[paleolimbot]

@github-actions crossbow submit test-r-rhub-ubuntu-gcc-release-latest

[github-actions]

Revision: 4b67dd3

Submitted crossbow builds: ursacomputing/crossbow @ actions-0fcecc8ac9

Task	Status
test-r-rhub-ubuntu-gcc-release-latest

[paleolimbot]

Ok, the error is:

Error: Invalid: Cannot locate timezone '/UTC': /UTC not found in timezone database

[paleolimbot]

Sure enough, Sys.timezone() is returning "/UTC":

dewey@Deweys-Mac-mini arrow % docker run --rm -it rhub/ubuntu-gcc-release bash
root@d5a03ff5a71d:/# timedatectl
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to create bus connection: Host is down
root@d5a03ff5a71d:/# R

R version 4.2.2 Patched (2022-11-10 r83330) -- "Innocent and Trusting"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> Sys.timezone()
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to create bus connection: Host is down
[1] "/UTC"
Warning message:
In system("timedatectl", intern = TRUE) :
  running command 'timedatectl' had status 1

@assignUser do you have any ideas about what could change in the image to make this not be the case?

[assignUser]

The problem is this:

root@eaa7ac74e396:/# cat /etc/timezone
/UTC

For some reason the timezone is prepended with / and this gets picked up by Sys.timezone

[assignUser]

Got it, is caused by noninteractive install of tzdata setting /UTC as default timezone:

Preparing to unpack .../tzdata_2022g-0ubuntu0.20.04.1_all.deb ...
Unpacking tzdata (2022g-0ubuntu0.20.04.1) ...
Setting up tzdata (2022g-0ubuntu0.20.04.1) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
Configuring tzdata
------------------

Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.

  1. Africa      4. Australia  7. Atlantic  10. Pacific  13. Etc
  2. America     5. Arctic     8. Europe    11. SystemV
  3. Antarctica  6. Asia       9. Indian    12. US
Geographic area: 
Use of uninitialized value $_[1] in join or string at /usr/share/perl5/Debconf/DbDriver/Stack.pm line 111.

Current default time zone: '/UTC'

[paleolimbot]

You're a genius. I have ENV TZ=America/Halifax sitting in one of my Docker images that I use for Arrow dev...would ENV TZ=UTC do it?

[assignUser]

I think this is either an issue with the Dockerfile (I looked at it and couldn't see anything obvious) or it might be a packaging issue of tzdata but I am not familiar enough with linux packaging... @kou thoughts?

[assignUser]

would ENV TZ=UTC do it?

Looking at Sys.timezone: tz <- Sys.getenv("TZ") that should do the trick :)

[paleolimbot]

I have a fork of rhub's linux builders from the last time something like this happened...I'll experiment and try to fix this upstream tomorrow.

Before I go on a goose chase, does anybody know off the top of their head which file I would modify to override this?

[assignUser]

tzdata is installed here: https://github.com/r-hub/rhub-linux-builders/blob/master/ubuntu/Dockerfile

[kou]

I think that this is a problem of Dockerfile. If we reinstall tzdata, Etc/UTC (expected value) is used:

$ docker run -it --rm rhub/ubuntu-gcc-release bash
root@496da8432c2e:/# cat /etc/timezone
/UTC
root@496da8432c2e:/# apt purge -y tzdata > /dev/null

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

root@496da8432c2e:/# echo "debconf debconf/frontend select Noninteractive" | debconf-set-selections
root@496da8432c2e:/# apt install -y tzdata
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  autoconf automake autopoint autotools-dev build-essential dbus-user-session dconf-gsettings-backend dconf-service
  dh-strip-nondeterminism distro-info-data dpkg-dev dwz fontconfig gettext-base gir1.2-freedesktop gir1.2-glib-2.0
  gir1.2-harfbuzz-0.0 gir1.2-packagekitglib-1.0 gir1.2-pango-1.0 glib-networking glib-networking-common glib-networking-services
  gsettings-desktop-schemas iso-codes libapache-pom-java libarchive-zip-perl libargon2-1 libbibutils6 libcap2 libcap2-bin
  libcommons-logging-java libcommons-parent-java libcryptsetup12 libdatrie-dev libdatrie1 libdconf1 libdebhelper-perl
  libdevmapper1.02.1 libfile-stripnondeterminism-perl libfontbox-java libfribidi-dev libfribidi0 libghc-pandoc-citeproc-data
  libgirepository-1.0-1 libgraphite2-dev libgstreamer1.0-0 libharfbuzz-gobject0 libip4tc2 libjson-c4 libkmod2 libkpathsea6 liblmdb0
  libmime-charset-perl libpackagekit-glib2-18 libpam-systemd libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpangoxft-1.0-0
  libpdfbox-java libpipeline1 libpolkit-agent-1-0 libpolkit-gobject-1-0 libproxy1v5 libptexenc1 libsombok3 libstemmer0d
  libsub-override-perl libsynctex2 libteckit0 libtexlua53 libtexluajit2 libtext-unidecode-perl libthai-data libthai-dev libthai0
  libtool libunicode-linebreak-perl libxft-dev libxft2 libxml-namespacesupport-perl libxml-sax-base-perl libxml-sax-perl libxss-dev
  libxss1 libyaml-0-2 libzzip-0-13 lsb-release make man-db pango1.0-tools patch policykit-1 preview-latex-style python-apt-common
  python3-apt python3-certifi python3-chardet python3-dbus python3-gi python3-idna python3-pkg-resources python3-requests
  python3-requests-unixsocket python3-six python3-software-properties python3-urllib3 r-doc-html systemd systemd-sysv
  systemd-timesyncd t1utils x11proto-scrnsaver-dev zip
Use 'apt autoremove' to remove them.
The following NEW packages will be installed:
  tzdata
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 286 kB of archives.
After this operation, 3,948 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 tzdata all 2022g-0ubuntu0.20.04.1 [286 kB]
Fetched 286 kB in 2s (114 kB/s)  
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package tzdata.
(Reading database ... 26551 files and directories currently installed.)
Preparing to unpack .../tzdata_2022g-0ubuntu0.20.04.1_all.deb ...
Unpacking tzdata (2022g-0ubuntu0.20.04.1) ...
Setting up tzdata (2022g-0ubuntu0.20.04.1) ...

Current default time zone: 'Etc/UTC'
Local time is now:      Thu Dec 15 02:37:30 UTC 2022.
Universal Time is now:  Thu Dec 15 02:37:30 UTC 2022.
Run 'dpkg-reconfigure tzdata' if you wish to change it.

root@496da8432c2e:/# cat /etc/timezone 
Etc/UTC

How about reporting this to https://github.com/r-hub/rhub-linux-builders ?

[eitsupi]

FYI, when we switched the base image of rocker/r-ver:latest from ubuntu:focal to ubuntu:jammy, the similar problem occurred and we had to update our Dockerfile to explicitly install tzdata.
rocker-org/rocker-versioned2#559

[paleolimbot]

@github-actions crossbow submit test-r-rhub-ubuntu-gcc-release-latest

[github-actions]

Revision: 96bd683

Submitted crossbow builds: ursacomputing/crossbow @ actions-e090e45d14

Task	Status
test-r-rhub-ubuntu-gcc-release-latest

[ursabot]

Benchmark runs are scheduled for baseline = eeeab9b and contender = b640b94. b640b94 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.59% ⬆️0.0%] test-mac-arm
[Finished ⬇️0.27% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.62% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] b640b943 ec2-t3-xlarge-us-east-2
[Finished] b640b943 test-mac-arm
[Finished] b640b943 ursa-i9-9960x
[Finished] b640b943 ursa-thinkcentre-m75q
[Finished] eeeab9b1 ec2-t3-xlarge-us-east-2
[Finished] eeeab9b1 test-mac-arm
[Finished] eeeab9b1 ursa-i9-9960x
[Finished] eeeab9b1 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

[ursabot]

['Python', 'R'] benchmarks have high level of regressions.
test-mac-arm