Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This MySQL server doesn't support dates later than 2038 #47

Closed
wieluk opened this issue Apr 18, 2021 · 21 comments
Closed

This MySQL server doesn't support dates later than 2038 #47

wieluk opened this issue Apr 18, 2021 · 21 comments

Comments

@wieluk
Copy link

wieluk commented Apr 18, 2021

Hi I am getting this error i am trying to use this db on rasbian 32 with nextcloud anyway to fix it?

@Slartibartfast27
Copy link

Slartibartfast27 commented Apr 18, 2021

I also have this problem on an 32 bit armv7 architecture:

root# uname -a
Linux odroid2020 5.4.98-odroidxu4 #21.02.2 SMP PREEMPT Sun Feb 14 22:29:57 CET 2021 armv7l armv7l armv7l GNU/Linux

10.4.17 works fine.
Only 10.5.8 has this problem.

Steps to reproduce with 10.5.8 (this burns the CPU for about 5 minutes on an https://www.hardkernel.com/shop/odroid-xu4q-special-price/ and then stops the container):

root# docker run --rm yobasystems/alpine-mariadb:10.5.8
[i] mysqld not found, creating....
[i] MySQL data directory not found, creating initial DBs
340444-06-16 16:13:00 0 [ERROR] This MySQL server doesn't support dates later than 2038
[i] MySQL root Password: nohl2Eex2SohZuth
4282-12-18 17:49:17 0 [ERROR] This MySQL server doesn't support dates later than 2038
/scripts/run.sh: ignoring or entrypoint initdb empty /docker-entrypoint-initdb.d/*


MySQL init process done. Ready for start up.

exec /usr/bin/mysqld --user=mysql --console --skip-name-resolve --skip-networking=0
442234-11-13 17:14:34 0 [ERROR] This MySQL server doesn't support dates later than 2038
root#

Just for comparison: 10.4.17 working fine:

root# docker run --rm yobasystems/alpine-mariadb:10.4.17
[i] mysqld not found, creating....
[i] MySQL data directory not found, creating initial DBs
...
MySQL init process done. Ready for start up.
...
2021-04-18  5:50:22 0 [Note] /usr/bin/mysqld: ready for connections.
Version: '10.4.17-MariaDB'  socket: '/run/mysqld/mysqld.sock'  port: 3306  MariaDB Server

@dominictayloruk
Copy link
Member

For any containers running Alpine 3.13 you need a minimum Docker version on the host of 19.03.9. More info here

time64 requirements

The following important information applies for users of x86, armv7, and armhf (currently supported 32-bit architectures), including 32-bit Docker containers on 64-bit hosts.

All self-compiled packages must be manually rebuilt after upgrading, even if relocation/SONAME errors are not encountered.

musl 1.2 uses new time64-compatible system calls. Due to runc issue 2151, these system calls incorrectly return EPERM instead of ENOSYS when invoked under a Docker or libseccomp version predating their release. Therefore, Alpine Linux 3.13.0 requires the host Docker to be version 19.03.9 (which contains backported moby commit 89fabf0) or greater and the host libseccomp to be version 2.4.2 (which contains backported libseccomp commit bf747eb) or greater. Docker for Windows issue 8326 tracks the process of updating libseccomp in Docker for Windows.

Therefore, the following platforms are not suitable as Docker hosts for 32-bit Alpine Linux 3.13.0, due to containing out-of-date libseccomp: Amazon Linux 1 or 2, CentOS 7 or 8, Debian stable without debian-backports, Raspbian stable, Ubuntu 14.04 or earlier, and Windows. This applies regardless of whether the Linux distribution Docker packages or separate Docker package repositories are used.

To check if your host libseccomp is time64-compatible, invoke scmp_sys_resolver -a x86 clock_gettime64 for x86 containers, or scmp_sys_resolver -a arm clock_gettime64 for armhf or armv7 containers. If 403 is returned, time64 is supported. If -1 is returned, time64 is not supported. Note that Docker must still be at least version 19.03.9, regardless of the result of this command.

In order to run under old Docker or libseccomp versions, the moby default seccomp profile should be downloaded and on line 2, defaultAction changed to SCMP_ACT_TRACE, then --seccomp-profile=default.json can be passed to dockerd, or --security-opt=seccomp=default.json passed to docker create or docker run. This will cause the system calls to return ENOSYS instead of EPERM, allowing the container to fall back to 32-bit time system calls. In this case, the container will not be compatible with dates past 2038.

Alternatively, --security-opt=seccomp=unconfined can be passed with no default.json required, but note that this will reduce the security of the host against malicious code in the container.

@Slartibartfast27
Copy link

Thanks for the extremely fast answer ;-)
At the moment I'm running on docker 19.03.8

docker --version
Docker version 19.03.8, build afacb8b7f0

I'll check again as soon as Docker version 19.03.9 is available via apt on my system ( https://www.armbian.com/ )

@dominictayloruk
Copy link
Member

dominictayloruk commented Apr 18, 2021

Armbian is what i'm running on the tinkerboard builders. I updated docker like so;

Remove Docker

sudo apt-get remove docker docker-engine docker.io containerd runc

Edit the sources file

sudo vi /etc/apt/sources.list.d/docker.list

You want to change 'focal' to 'bionic' so it looks like

deb [arch=armhf] https://download.docker.com/linux/ubuntu bionic stable

Then we update apt source and install

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

This should leave you with Docker version 20.10.6, build 370c289

@swood387
Copy link

Same Error:

This MySQL server doesn't support dates later than 2038

System:
Raspberry PI: Raspbian GNU/Linux 10 (buster)
Docker: Docker version 20.10.6, build 370c289

@dominictayloruk
Copy link
Member

I get this running on a Asus Tinkerboard;

OS VERSION

Armbian 21.02.3 Focal with Linux 5.10.21-rockchip

KERNEL

Linux k3s-armhf-006 5.10.21-rockchip #21.02.3 SMP PREEMPT Mon Mar 8 07:34:33 UTC 2021 armv7l armv7l armv7l GNU/Linux

DOCKER VERSION

Docker version 20.10.6, build 370c289

CONTAINER LOGS

[i] mysqld not found, creating....
[i] MySQL data directory not found, creating initial DBs
[i] Creating database: kubernetes-secret-test
[i] with character set [utf8mb4] and collation [utf8mb4_general_ci]
[i] Creating user: kubernetes-secret-test with password sCbM45P92kkCmWYpH24
2021-04-18 15:07:06 0 [Note] /usr/bin/mysqld (mysqld 10.5.8-MariaDB) starting as process 54 ...
2021-04-18 15:07:06 0 [Note] InnoDB: Using Linux native AIO
2021-04-18 15:07:06 0 [Note] InnoDB: Uses event mutexes
2021-04-18 15:07:06 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-04-18 15:07:06 0 [Note] InnoDB: Number of pools: 1
2021-04-18 15:07:06 0 [Note] InnoDB: Using generic crc32 instructions
2021-04-18 15:07:06 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2021-04-18 15:07:06 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2021-04-18 15:07:06 0 [Note] InnoDB: Completed initialization of buffer pool
2021-04-18 15:07:06 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-04-18 15:07:06 0 [Note] InnoDB: 128 rollback segments are active.
2021-04-18 15:07:06 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-04-18 15:07:06 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-04-18 15:07:06 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-04-18 15:07:06 0 [Note] InnoDB: 10.5.8 started; log sequence number 45118; transaction id 20
2021-04-18 15:07:06 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2021-04-18 15:07:06 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-04-18 15:07:06 0 [Note] InnoDB: Buffer pool(s) load completed at 210418 15:07:06
2021-04-18 15:07:06 1 [Warning] 'user' entry '@test-db-secrets-5fc59d7d8c-n562r' ignored in --skip-name-resolve mode.
2021-04-18 15:07:06 1 [Warning] 'proxies_priv' entry '@% root@test-db-secrets-5fc59d7d8c-n562r' ignored in --skip-name-resolve mode.
2021-04-18 15:07:06 1 [Warning] 'user' entry '@test-db-secrets-5fc59d7d8c-n562r' ignored in --skip-name-resolve mode.
2021-04-18 15:07:06 1 [Warning] 'proxies_priv' entry '@% root@test-db-secrets-5fc59d7d8c-n562r' ignored in --skip-name-resolve mode.
/scripts/run.sh: ignoring or entrypoint initdb empty /docker-entrypoint-initdb.d/*
MySQL init process done. Ready for start up.
exec /usr/bin/mysqld --user=mysql --console --skip-name-resolve --skip-networking=0
2021-04-18 15:07:07 0 [Note] /usr/bin/mysqld (mysqld 10.5.8-MariaDB) starting as process 1 ...
2021-04-18 15:07:07 0 [Note] InnoDB: Using Linux native AIO
2021-04-18 15:07:07 0 [Note] InnoDB: Uses event mutexes
2021-04-18 15:07:07 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-04-18 15:07:07 0 [Note] InnoDB: Number of pools: 1
2021-04-18 15:07:07 0 [Note] InnoDB: Using generic crc32 instructions
2021-04-18 15:07:07 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2021-04-18 15:07:08 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2021-04-18 15:07:08 0 [Note] InnoDB: Completed initialization of buffer pool
2021-04-18 15:07:08 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-04-18 15:07:08 0 [Note] InnoDB: 128 rollback segments are active.
2021-04-18 15:07:08 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-04-18 15:07:08 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-04-18 15:07:08 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-04-18 15:07:08 0 [Note] InnoDB: 10.5.8 started; log sequence number 45130; transaction id 20
2021-04-18 15:07:08 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2021-04-18 15:07:08 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-04-18 15:07:08 0 [Note] InnoDB: Buffer pool(s) load completed at 210418 15:07:08
2021-04-18 15:07:08 0 [Note] Server socket created on IP: '::'.
2021-04-18 15:07:08 0 [Warning] 'user' entry '@test-db-secrets-5fc59d7d8c-n562r' ignored in --skip-name-resolve mode.
2021-04-18 15:07:08 0 [Warning] 'proxies_priv' entry '@% root@test-db-secrets-5fc59d7d8c-n562r' ignored in --skip-name-resolve mode.
2021-04-18 15:07:08 0 [Note] Reading of all Master_info entries succeeded
2021-04-18 15:07:08 0 [Note] Added new Master_info '' to hash table
2021-04-18 15:07:08 0 [Note] /usr/bin/mysqld: ready for connections.
Version: '10.5.8-MariaDB'  socket: '/run/mysqld/mysqld.sock'  port: 3306  MariaDB Server
2021-04-18 15:08:40 3 [Warning] Access denied for user 'root'@'localhost' (using password: NO)
2021-04-18 15:08:48 4 [Warning] Access denied for user 'root'@'localhost' (using password: NO)

KUBERNETES YAML

apiVersion: apps/v1
kind: Deployment
metadata:
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
  strategy:
    type: Recreate
  template:
    metadata:
      annotations:
      creationTimestamp: null
      labels:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm
      containers:
      - envFrom:
        - secretRef:
            name: test-db
            optional: false
        image: yobasystems/alpine-mariadb
        imagePullPolicy: Always
        name: test-db-secrets
        resources:
          limits:
            cpu: "1"
            memory: 512Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities: {}
          privileged: false
          readOnlyRootFilesystem: false
          runAsNonRoot: false
        stdin: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        tty: true
      dnsConfig: {}
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2021-04-18T15:06:51Z"
    lastUpdateTime: "2021-04-18T15:06:51Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2021-04-18T15:04:19Z"
    lastUpdateTime: "2021-04-18T15:06:51Z"
    message: ReplicaSet "test-db-secrets-5fc59d7d8c" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 4
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

@zynaa
Copy link

zynaa commented Apr 18, 2021

I also started getting This MySQL server doesn't support dates later than 2038 and constant 100% CPU usage on my RPI4. Reverting to version 10.4.17 fixes the issue.

System: Linux rpi 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux
Docker: Docker version 20.10.6, build 370c289

@dominictayloruk
Copy link
Member

I also started getting This MySQL server doesn't support dates later than 2038 and constant 100% CPU usage on my RPI4. Reverting to version 10.4.17 fixes the issue.

System: Linux rpi 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux
Docker: Docker version 20.10.6, build 370c289

The rpi4 arm64 host isn't susceptible to this problem, only 32bit arm hosts, arm/armhf, it's because you are running a 32bit OS on the rpi4. You should really run a 64bit OS, like so;

Linux k3s-w001 5.4.0-1032-raspi #35-Ubuntu SMP PREEMPT Fri Mar 19 20:52:40 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

@dominictayloruk
Copy link
Member

I don't get the other issues but suffer from 100% cpu usage on armhf board.
Just been researching this a little more and it seems to be a problem,

Maybe an update to MariaDB 10.5.9 will resolve this, apparently it's a musl problem.

@NeoTrace82
Copy link

I also started getting This MySQL server doesn't support dates later than 2038 and constant 100% CPU usage on my RPI4. Reverting to version 10.4.17 fixes the issue.

System: Linux rpi 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux
Docker: Docker version 20.10.6, build 370c289

thanks for the tip, solved it for me for now, hope there will be soon a working 10.5.x

@Slartibartfast27
Copy link

How about adding and maintaining some new tags

yobasystems/alpine-mariadb:10.3
yobasystems/alpine-mariadb:10.4
yobasystems/alpine-mariadb:10.5

This would allow us to use the latest 10.3 / 10.4 / 10.5 without (accidently) switching to a new major version, e.g.
10.4 would automatically switch from 10.4.15 --> 10.4.16 --> 10.4.17 but NOT to 10.5.x

Switching major versions typically also requires additional maintenance:
https://mariadb.com/kb/en/mysql_upgrade/
I'm not sure, whether the mysql_upgrade is done when starting the container. Therefore I strongly recommend never to use tag "latest" but always use tags 10.3 / 10.4 / 10.5.

This would allow us to avoid 10.5 as long as it has problems, but still stay on the latest and greatest 10.4 ;-)

https://hub.docker.com/_/mariadb has the same tag convention (10.3 / 10.4 / 10.5)

@dominictayloruk
Copy link
Member

There are tags on this image that are pinned to versions which you should be using anyway for production.

yobasystems/alpine-mariadb:10.4.17
yobasystems/alpine-mariadb:10.5.8

These are multi arch too. I wouldn't recommend using :latest for any production workloads and if your cluster is only armhf nodes then stick to 10.4.17 due to the cpu bug.

@leolivier
Copy link

leolivier commented Apr 24, 2021

Hi, I get the same problem here but downgrading to 10.4.17 in my nextcloud stack sends an error and mariadb doesn't start:
"Unsupported redo log format. The redo log was created with MariaDB 10.5.8."
I don't find on the internet any way to manage that.

I'm very concerned about this issue, I can't run my nextcloud anymore :/
With the latest version, I use 100% of the CPU and I can't revert to previous version (MariaDB site says:
Downgrading MariaDB is not supported. The only reliable way to downgrade is to restore from a full backup made before upgrading, and start the old version of MariaDB.)

Do you plan to release a 10.5.9 version any time soon?

@WillJBrown
Copy link

WillJBrown commented May 6, 2021

Same issue for me - I've ended up implementing cpu_quota: 5000 in my docker-compose/ portainer stack to limit the database to 5% of a core.
Similarly I can't now downgrade to 10.4.17.
Is upgrading this container to 10.5.9 a realistic ask? Is it a huge job or a relatively simple one?

FYI also RPi4 running
Linux hostname 5.10.25-v8+ #1408 SMP PREEMPT Mon Mar 22 12:52:01 GMT 2021 aarch64 GNU/Linux

@leolivier
Copy link

As I didn't see any progress on that, I decided, with a heavy heart, to switch to linuxserver.io image which is working and can be used with a very little change in the compose file. 🥺

@dominictayloruk
Copy link
Member

Image updated to MariaDB 10.5.9, can confirm armhf cpu bug is fixed.

@dominictayloruk
Copy link
Member

Asus Tinkerboard armhf

4 instances running on a k8s cluster (6 in total, 4 on same host)

Screenshot 2021-05-06 at 21 50 25

HTOP of host, cpu freq at minimum 600mhz, no cores pinned to 100%

Screenshot 2021-05-06 at 21 51 14

@dominictayloruk
Copy link
Member

Same issue for me - I've ended up implementing cpu_quota: 5000 in my docker-compose/ portainer stack to limit the database to 5% of a core.
Similarly I can't now downgrade to 10.4.17.
Is upgrading this container to 10.5.9 a realistic ask? Is it a huge job or a relatively simple one?

FYI also RPi4 running
Linux hostname 5.10.25-v8+ #1408 SMP PREEMPT Mon Mar 22 12:52:01 GMT 2021 aarch64 GNU/Linux

try running yobasystems/alpine-mariadb:aarch64, this bug only affects arm/armhf/arm32

@dominictayloruk
Copy link
Member

How about adding and maintaining some new tags

yobasystems/alpine-mariadb:10.3
yobasystems/alpine-mariadb:10.4
yobasystems/alpine-mariadb:10.5

This would allow us to use the latest 10.3 / 10.4 / 10.5 without (accidently) switching to a new major version, e.g.
10.4 would automatically switch from 10.4.15 --> 10.4.16 --> 10.4.17 but NOT to 10.5.x

Switching major versions typically also requires additional maintenance:
https://mariadb.com/kb/en/mysql_upgrade/
I'm not sure, whether the mysql_upgrade is done when starting the container. Therefore I strongly recommend never to use tag "latest" but always use tags 10.3 / 10.4 / 10.5.

This would allow us to avoid 10.5 as long as it has problems, but still stay on the latest and greatest 10.4 ;-)

https://hub.docker.com/_/mariadb has the same tag convention (10.3 / 10.4 / 10.5)

This should be done now, just testing the build for 10.5 & 10, i'll then do the stable 10.4 ;)

@Slartibartfast27
Copy link

Perfect ;-) Thank you very much!

@marcioferreirap
Copy link

I also have this problem on an 32 bit armv7 architecture:

root# uname -a
Linux odroid2020 5.4.98-odroidxu4 #21.02.2 SMP PREEMPT Sun Feb 14 22:29:57 CET 2021 armv7l armv7l armv7l GNU/Linux

10.4.17 works fine. Only 10.5.8 has this problem.

Steps to reproduce with 10.5.8 (this burns the CPU for about 5 minutes on an https://www.hardkernel.com/shop/odroid-xu4q-special-price/ and then stops the container):

root# docker run --rm yobasystems/alpine-mariadb:10.5.8
[i] mysqld not found, creating....
[i] MySQL data directory not found, creating initial DBs
340444-06-16 16:13:00 0 [ERROR] This MySQL server doesn't support dates later than 2038
[i] MySQL root Password: nohl2Eex2SohZuth
4282-12-18 17:49:17 0 [ERROR] This MySQL server doesn't support dates later than 2038
/scripts/run.sh: ignoring or entrypoint initdb empty /docker-entrypoint-initdb.d/*


MySQL init process done. Ready for start up.

exec /usr/bin/mysqld --user=mysql --console --skip-name-resolve --skip-networking=0
442234-11-13 17:14:34 0 [ERROR] This MySQL server doesn't support dates later than 2038
root#

Just for comparison: 10.4.17 working fine:

root# docker run --rm yobasystems/alpine-mariadb:10.4.17
[i] mysqld not found, creating....
[i] MySQL data directory not found, creating initial DBs
...
MySQL init process done. Ready for start up.
...
2021-04-18  5:50:22 0 [Note] /usr/bin/mysqld: ready for connections.
Version: '10.4.17-MariaDB'  socket: '/run/mysqld/mysqld.sock'  port: 3306  MariaDB Server

Thank you so much. Works fine for me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants