Could not initialize seastar when starting scylla 5.2.9

[qiuqiu-007]

It occured when I do gemini test at the beginning of starting scylla image with the version of 5.2.9. Msg is Could not initialize seastar: std::system_error (error system:1, Operation not permitted), all detatiled msg is attached in file

[qiuqiu-007]

It occured when I do gemini test at the beginning of starting scylla image with the version of 5.2.9. Msg is Could not initialize seastar: std::system_error (error system:1, Operation not permitted), all detatiled msg is attached in file

log.txt

[mykaul]

Can you provide more information on how you are trying to run it? OS, kernel, container/VM, versions?

[qiuqiu-007]

Can you provide more information on how you are trying to run it? OS, kernel, container/VM, versions?

• I was trying to run gemini test, and i just run ./script/test.sh scylla
• Env: CentOS 7
• gemini : https://github.com/scylladb/gemini/tree/master/scripts
• I just modified the version of scylla image from 5.2.2 to 5.2.9
• `[root@centos02 gemini]# cat scripts/docker-compose-scylla.yml
  version: '3.5'

networks:
gemini:
name: gemini
driver: bridge
ipam:
driver: default
config:
- subnet: 192.168.100.0/24

services:
gemini-oracle:
image: scylladb/scylla:5.2.9
container_name: gemini-oracle
command: --smp 2 --memory 512M --api-address 0.0.0.0
networks:
gemini:

gemini-test:
image: scylladb/scylla:5.2.9
container_name: gemini-test
command: --smp 2 --memory 512M --api-address 0.0.0.0
networks:
gemini:
`

[mykaul]

I think we can move the discussion to Gemini repo, no?
(I can move the issue, I believe)

[mykaul]

• Env: CentOS 7

This is a bit of an old environment and kernel (and Docker...)

[qiuqiu-007]

• Env: CentOS 7

This is a bit of an old environment and kernel (and Docker...)

Is it the potential cause of this issue ?

[qiuqiu-007]

I think we can move the discussion to Gemini repo, no? (I can move the issue, I believe)

I agree.

[fruch]

@qiuqiu-007

2023-12-13T01:19:33.687533571Z WARN  2023-12-13 01:19:33,684 seastar - Requested AIO slots too large, please increase request capacity in /proc/sys/fs/aio-max-nr. available:65536 requested:102052
2023-12-13T01:19:33.687550162Z WARN  2023-12-13 01:19:33,684 seastar - max-networking-io-control-blocks adjusted from 50000 to 31742, since AIO slots are unavailable
2023-12-13T01:19:33.687554111Z INFO  2023-12-13 01:19:33,684 seastar - Reactor backend: epoll

this should fix it:

echo "fs.aio-max-nr = 1048576" >> /etc/sysctl.conf

[qiuqiu-007]

@fruch I tried, but it doesn't work.
log2.txt

[fruch]

@fruch I tried, but it doesn't work. log2.txt

forgot the actual update, that would get it updated:

sysctl -p 

[qiuqiu-007]

@fruch I tried, but it doesn't work. log2.txt

forgot the actual update, that would get it updated:

sysctl -p 

The aio fixed. But others.
log3.txt

[mykaul]

I guess the issue is:

2023-12-14T01:15:35.620341245Z [0.013s][warning][os,thread] Failed to start thread "VM Thread" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
2023-12-14T01:15:35.620368712Z Error occurred during initialization of VM

I wonder if it happens with newer kernels.