-
Notifications
You must be signed in to change notification settings - Fork 24.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ES doesn't start when there are empty cgroup controller names in '/proc/self/cgroup' #23486
Comments
Thanks for the report @phile314-fh and sorry for the issue. I'll put together a fix soon. What Linux distribution are you using (including version, and kernel version)? Would you share the output of |
Kernel: Linux nixos 4.9.9 #1-NixOS SMP Thu Feb 9 07:08:40 UTC 2017 x86_64 GNU/Linux
#subsys_name hierarchy num_cgroups enabled cpuset 1 1 1 cpu 2 1 1 cpuacct 2 1 1 blkio 5 1 1 memory 3 1 1 devices 7 38 1 freezer 4 1 1 net_cls 8 1 1 pids 6 43 1
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup2 (rw,nosuid,nodev,noexec,relatime) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) Although you don't officially support NixOS, this ought to be fixed as the problem could also happen on other distributions. Furthermore, as the error (when it occurs) is quite severe, a more liberal parsing of the cgroups seems appropriate to me. |
Okay, I wanted to ensure it was only because the cgroup version 2 hierarchy was mistakenly accounted for and that's exactly what is happening here. I opened #23493. |
Faced this on FC26 as well.... For ES v5.2.2
|
@jay-dihenkar You should upgrade, this is fixed in 5.3.1. |
@jasontedor 5.3.1 are incompatible with my services already in production on AWS, theres some workaround or fix for it? |
@aymone You can disable the cgroup version 2 hierarchy on your system, otherwise you have to upgrade. |
@jasontedor do you know how to do it? |
this is to fix the issue with some linux distros see: @ED-3197 elastic/elasticsearch#23486
I can't update my ES, same as @aymone , because services depend on this version. Any workarounds? How one can disable cgroup hierarchy and what does it mean / what side-effects can it have? I have on Ubuntu 17.10:
|
@aymone @KrzysztofMadejski Please poke around in documentation and the web for that, that is a general Linux issue, not an Elasticsearch issue. |
@jasontedor it would be good to add "wontfix" label here. |
What do you mean? It is fixed in #23493 released in 5.3.1. |
The bug report is against version 5.2.2 so I see it as "won't fix" for branch 5.2.x. Such notion makes sense to me because minor versions may introduce backwards incompatible changes (5.3 does) so an upgrade is not straightforward operation if you have ES in production. The other issue which is troubling me more is why you introduce backwards incompatible changes in minor versions, which is contrary to the declaration at https://www.elastic.co/support/eol. But for clarity let's put it into another issue. |
I understand where you’re coming from, but our maintenance policy is very clear (when 5.3.0 is released, 5.2 sees no more releases) and all the information needed to determine what version this is fixed in is already available.
Please do. |
The ElasticSearch connector tests are failing on some machines, due to an issue with a regex to parse cgroups: elastic/elasticsearch#23486.
The ElasticSearch connector tests are failing on some machines, due to an issue with a regex to parse cgroups: elastic/elasticsearch#23486.
Elasticsearch version: 5.2.2
Plugins installed: []
JVM version: 8u122
OS version: Linux
Description of the problem including expected versus actual behavior:
The OS Probe Regex fails if there is a cgroup entry with no controller and crashes. Example
/proc/self/cgroup
(see last line):Related issue: #23218
Steps to reproduce:
1.
2.
3.
Provide logs (if relevant):
Describe the feature:
Make the regex more robust.Changing the
+
to a*
in the failing regex for the part matching the cgroup controller name should do the trick. I would make a PR, but I am not willing to sign the CLA.The text was updated successfully, but these errors were encountered: