New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SecComp fails on CentOS 6 #22899

Closed
tomsommer opened this Issue Feb 1, 2017 · 12 comments

Comments

Projects
None yet
7 participants
@tomsommer

tomsommer commented Feb 1, 2017

CentoOS 6 does not (as far as I can research) support SecComp, but ES tries to use it anyway, and fails to start due to bootstrap.system_call_filter: true.

The only workaround currently is to disable bootstrap.system_call_filter

ES 5.2.0

[2017-02-01T10:46:14,173][INFO ][o.e.b.BootstrapChecks    ] [elasticclient] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-02-01T10:46:14,175][ERROR][o.e.b.Bootstrap          ] [elasticclient] node validation exception
bootstrap checks failed
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
[2017-02-01T10:46:10,639][WARN ][o.e.b.JNANatives         ] unable to install syscall filter:
java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed
        at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:363) ~[elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:638) ~[elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:215) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:99) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:110) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:203) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) [elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) [elasticsearch-5.2.0.jar:5.2.0]

@jasontedor

This comment has been minimized.

Show comment
Hide comment
@jasontedor

jasontedor Feb 1, 2017

Member

This is operating as designed, and is documented. Your kernel does not support seccomp. Elasticsearch attempts to utilize seccomp by default (via the setting bootstrap.system_call_filter). Starting in 5.2.0, if you're in production mode, bootstrap.system_call_filter is enabled, and initializing seccomp fails, then Elasticsearch will refuse to bootstrap. This to prevent scenarios when seccomp is silently not initialized yet configured to be so. You either have to migrate to a kernel that supports seccomp, or disable bootstrap.system_call_filter.

Member

jasontedor commented Feb 1, 2017

This is operating as designed, and is documented. Your kernel does not support seccomp. Elasticsearch attempts to utilize seccomp by default (via the setting bootstrap.system_call_filter). Starting in 5.2.0, if you're in production mode, bootstrap.system_call_filter is enabled, and initializing seccomp fails, then Elasticsearch will refuse to bootstrap. This to prevent scenarios when seccomp is silently not initialized yet configured to be so. You either have to migrate to a kernel that supports seccomp, or disable bootstrap.system_call_filter.

@jasontedor jasontedor closed this Feb 1, 2017

@tomsommer

This comment has been minimized.

Show comment
Hide comment
@tomsommer

tomsommer Feb 1, 2017

Fair enough, https://www.elastic.co/support/matrix might need an update then.

tomsommer commented Feb 1, 2017

Fair enough, https://www.elastic.co/support/matrix might need an update then.

@jasontedor

This comment has been minimized.

Show comment
Hide comment
@jasontedor

jasontedor Feb 1, 2017

Member

No, it doesn't. You can still run Elasticsearch on CentOS 6, and we still support it on CentOS 6.

Member

jasontedor commented Feb 1, 2017

No, it doesn't. You can still run Elasticsearch on CentOS 6, and we still support it on CentOS 6.

@nik9000

This comment has been minimized.

Show comment
Hide comment
@nik9000

nik9000 Feb 2, 2017

Contributor

Maybe we should suppress the stack trace here, leaving the warning. Maybe even explaining that elasticsearch will still run just fine but doesn't have the extra fork protection.

Contributor

nik9000 commented Feb 2, 2017

Maybe we should suppress the stack trace here, leaving the warning. Maybe even explaining that elasticsearch will still run just fine but doesn't have the extra fork protection.

@jasontedor

This comment has been minimized.

Show comment
Hide comment
@jasontedor

jasontedor Feb 2, 2017

Member

I don't think we should suppress the stack trace (there's a few different ways this can fail). We say this in the tail of the logs:

[2017-02-02T03:22:24,059][INFO ][o.e.b.BootstrapChecks    ] [2oC8mTv] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: bootstrap checks failed
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
[2017-02-02T03:22:24,078][INFO ][o.e.n.Node               ] [2oC8mTv] stopping ...
[2017-02-02T03:22:24,109][INFO ][o.e.n.Node               ] [2oC8mTv] stopped
[2017-02-02T03:22:24,109][INFO ][o.e.n.Node               ] [2oC8mTv] closing ...
[2017-02-02T03:22:24,126][INFO ][o.e.n.Node               ] [2oC8mTv] closed

And the bootstrap check docs say this:

To pass the system call filter check you must either fix any configuration errors on your system that prevented system call filters from installing (check your logs), or at your own risk disable system call filters by setting bootstrap.system_call_filter to false.

Member

jasontedor commented Feb 2, 2017

I don't think we should suppress the stack trace (there's a few different ways this can fail). We say this in the tail of the logs:

[2017-02-02T03:22:24,059][INFO ][o.e.b.BootstrapChecks    ] [2oC8mTv] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: bootstrap checks failed
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
[2017-02-02T03:22:24,078][INFO ][o.e.n.Node               ] [2oC8mTv] stopping ...
[2017-02-02T03:22:24,109][INFO ][o.e.n.Node               ] [2oC8mTv] stopped
[2017-02-02T03:22:24,109][INFO ][o.e.n.Node               ] [2oC8mTv] closing ...
[2017-02-02T03:22:24,126][INFO ][o.e.n.Node               ] [2oC8mTv] closed

And the bootstrap check docs say this:

To pass the system call filter check you must either fix any configuration errors on your system that prevented system call filters from installing (check your logs), or at your own risk disable system call filters by setting bootstrap.system_call_filter to false.

@tomsommer

This comment has been minimized.

Show comment
Hide comment
@tomsommer

tomsommer Feb 2, 2017

Well, 'at your own risk' makes you go looking for a way to make system call filters work on CentOS 6, which it can't.

You will have all CentOS 6 users setting bootstrap.system_call_filter to false now, and forgetting to unset it when they upgrade to CentOS 7+.

Just my two cents :)

tomsommer commented Feb 2, 2017

Well, 'at your own risk' makes you go looking for a way to make system call filters work on CentOS 6, which it can't.

You will have all CentOS 6 users setting bootstrap.system_call_filter to false now, and forgetting to unset it when they upgrade to CentOS 7+.

Just my two cents :)

@jasontedor

This comment has been minimized.

Show comment
Hide comment
@jasontedor

jasontedor Feb 2, 2017

Member

Okay, but at least they are aware of the problem whereas they weren't before since we silently failed.

Member

jasontedor commented Feb 2, 2017

Okay, but at least they are aware of the problem whereas they weren't before since we silently failed.

@dhirenshumsher

This comment has been minimized.

Show comment
Hide comment
@dhirenshumsher

dhirenshumsher Feb 13, 2017

Hi jasontedor, What is a safer way to disable bootstrap.system_call_filter?

dhirenshumsher commented Feb 13, 2017

Hi jasontedor, What is a safer way to disable bootstrap.system_call_filter?

@datayjz

This comment has been minimized.

Show comment
Hide comment
@datayjz

datayjz Feb 13, 2017

in elasticsearch.yml memory after config

datayjz commented Feb 13, 2017

in elasticsearch.yml memory after config

@jasontedor

This comment has been minimized.

Show comment
Hide comment
@jasontedor

jasontedor Feb 13, 2017

Member

Hi jasontedor, What is a safer way to disable bootstrap.system_call_filter?

Just configure it like you would any other configuration option (e.g., in the elasticsearch.yml).

Member

jasontedor commented Feb 13, 2017

Hi jasontedor, What is a safer way to disable bootstrap.system_call_filter?

Just configure it like you would any other configuration option (e.g., in the elasticsearch.yml).

@Exlsunshine

This comment has been minimized.

Show comment
Hide comment
@Exlsunshine

Exlsunshine Sep 9, 2017

same reason brings me here, now that it's a warn... I decide to ignore it lol

Exlsunshine commented Sep 9, 2017

same reason brings me here, now that it's a warn... I decide to ignore it lol

@chitraj8

This comment has been minimized.

Show comment
Hide comment
@chitraj8

chitraj8 Feb 5, 2018

For me it doesnt get the ES started :(

chitraj8 commented Feb 5, 2018

For me it doesnt get the ES started :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment