Do not run sysctl for vm.max_map_count when its already set

[majormoses]

Context: This fails in a docker container otherwise. The change was inspired by issues getting it working and after researching I found a similar change applied to another project: spujadas/elk-docker#106 the major difference here is that we include equality in the condition rather than only if its greater than.

This was the error I encountered:

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    service[elasticsearch] (/opt/kitchen/cache/cookbooks/elasticsearch/libraries/provider_service.rb line 132) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
    ---- Begin output of /etc/init.d/elasticsearch start ----
    STDOUT: * Starting Elasticsearch Server
       ...fail!
    STDERR: /etc/init.d/elasticsearch: line 122: ulimit: max locked memory: cannot modify limit: Operation not permitted
    sysctl: setting key "vm.max_map_count": Read-only file system
    ---- End output of /etc/init.d/elasticsearch start ----
    Ran /etc/init.d/elasticsearch start returned 1

After applying the patch I was able to see that error go away (other errors still but one at a time):

root@dokken:/# service elasticsearch start
 * Starting Elasticsearch Server                                            /etc/init.d/elasticsearch: line 122: ulimit: max locked memory: cannot modify limit: Operation not permitted
                                                                     [fail]

Signed-off-by: Ben Abrams me@benabrams.it

[jasontedor]

I don't think this is needed. Rather, I think that you can mask systemd-sysctl.service (systemctl mask systemd-sysctl.service; install Elasticsearch; systemctl unmask systemd-sysctl.service). Would you try?

[elasticmachine]

Pinging @elastic/es-core-infra

[majormoses]

@jasontedor This is for a sysv-init not a systemd setup and as such I don't have systemctl in the container:

root@dokken:/# which systemctl
root@dokken:/# 

[jasontedor]

Oh, I missed that; sorry.

Would you add a packaging test for this please?

See for example:

elasticsearch/qa/vagrant/src/test/resources/packaging/tests/60_systemd.bats

Lines 214 to 221 in 0bfd18c

	@test "[SYSTEMD] masking systemd-sysctl" {
	clean_before_test
	systemctl mask systemd-sysctl.service
	install_package
	systemctl unmask systemd-sysctl.service
	}

and check the contributing docs for how to run these tests. You can also push such a change to the PR and we can run the tests in our CI, but that might be painful to iterate on.

Effectively we would want a test here that shows that if the vm.max_map_count exceeds 262144 then it is still that value after Elasticsearch is installed.

[majormoses]

Ya I can take a stab at it, I see two options regarding testing:

1. set vm.max_map_count to some ridiculously low number and then verify that after starting the service it has been changed. The advantage to this test is that if the default values change in the future the test will keep on working without needing a tweak. As the trend is that these numbers are going up rather than down I think this is more future proof.
2. set MAX_MAP_COUNT in /etc/default/elasticsearch or /etc/sysconfig/elasticsearch depending on the distribution. As this has more complexity and could need updating if we bump the default number again in the future I would prefer to not use this method if the above works for you.

[jasontedor]

Thank you for this PR, and thank you for agreeing to try contributing a test.

I think that a test along the lines of what you describe in option one is good. I do think that there should be a test though for when the current value already exceeds what we set it to since the point of your change is that we do not do anything in that case?

[majormoses]

I do think that there should be a test though for when the current value already exceeds what we set it to since the point of your change is that we do not do anything in that case?

That makes sense, I will try look through it and figure out what's the easiest way to test that depending on distribution. If you happen to know if there is some helper function that would allow me to specify it in an distribution agnostic method that would save me some trouble.

I will work on the first test as its easier and demonstrates this is not a breaking change and will then work on the test to ensure that it does what I expect.

[jasontedor]

That makes sense, I will try look through it and figure out what's the easiest way to test that depending on distribution. If you happen to know if there is some helper function that would allow me to specify it in an distribution agnostic method that would save me some trouble.

It is okay; I think you can call sysctl to set the value of vm.max_map_count before the test starts to some pre-defined value rather than jumping through hoops to know what the defaults are on all of the boxes that we test on.

[majormoses]

@jasontedor any chance you can give me your username in IRC so I can hit you up? If you prefer to hit me up I a share my github and irc username. I am having difficulty getting all the dependencies in place locally. While I could just push and have you run it on your CI I'd rather learn to fish.

[jasontedor]

@majormoses I appreciate your dedication here.

I would be happy to help yet it’s quite late in my timezone. Could we pick this up tomorrow? If you send to me an email at my first name at elastic.co we can connect.

[alpar-t]

elastic bot: run all packaging tests

[majormoses]

I am working on trying to get them running locally but the failure seems to be a timeout rather than a failed spec.

Build timed out (after 400 minutes). Marking the build as failed.

[majormoses]

I got the tests working locally:

./gradlew packagingTest -Pvagrant.boxes=centos-6,ubuntu-1404
# intentionally truncated
BUILD SUCCESSFUL in 54m 48s
396 actionable tasks: 97 executed, 299 up-to-date

I will create a gist and link it here.

[majormoses]

Testing: https://gist.github.com/majormoses/5d3ab8f50f53cdbc10a17943ef4cec87

[majormoses]

elastic bot: run all packaging tests

Not sure if its open to the public but it was worth a try.

[jasontedor]

It is not otherwise that would be a vehicle for remote code execution on our infrastructure.

[majormoses]

It is not otherwise that would be a vehicle for remote code execution on our infrastructure.

makes sense, if it was ephemeral it probably would not matter beyond people using it to do something like spinning up crypto miners but better to be safe than sorry.

[jasontedor]

@elasticmachine test this please

[jasontedor]

@elasticmachine test this please

[jasontedor]

I merged master into your branch and started a new build because the current one was not going to succeed on the BWC aspect of the build given that we bumped versions a few hours ago.

[jasontedor]

This looks good and the tests look good; thanks so much for writing them and getting them working. 💯

I left one comment.

[jasontedor]

I would be okay with us asserting that this is 262144.

[majormoses]

I personally do not think it would be better as if the defaults change this is one more test to update and if its not 100 and the previous command did not error (which I assume would trigger a failed build) it should essentially be the same but if you feel otherwise I can certainly change it.

[jasontedor]

Well, today we want to ensure that the value is set to what we specify: 262144. It's not much overhead to have to change this if we were to make a change to the default here.

[majormoses]

k, will update.

[majormoses]

updated

[jasontedor]

@elasticmachine test this please

[jasontedor]

@elasticmachine test this please

[jasontedor]

@elasticmachine run all packaging tests

[jasontedor]

LGTM.

[jasontedor]

Thanks @majormoses.