Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hotfix(globalpatches) use OpenSSL to seed PRNG #1754

Merged
merged 1 commit into from
Oct 28, 2016

Conversation

thibaultcha
Copy link
Member

@thibaultcha thibaultcha commented Oct 19, 2016

Note: this is a hotfix to be manually merged in both master and next

Summary

This changes the seeding technique for LuaJIT's PRNG from using a
combination of time (s precision) + worker PID to using OpenSSL's
RAND_bytes().

Reasoning: in modern deployment setups such as application containers
and AWS AMIs (etc...), it is a common practise to deploy from a forked
VM, resulting in high probability of collision for PIDs at a seconds
precision. This could result in duplicated PRNG seeds, which is
ultimately the PRNG used to generate UUIDs in Kong, by the use of
lua-resty-jit-uuid.

Solution: in order to have a higher entropy when seeding LuaJIT's PRNG,
a proposed fix was to use /dev/urandom. This implementation however
uses OpenSSL's RAND_bytes(), which has the advantage of returning an
error if the entropy is estimated to be too low. However, this won't
cover use cases where the VM has been forked, resulting in multiple VM
clones with a high entropy, but equal to that of the other clones. We
suggest that such deployment environment increase their cloned VMs
entropy before starting Kong.

Full changelog

  • use OpenSSL's RAND_bytes() to read random bytes
  • truncate the final seed to 12 digits to prevent integer overflows
  • update fallback seeding technique (time + worker PID) to use ms
    precision, just in case
  • introduce a new kong lua shared dict. This dictionary's purpose is
    to hold essential data through Kong's lifecycle, and should eventually
    only be used through safe_set() (an abstraction for this could be
    envisaged later on, but is not the purpose of this patch)
  • chosen seeds for each worker are stored in the kong shm, and can be
    consulted via the / endpoint. There is currently no way to re-seed all
    the workers at once unless by sending SIGHUP, because only 1 worker
    would be receiving such a request through the Kong Admin API.
  • update debug.traceback() calls to use lvl 2 of the call stack, to
    show the actual caller of our patched math.randomseed()
  • update log messages to be more explicit

Issues resolved

Fix #1751 #1739 #1623

This changes the seeding technique for LuaJIT's PRNG from using a
combination of `time (s precision) + worker PID` to using OpenSSL's
`RAND_bytes()`.

Reasoning: in modern deployment setups such as application containers
and AWS AMIs (etc...), it is a common practise to deploy from a forked
VM, resulting in high chances of collision for PIDs at a seconds
precision. This could result in duplicated PRNG seeds, which is
ultimately the PRNG used to generate UUIDs in Kong, by the use of
[lua-resty-jit-uuid](https://github.com/thibaultcha/lua-resty-jit-uuid).

Solution: in order to have a higher entropy when seeding LuaJIT's PRNG,
a proposed fix was to use `/dev/urandom`. This implementation however
uses OpenSSL's `RAND_bytes()`, which has the advantage of returning an
error if the entropy is estimated to be too low. However, this won't
cover use cases where the VM has been forked, resulting in multiple VM
clones with a high entropy, but equal to that of the other clones. We
suggest that such deployment environment increase their cloned VMs
entropy before starting Kong.

Full changelog:
* use OpenSSL's `RAND_bytes()` to read random bytes
* truncate the final seed to 12 digits to prevent integer overflows
* update fallback seeding technique (time + worker PID) to use ms
precision, just in case
* introduce a new `kong` lua shared dict. This dictionary's purpose is
to hold essential data through Kong's lifecycle, and should eventually
only be used through `safe_set()` (an abstraction for this could be
envisaged later on, but is not the purpose of this patch)
* chosen seeds for each worker are stored in the kong shm, and can be
consulted via the `/` endpoint. There is currently no way to re-seed all
the workers at once unless by sending `SIGHUP`, because only 1 worker
would be receiving such a request through the Kong Admin API.
* update `debug.traceback()` calls to use lvl 2 of the call stack, to
show the actual caller of our patched `math.randomseed()`
* update log messages to be more explicit

Fix #1751 #1739 #1623
@thibaultcha thibaultcha merged commit 06de2da into master Oct 28, 2016
@thibaultcha
Copy link
Member Author

Manually merged to both next and master.

@thibaultcha thibaultcha deleted the hotfix/urandom-seed branch October 28, 2016 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kong nodes can't find each other and join cluster
1 participant