Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to join cluster #1592

Closed
seedotlee opened this issue Sep 4, 2016 · 9 comments
Closed

How to join cluster #1592

seedotlee opened this issue Sep 4, 2016 · 9 comments
Labels
task/bug
Milestone
0.9.2

Comments

@seedotlee
Copy link

Summary

I have 2 servers ,call them ser1 and ser2.

I started ser1 with config

pg_host = 1.1.1.1 #ser1's IP

cluster_listen = 1.1.1.1:7946  # ser1's IP
cluster_listen_rpc = 127.0.0.1:7373 
cluster_profile = lan

After, I started ser2 with config

pg_host = 1.1.1.1 #ser1's IP

cluster_listen = 2.2.2.2:7946  # ser2's IP
cluster_listen_rpc = 127.0.0.1:7373 
cluster_profile = lan

Two servers can visited, but I get http://1.1.1.1:8001/cluster just 1 cluster.

Using cluster should I pay attention to ？

Additional Details & Logs

  • Kong version (0.9.0)
  • Operating System ubuntu 14.04
@seedotlee
Copy link
Author

I used serf to join 2 servers all going well

@seedotlee
Copy link
Author

I'm solving the problem.

Becuase ser2 is ser1's copy , so the serf.id is the same with ser1.

The serf.id is not change with kong's start that not good enough.

@subnetmarco
Copy link
Member

@seedotlee what do you mean by "serf.id" ?

@seedotlee
Copy link
Author

seedotlee commented Sep 9, 2016 •
edited
Loading

/usr/local/kong/serf/serf.id

@thefosk

@subnetmarco
Copy link
Member

@seedotlee you would need to specify different prefix for each Kong instance. Ie:

prefix=/tmp/kong1
pg_host = 1.1.1.1 #ser1's IP

cluster_listen = 1.1.1.1:7946  # ser1's IP
cluster_listen_rpc = 127.0.0.1:7373 
cluster_profile = lan

and

prefix=/tmp/kong2
pg_host = 1.1.1.1 #ser1's IP

cluster_listen = 1.1.1.1:7946  # ser1's IP
cluster_listen_rpc = 127.0.0.1:7373 
cluster_profile = lan

@seedotlee
Copy link
Author

@thefosk
Uh, I started in 2 machines, there is need different prefix?

@thibaultcha
Copy link
Member

thibaultcha commented Sep 14, 2016 •
edited
Loading

It could be that you have the same hostname for both machines, and that the random part of this line: https://github.com/Mashape/kong/blob/master/kong/cmd/utils/prefix_handler.lua#L218 is actually not so random since from what I can tell, the random number generator is not seeded when invoked from the CLI, hence both nodes generate the same uuid.

If you could add some debug statement right there and confirm with us that they are both generating the same uuid (and that you have the same hostname on both nodes), then I would qualify this as a bug. The fix would be to - properly - seed the random number generation.

This is definitely tricky with Lua but once done right, the random numbers are of good quality (with the LuaJIT implementation, especially).

@thibaultcha thibaultcha modified the milestone: 0.9.2 Sep 14, 2016
@subnetmarco
Copy link
Member

@thibaultcha this could be a problem. Do we need a specific patch for this in 0.9.2?

We should also check everywhere where utils.random_string() is been used, because every caller expects the returning value to be a universally unique string.

thibaultcha added a commit that referenced this issue Sep 15, 2016
@thibaultcha
hotfix(cli) seed random number generator in CLI
7b26793 
If spawning multiple nodes at once (making use of the CLI from different
host machines), we need to make sure none of them are using the same
seed. To enforce this, we make use of the patched 'math.randomseed()'
function, which should greatly reduce the probability of seed collision.

To allow for this change, we need a special flag indicating our scripts
if we are running inside of our CLI, so that out 'math.randomseed()'
  does not complain about being called in resty-cli's 'timer' context.

* add `ngx.RESTY_CLI` flag in `bin/kong`
* add an edge case in our patched `math.randomseed()`
* apply `kong.core.globalpatches` to our CLI environment

Fix #1592
@thibaultcha thibaultcha mentioned this issue Sep 15, 2016
Closed
@thibaultcha
Copy link
Member

Should be fixed with #1641, to be released in 0.9.2, thanks for reporting.

subnetmarco pushed a commit that referenced this issue Oct 28, 2016
@thibaultcha @subnetmarco
hotfix(cli) seed random number generator in CLI
580926a 
If spawning multiple nodes at once (making use of the CLI from different
host machines), we need to make sure none of them are using the same
seed. To enforce this, we make use of the patched 'math.randomseed()'
function, which should greatly reduce the probability of seed collision.

To allow for this change, we need a special flag indicating our scripts
if we are running inside of our CLI, so that out 'math.randomseed()'
  does not complain about being called in resty-cli's 'timer' context.

* add `ngx.RESTY_CLI` flag in `bin/kong`
* add an edge case in our patched `math.randomseed()`
* apply `kong.core.globalpatches` to our CLI environment

Fix #1592
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
task/bug
Projects
None yet
Milestone
0.9.2
Development

Successfully merging a pull request may close this issue.

hotfix(cli) seed random number generator in CLI Kong/kong
3 participants
@subnetmarco @thibaultcha @seedotlee