Skip to content
This repository was archived by the owner on Mar 24, 2026. It is now read-only.

start distinct Go process per 2 (virtual) cores.#1472

Merged
LadyMozzarella merged 3 commits intoTechEmpower:masterfrom
funny-falcon:master
Apr 4, 2015
Merged

start distinct Go process per 2 (virtual) cores.#1472
LadyMozzarella merged 3 commits intoTechEmpower:masterfrom
funny-falcon:master

Conversation

@funny-falcon
Copy link
Copy Markdown
Contributor

It is very similar to usage nodejs cluster package or Ruby Unicorn server, so i think it is legitimate.

It is simple version, just to benchmark, but it could be easily extended to match Ruby Unicorn or nodejs cluster functionality.

I think it is able to greatly improve Go performance on Peak and EC2. Even i7 benchmark can benefit from this change.

It is very similar to usage nodejs cluster package or Ruby Unicorn server,
so that i think it is legitimate.

It is simple version, just to benchmark, but it could be easily
extended to match Ruby Unicorn or nodejs cluster functionality.
@methane
Copy link
Copy Markdown
Contributor

methane commented Apr 2, 2015

While I'm interested in effect of this change, I think preforking is not a usual way in Go's world.
I want to keep our code in good shape: new Gophers can use our code as sample code for writing production ready server.

I want to see future version of Go resolve scaling issue.
Go 1.4 has lock contention problem in database/sql. It is found in this benchmark and I've fixed it.
I'll try to investigate this problem before Go 1.6 code freeze, too.

@funny-falcon
Copy link
Copy Markdown
Contributor Author

But "prefork" helps even with "plain text" and "json" benchmarks. So it is not tied to database/sql.

If i made it as a public library, will you use it for benchmark? Perhaps, as a separate "framework".

Or as external tool like https://github.com/stripe/einhorn with library as adapter for?

@methane
Copy link
Copy Markdown
Contributor

methane commented Apr 3, 2015

First of all, this PR may not be merged Round 10 because Round 10 is in final phase for now.
(But I don't know, because I'm not a member of TechEmpower).
So we have enough time to chose better way.

General requirements says:

All test implementations should be production-grade. The particulars of this will vary by framework and platform, but the general sentiment is that the code and configuration should be suitable for a production deployment.

So I want our code and configuration are good enough for reference.
Using library or not is not a problem. The problem is: Is prefork a recommended / commonly used technique for high load production environment?

If the answer is yes, I'm OK to merge this.
If not, I think preforking benchmark should be separated from normal "go" benchmark. "go-prefork", maybe.

Split benchmark doesn't require split the code. Adding -prefork flag is enough, maybe.
I don't like einhorn since it depends Ruby. It makes hard to run test for new contributers.

@msmith-techempower
Copy link
Copy Markdown
Member

I agree with @methane

default mode is classic one-process.
preforking for special benchmarking mode go-prefork
@funny-falcon
Copy link
Copy Markdown
Contributor Author

I've modified code, so by default it runs in one process.

If flag -prefork passed then prefork performed. It will be great, if it is added as go-prefork benchmark.

@hamiltont
Copy link
Copy Markdown
Contributor

Split benchmark doesn't require split the code. Adding -prefork flag is enough, maybe.

If this is good with both of you, it's good with our team. You'll need to add the new configuration to the benchmark_config, and create a new setup file like setup_prefork to do exactly what setup.sh does but also pass the -prefork flag

@funny-falcon
Copy link
Copy Markdown
Contributor Author

Ok, i've edit benchmark_config and add setup_prefork.sh . Hope i've done it right.

@hamiltont
Copy link
Copy Markdown
Contributor

@funny-falcon Yup, that looks correct to me! We will now see what travis says

@funny-falcon
Copy link
Copy Markdown
Contributor Author

looks like travis doesn't mind.

LadyMozzarella added a commit that referenced this pull request Apr 4, 2015
start distinct Go process per 2 (virtual) cores.
@LadyMozzarella LadyMozzarella merged commit 4667f10 into TechEmpower:master Apr 4, 2015
@funny-falcon
Copy link
Copy Markdown
Contributor Author

:( why results could not be viewed?

@methane
Copy link
Copy Markdown
Contributor

methane commented Apr 21, 2015

I suppose it because it's too late for Round 10.
Please wait for round 11.

FYI, Go 1.5 improved schedular:
golang/go@f47e581

So performance of Go without prefork will be improved slightly.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants