Skip to content

Memory allocation optimizations #71

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 12 commits into from
Closed

Memory allocation optimizations #71

wants to merge 12 commits into from

Conversation

julienschmidt
Copy link
Member

Based on the great work by @bradfitz in #68

I can see a small performance regression on my Windows system. But Windows is sometimes a bit uh.. strange and Go is more optimized for Linux systems.
Unfortunately I currently have no Linux system where I could properly test this.

So can you please provide benchmark results?
I'd like to see the results of this change on various systems.

@ghost ghost assigned julienschmidt May 9, 2013
@julienschmidt julienschmidt mentioned this pull request May 9, 2013
Closed
@arnehormann
Copy link
Member

LGTM - very, even but I can't test before monday.

@arnehormann
Copy link
Member

I didn't test on Linux, but I ran the benchmarks on OS X - with the regression you mentioned.
But it was faster before. Strange. I'm using Go1.1rc3.

Before (master 5f91140):

*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 1]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 2.137646559s [ 4678 queries/second ]
PreparedQuery: 1.788349429s [ 5592 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 16.856603597s [ 5932 queries/second ]
SimpleQueryRow: 11.322794181s [ 8832 queries/second ]
PreparedQueryRow: 10.718619116s [ 9330 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 4.838119952s [ 20669 queries/second ]
PreparedExec: 4.159680745s [ 24040 queries/second ]


*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 2]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 2.156416112s [ 4637 queries/second ]
PreparedQuery: 1.812604363s [ 5517 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 16.517819146s [ 6054 queries/second ]
SimpleQueryRow: 11.333591641s [ 8823 queries/second ]
PreparedQueryRow: 10.552144453s [ 9477 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 5.116263012s [ 19546 queries/second ]
PreparedExec: 4.884710017s [ 20472 queries/second ]

After (mem-pool a2310d9):

*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 1]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 2.246582916s [ 4451 queries/second ]
PreparedQuery: 1.842219447s [ 5428 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 17.016636702s [ 5877 queries/second ]
SimpleQueryRow: 11.646493473s [ 8586 queries/second ]
PreparedQueryRow: 10.819693275s [ 9242 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 4.924230395s [ 20308 queries/second ]
PreparedExec: 4.511269904s [ 22167 queries/second ]


*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 2]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 2.40373275s [ 4160 queries/second ]
PreparedQuery: 1.973170376s [ 5068 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 17.344655262s [ 5765 queries/second ]
SimpleQueryRow: 11.562155462s [ 8649 queries/second ]
PreparedQueryRow: 10.528550688s [ 9498 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 4.668633706s [ 21420 queries/second ]
PreparedExec: 4.592526832s [ 21775 queries/second ]

@arnehormann
Copy link
Member

For completeness, here's the results from Brads master 1624047:

*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 1]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 2.194595403s [ 4557 queries/second ]
PreparedQuery: 1.836898122s [ 5444 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 17.937962042s [ 5575 queries/second ]
SimpleQueryRow: 11.616752149s [ 8608 queries/second ]
PreparedQueryRow: 10.506572362s [ 9518 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 4.974847119s [ 20101 queries/second ]
PreparedExec: 4.247629189s [ 23543 queries/second ]


*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 2]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 2.257115597s [ 4430 queries/second ]
PreparedQuery: 1.903383193s [ 5254 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 17.390363151s [ 5750 queries/second ]
SimpleQueryRow: 11.529851168s [ 8673 queries/second ]
PreparedQueryRow: 10.491738573s [ 9531 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 4.751487062s [ 21046 queries/second ]
PreparedExec: 4.441372559s [ 22516 queries/second ]

@bradfitz
Copy link

Feel free to revert any parts which make things slower for now. But also
consider the results with a higher -benchtime, like -benchtime=30s, so show
the effects of GC more.

In the future, once we have
https://code.google.com/p/go/issues/detail?id=4720 this CL should be
faster, though. Even if manually managing this memory today is a bit
slower, I'm confident that generating less garbage will be better long-term.

Feel free to do whatever in the meantime.

On Mon, May 13, 2013 at 3:40 AM, Arne Hormann notifications@github.comwrote:

For completeness, here's the results from Brads master 16240471624047
:

BENCHMARKING Go-MySQL-Driver [run 1]

[10000 * Query 100 Rows]

SimpleQuery: 2.194595403s [ 4557 queries/second ]
PreparedQuery: 1.836898122s [ 5444 queries/second ]

[100 * QueryRow] * 1000

AutoQueryRow: 17.937962042s [ 5575 queries/second ]
SimpleQueryRow: 11.616752149s [ 8608 queries/second ]
PreparedQueryRow: 10.506572362s [ 9518 queries/second ]

[100000 * Exec]

SimpleExec: 4.974847119s [ 20101 queries/second ]
PreparedExec: 4.247629189s [ 23543 queries/second ]

BENCHMARKING Go-MySQL-Driver [run 2]

[10000 * Query 100 Rows]

SimpleQuery: 2.257115597s [ 4430 queries/second ]
PreparedQuery: 1.903383193s [ 5254 queries/second ]

[100 * QueryRow] * 1000

AutoQueryRow: 17.390363151s [ 5750 queries/second ]
SimpleQueryRow: 11.529851168s [ 8673 queries/second ]
PreparedQueryRow: 10.491738573s [ 9531 queries/second ]

[100000 * Exec]

SimpleExec: 4.751487062s [ 21046 queries/second ]
PreparedExec: 4.441372559s [ 22516 queries/second ]

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/71#issuecomment-17804306
.

@julienschmidt julienschmidt mentioned this pull request Jun 5, 2013
Closed
@julienschmidt julienschmidt mentioned this pull request Oct 23, 2013
Merged
@julienschmidt
Copy link
Member Author

What is better than one allocation?
Tip: It's not 2 allocations.

I drop this PR for a series of 3 (?) PRs which use another approach.
The first two are #133 and #134

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@julienschmidt julienschmidt

Labels
performance
Projects
None yet
Milestone
v1.1
Development

Successfully merging this pull request may close these issues.
3 participants
@julienschmidt @arnehormann @bradfitz