Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

database/sql: slower performance of sql driver with goroutines #24337

Closed
ankit2806 opened this issue Mar 10, 2018 · 4 comments
Closed

database/sql: slower performance of sql driver with goroutines #24337

ankit2806 opened this issue Mar 10, 2018 · 4 comments

Comments

@ankit2806
Copy link

@ankit2806 ankit2806 commented Mar 10, 2018

What version of Go are you using (go version)?

1.9

Does this issue reproduce with the latest release?

NA

What operating system and processor architecture are you using (go env)?

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 13
model name : QEMU Virtual CPU version (cpu64-rhel6)
stepping : 3
microcode : 0x1
cpu MHz : 2099.998
cache size : 4096 KB
physical id : 1
siblings : 1
core id : 0
cpu cores : 1
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm nopl pni cx16 hypervisor lahf_lm abm
bogomips : 4199.99
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
power management:

OS: Cent OS 6

What did you do?

func ExecuteSqlQuery(sqlConn *sql.DB, strQuery string, params []interface{}) (map[int]map[string]string, error) {

rows, err := sqlConn.Query(strQuery, params...)
if err != nil {
	//fmt.Println("Query execution failed.")
}
defer rows.Close()

cols, err := rows.Columns()
if err != nil {
	//fmt.Println("Failed to get columns", err)
	return nil, err
}

rawResult := make([][]byte, len(cols))
result := make(map[int]map[string]string)

dest := make([]interface{}, len(cols)) // A temporary interface{} slice
for i, _ := range rawResult {
	dest[i] = &rawResult[i] // Put pointers to each string in the interface slice
}

j := 0
for rows.Next() {
	err = rows.Scan(dest...)
	if err != nil {
		//fmt.Println("Failed to scan row", err)
		return nil, err
	}

	tmp := make(map[string]string)
	for i, raw := range rawResult {
		tmp[cols[i]] = string(raw)
	}

	result[j] = tmp
	j++
}

return result, err

}

What is the problem ?

I am executing 6 queries. The individual query is taking around 50-60 ms but when they are executed via goroutines the execution time is around 1 sec.

I expected these to be run under 60 ms too. Do I need to make any changes in the code ?

@rasky rasky changed the title Slower performance of sql driver with goroutines database/sql: slower performance of sql driver with goroutines Mar 10, 2018
@rasky
Copy link
Member

@rasky rasky commented Mar 10, 2018

Hello, please provide a complete source code which we can use to reproduce the issue. We need something that we can download, compile and run to see the issue.

@mvdan
Copy link
Member

@mvdan mvdan commented Mar 10, 2018

Also, please try your best to provide a minimal piece of code. 20 lines of self-contained Go code to reproduce the isssue are orders of magnitude better than 200 lines.

@ankit2806
Copy link
Author

@ankit2806 ankit2806 commented Mar 11, 2018

Hi, I set the vlue of DB.SetMaxIdleConns to 1000 and it resolved the issue. Could you please tell me the default value of SetMaxIdleConns set. This information is not mentioned in the document.

@kardianos kardianos closed this Mar 16, 2018
@golang golang locked and limited conversation to collaborators Mar 16, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.