Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fmt: Printf arguments escape to heap #19720

Closed
go101 opened this issue Mar 26, 2017 · 13 comments
Closed

fmt: Printf arguments escape to heap #19720

go101 opened this issue Mar 26, 2017 · 13 comments

Comments

@go101
Copy link

@go101 go101 commented Mar 26, 2017

package main

import "fmt"

func main() {
	var a = []int{}
	fmt.Println(a) // a escapes to heap
}

It looks it is caused by the pp struct in print.go which holds the printed arguments and pp values are referenced by other heap values. This can be avoided.

type pp struct {
	buf buffer

	// arg holds the current item, as an interface{}.
	arg interface{}

	// value is used instead of arg for reflect values.
	value reflect.Value
	
	...
}
@mvdan
Copy link
Member

@mvdan mvdan commented Mar 26, 2017

I'm not familiar with the compiler and the fmt package enough to say whether this should or could be fixed. But my undesrtanding is that in general you should not use fmt in hot paths where an allocation or interface{} work is not acceptable.

@ALTree
Copy link
Member

@ALTree ALTree commented Mar 26, 2017

This is #8618 I believe.

@go101
Copy link
Author

@go101 go101 commented Mar 26, 2017

This is not related to interface. Following code doesn't make the input arguments escape.

package main

func f(a interface{}){
	switch a.(type) {
	case int:
		println("int")
	case []int:
		println("[]int")
	case *int:
		println("*int")
	case *[]int:
		println("*[]int")
	}
}

func main() {
	var x = []int{0, 1, 2}
	
	f(x)
	f(x[0])
	f(&x)
	f(&x[0])
}
@go101
Copy link
Author

@go101 go101 commented Mar 26, 2017

@mvdan, I use log package quite often. The log package warps the fmt functions. So I hope the fix is good to optimize log functions.

But if there are many different strings needed to be passed to the fmt.Print functions in a goroutine, maybe it is not bad to allocate them on heap, to avoid stack growing and shinking.

@mvdan
Copy link
Member

@mvdan mvdan commented Mar 26, 2017

I'm just surprised to see that the performance of log is cause of concern. Or is this just out of curiosity?

@bradfitz bradfitz changed the title fmt.Print functions will make argument escape to heap, not reasonable. fmt: Printf arguments escape to heap Mar 26, 2017
@bradfitz bradfitz added this to the Unplanned milestone Mar 26, 2017
@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Mar 26, 2017

I'll keep this open as a long-term meta bug about escape analysis, but small examples are generally more actionable.

@randall77
Copy link
Contributor

@randall77 randall77 commented Mar 26, 2017

Analyzing fmt routines to know that the arguments don't escape is hard. The callgraph is

 Println
 Fprintln
 doPrintln
 printArg
 handleMethods

handleMethods will call the Format, GoString, String, or Error methods on the passed-in type. Escape analysis doesn't know what those functions will do, so their arguments escape. That escape propagates all the way up to Println.

package main

import "fmt"

type A []int

var sink A

func (a A) String() string {
	sink = a
	return "foo"
}
func main() {
	a := []int{}
	fmt.Println(A(a))
}

a escapes!

The only way I see to fix this is to have type-dependent escape analysis. I don't see that happening any time soon.

@rasky
Copy link
Member

@rasky rasky commented Mar 26, 2017

fmt.Sprint() was being cited as the "final litmus test" for escape analysis by @dvyukov in his document:
https://docs.google.com/document/d/1CxgUBPlx9iJzkz9JWkb6tIpTe5q32QDmz8l0BouG0Cw/preview

@go101
Copy link
Author

@go101 go101 commented Mar 27, 2017

@mvdan

I'm just surprised to see that the performance of log is cause of concern.

yes, sometimes, it is.

Another bad side effect is these functions change what it measure when it participate.
For example, I want to check how much memory is allocated on stacks and heap, but the results of the fmt functions affect the query results.

@go101
Copy link
Author

@go101 go101 commented Mar 27, 2017

@mvdan

I'm just surprised to see that the performance of log is cause of concern.

yes, sometimes, it is.

Another bad side effect is these functions change what they measure when they participate.
For example, I want to check how much memory is allocated on stacks and heap, but print some values by calling the fmt functions will affects the query results.

@bronze1man
Copy link
Contributor

@bronze1man bronze1man commented Mar 27, 2017

Is it easier to write something like //go:escapesArgToHeap , then put it before the function body?
When you know that something can be escapes heap, I think you can just tell the compiler to do that with import "unsafe" package.

@robpike
Copy link
Contributor

@robpike robpike commented Mar 27, 2017

I am pretty sure this is a dup of #8618. The counterexample listed above is all in one package, which is a much easier problem. #8618 requires escape analysis across a package boundary.

@randall77
Copy link
Contributor

@randall77 randall77 commented Mar 28, 2017

Yes, I confirm duplicate of #8618.

@randall77 randall77 closed this Mar 28, 2017
@golang golang locked and limited conversation to collaborators Mar 28, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants
You can’t perform that action at this time.