Memory Issues with ListUsers

[robbertvdg]

Hi,

I saw a high memory consumption in our go application that uses the minio admin sdk. I tracked it down to the ListUsers function. It seems the results are not properly garbage collected with multiple calls.

Here is a code snippet and the memory consumption:

ctx := context.Background()
for i := 1; i <= 10; i++ {
    PrintMemUsage()
    _, err := r.MinioAdminClient.ListUsers(ctx)
    if err != nil {
	return nil, err
    }
}

Memory usages:
Alloc = 99 MiB TotalAlloc = 123 MiB Sys = 139 MiB NumGC = 9
Alloc = 92 MiB TotalAlloc = 189 MiB Sys = 206 MiB NumGC = 10
Alloc = 156 MiB TotalAlloc = 253 MiB Sys = 206 MiB NumGC = 10
Alloc = 156 MiB TotalAlloc = 253 MiB Sys = 206 MiB NumGC = 10
Alloc = 156 MiB TotalAlloc = 253 MiB Sys = 206 MiB NumGC = 10
Alloc = 220 MiB TotalAlloc = 317 MiB Sys = 272 MiB NumGC = 11
Alloc = 220 MiB TotalAlloc = 318 MiB Sys = 272 MiB NumGC = 11
Alloc = 284 MiB TotalAlloc = 382 MiB Sys = 338 MiB NumGC = 11
Alloc = 284 MiB TotalAlloc = 382 MiB Sys = 338 MiB NumGC = 11
Alloc = 284 MiB TotalAlloc = 382 MiB Sys = 338 MiB NumGC = 11

Can this be investigated?

[robbertvdg]

I was able to lower memory consumption by setting a lower value for the GOGC env variable. My knowledge of the Go garbage collection is not sufficient to know whether this is still an issue or intended behaviour.

[klauspost]

@robbertvdg Alloc numbers is typically the total amount allocated, look at Heap In Use.

This looks like intended behavior, but you didn't share the PrintMemUsage(), I cannot know.

[klauspost]

Looking at the function, there is nothing there that can really be an issue.

[robbertvdg]

The function:

func PrintMemUsage() {
	var m run.MemStats
	run.ReadMemStats(&m)
	// For info on each, see: https://golang.org/pkg/runtime/#MemStats
	fmt.Printf("Alloc = %v MiB", bToMb(m.Alloc))
	fmt.Printf("\tTotalAlloc = %v MiB", bToMb(m.TotalAlloc))
	fmt.Printf("\tSys = %v MiB", bToMb(m.Sys))
	fmt.Printf("\tNumGC = %v\n", m.NumGC)
}

[robbertvdg]

Including m.HeapInUse:

heap = 72 MiB Alloc = 69 MiB TotalAlloc = 159 MiB Sys = 140 MiB NumGC = 13
heap = 107 MiB Alloc = 103 MiB TotalAlloc = 230 MiB Sys = 207 MiB NumGC = 14
heap = 171 MiB Alloc = 167 MiB TotalAlloc = 294 MiB Sys = 273 MiB NumGC = 14
heap = 171 MiB Alloc = 167 MiB TotalAlloc = 294 MiB Sys = 273 MiB NumGC = 14
heap = 171 MiB Alloc = 167 MiB TotalAlloc = 294 MiB Sys = 273 MiB NumGC = 14
heap = 235 MiB Alloc = 231 MiB TotalAlloc = 358 MiB Sys = 339 MiB NumGC = 15
heap = 235 MiB Alloc = 231 MiB TotalAlloc = 358 MiB Sys = 339 MiB NumGC = 15
heap = 299 MiB Alloc = 295 MiB TotalAlloc = 422 MiB Sys = 405 MiB NumGC = 15
heap = 363 MiB Alloc = 359 MiB TotalAlloc = 486 MiB Sys = 471 MiB NumGC = 15
heap = 363 MiB Alloc = 359 MiB TotalAlloc = 486 MiB Sys = 471 MiB NumGC = 15
Cycle completed
heap = 363 MiB Alloc = 359 MiB TotalAlloc = 487 MiB Sys = 471 MiB NumGC = 15
heap = 427 MiB Alloc = 423 MiB TotalAlloc = 551 MiB Sys = 537 MiB NumGC = 15
heap = 427 MiB Alloc = 423 MiB TotalAlloc = 551 MiB Sys = 537 MiB NumGC = 15
heap = 427 MiB Alloc = 423 MiB TotalAlloc = 551 MiB Sys = 537 MiB NumGC = 15
heap = 427 MiB Alloc = 423 MiB TotalAlloc = 551 MiB Sys = 537 MiB NumGC = 15
heap = 428 MiB Alloc = 423 MiB TotalAlloc = 551 MiB Sys = 537 MiB NumGC = 15
heap = 428 MiB Alloc = 423 MiB TotalAlloc = 551 MiB Sys = 537 MiB NumGC = 15
heap = 428 MiB Alloc = 424 MiB TotalAlloc = 551 MiB Sys = 537 MiB NumGC = 15
heap = 428 MiB Alloc = 424 MiB TotalAlloc = 551 MiB Sys = 537 MiB NumGC = 15
heap = 428 MiB Alloc = 424 MiB TotalAlloc = 551 MiB Sys = 537 MiB NumGC = 15

[klauspost]

Not really seeing any leak there.

Yes, the function could have pagination so a huge number of users don't take too much memory, but this is not really something that seems to worrysome.

[harshavardhana]

Not really seeing any leak there.

Yes, the function could have pagination so a huge number of users don't take too much memory, but this is not really something that seems to worrysome.

Yeah this looks like normal and will be relinquished properly to Go runtime as needed.