Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abnormal behaviour when passing &os.FileInfo to goroutine #41353

Closed
jinyus opened this issue Sep 12, 2020 · 2 comments
Closed

Abnormal behaviour when passing &os.FileInfo to goroutine #41353

jinyus opened this issue Sep 12, 2020 · 2 comments

Comments

@jinyus
Copy link

@jinyus jinyus commented Sep 12, 2020

What version of Go are you using:

go1.15

What operating system and processor architecture are you using?

windows/amd64

What did you do?:

I am processing a slice of os.FileInfo by passing each by pointer ref to a goroutine.
I am getting a weird behaviour where it would try to process the same file twice.
This issue doesn't happen when passing the file to the goroutine by value instead of pointer.

Full program to reproduce the error. Two or more zip files should be in the current dir. File type doesn't matter...replace the last parameter of findFilesByExt if you're not using zip files.

package main

import (
	"fmt"
	"os"
	"strings"
	"sync"
)

func main() {
	files := findFilesByExt(".", ".zip")

	var wg sync.WaitGroup

	for _, file := range files {
		wg.Add(1)
		go processFile(&file, &wg)
	}
	wg.Wait()
	println("Done processing")
}

func processFile(file *os.FileInfo, wg *sync.WaitGroup) {
	defer wg.Done()
	println("Processing " + (*file).Name())
}

func findFilesByExt(directory string, extension string) []os.FileInfo {
	f, err := os.Open(directory)
	if err != nil {
		fmt.Fprintf(os.Stderr, "could not open directory: %v\n", err)
		return nil
	}
	defer f.Close()

	entries, err := f.Readdir(0)
	if err != nil {
		fmt.Fprintf(os.Stderr, "could not read dir: %v\n", err)
	}

	var found []os.FileInfo
	for _, file := range entries {
		if strings.HasSuffix(file.Name(), extension) && !file.IsDir() {
			found = append(found, file)
		}
	}
	return found
}

What did you expect to see?

Processing file1.zip
Processing file2.zip
Done processing

What did you see instead?

Processing file2.zip
Processing file2.zip
Done processing
@randall77
Copy link
Contributor

@randall77 randall77 commented Sep 12, 2020

@randall77 randall77 closed this Sep 12, 2020
@davecheney
Copy link
Contributor

@davecheney davecheney commented Sep 12, 2020

Hello,

Thank you for raising this issue. This is not a bug in Go or the os package, what is occurning is on each iteration of the range loop the variable file is reused, thus its address does not change, hence the value pointed to by the pointer passed into processFile is overwritten on each range loop.

Here is a shorter example that shows the issue

https://play.golang.org/p/IvluztDX6dx

As a bonus, both this smaller example and your original code contain a data race. See https://blog.golang.org/race-detector .

I'm going to close this issue as, while suprising, not a bug. Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For asking questions, see:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.