x/sys/unix: consolidate duplicate information

[ianlancetaylor]

In the current golang.org/x/sys/unix package, there is a lot of duplication, especially in the GNU/Linux files. We currently support 13 different GOARCH values, and more are possible. If I diff, say, zerrors_linux_amd64.go and zerrors_linux_mips.go, I see that a bit more than 500 lines are different, but the files are just over 3000 lines long. That suggests that we are repeating 2500 lines of information 13 times.

This doesn't really matter all that much, but it does make the package somewhat larger than necessary and makes it slightly harder to review CLs. Since we generate the information automatically anyhow, I propose that we add another step: run a merge step on all the files for a given GOOS. For all constants, types, and functions that are defined precisely identically for each GOARCH, move them into a single unified file named simply zerrors_linux.go. Similarly for zsyscall, zsysnum, and ztypes, of course. It should be straightforward to write the merge program based using go/ast. It doesn't have to be smart: just merge definitions that are exactly identical. If in the future the values are different on some specific GOARCH, the automatic generation should automatically leave the definitions back in the GOARCH-specific files.

CC @tklauser @paulzhol @mdlayher @josephlr

[tklauser]

This sounds like a very good idea to me!

While adding this post-processing step using go/ast we could also port the regexp-based post-processing steps in mkpost.go to use go/ast (as has been proposed multiple times in CL reviews already).

[pierrec]

Hello,

I am having a "go" at this, since I have been using the go/ast recently and would like to finally contribute to the project.

I have a few questions as I have started looking into this.

1. You suggest using .go as the file name holding the factored out consts/types/funcs but there already are such files (e.g. syscall_linux.go exists as well as syscall_linux.go). Or should only the z.go files be considered?

2. Is preserving the blocks of consts important or could they be all merged?
   Merging them make the implementation (much?) easier.

Thanks.

[ianlancetaylor]

We should only be looking at merging information in the generated files, whose names all start with 'z'. The merged file names should also start with 'z'. We currently have 13 files zerrors_linux_GOARCH.go. I'm suggesting that the common aspects be merged into zerrors_linux.go.

We shouldn't touch the non-generated files, at least not with an automated process.

In the generated files it is not important to preserve blocks of constants.

[gopherbot]

Change https://golang.org/cl/221317 mentions this issue: unix: add tool for merging duplicate code

[gopherbot]

Change https://golang.org/cl/221318 mentions this issue: unix: merge duplicate code in linux files

[jupj]

I just sent CL 221318 which merges duplicate code for linux files. To make validation easier, please see https://gist.github.com/jupj/639b5cda305d251161fec2304c27c9b9 for a script, and my results, for comparing the changes in the generated files with the merged files.

[gopherbot]

Change https://golang.org/cl/221320 mentions this issue: unix: merge duplicate code in ztypes_linux_*.go

[gopherbot]

Change https://golang.org/cl/221319 mentions this issue: unix: merge duplicate code in zsyscall_linux_*.go