Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: SWIG causes generated C++ sources in CompiledGoFiles #37098

Open
noahgoldman opened this issue Feb 7, 2020 · 3 comments
Open

cmd/go: SWIG causes generated C++ sources in CompiledGoFiles #37098

noahgoldman opened this issue Feb 7, 2020 · 3 comments

Comments

@noahgoldman
Copy link

@noahgoldman noahgoldman commented Feb 7, 2020

What version of Go are you using (go version)?

$ go version
go version go1.13.7 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ngoldman/Library/Caches/go-build"
GOENV="/Users/ngoldman/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/ngoldman/projects/golang"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.7/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.7/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/ngoldman/projects/pixelserver/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/vz/db5rfql17xndlv94d_cxrw1c0000gr/T/go-build075487118=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

The function (golang.org/x/tools/go/packages).Load was called on a package that uses Swig to generate C++ interfaces. The minimal case to reproduce this behavior is to create a Go source with nothing but a package declaration package x, as well as an empty Swig interface file with the extension .swigcxx.

I can reproduce this with the following test case (utilizing the packagestest package).

package swigtest

import (
	"go/parser"
	"go/token"
	"io/ioutil"
	"testing"

	"golang.org/x/tools/go/packages"
	"golang.org/x/tools/go/packages/packagestest"
)

func TestSwigCompiledFilesCXX(t *testing.T) { packagestest.TestAll(t, testSwigCompiledFilesCXX) }
func testSwigCompiledFilesCXX(t *testing.T, exporter packagestest.Exporter) {
	exported := packagestest.Export(t, exporter, []packagestest.Module{{
		Name: "golang.org/fake",
		Files: map[string]interface{}{
			"a/a.go":      "package a",
			"a/a.swigcxx": "",
		}}})
	defer exported.Cleanup()

	initial, err := packages.Load(exported.Config, "golang.org/fake/a")
	if err != nil {
		t.Fatalf("failed to load the package: %v", err)
	}
	// Try and parse each of the files
	for _, pkg := range initial {
		for _, file := range pkg.CompiledGoFiles {
			fset := token.NewFileSet()
			_, err := parser.ParseFile(fset, file, nil, parser.ImportsOnly)
			if err != nil {
				t.Errorf("failed to parse file '%s': %v", file, err)

				contents, err := ioutil.ReadFile(file)
				if err != nil {
					t.Fatalf("failed to read the un-parsable file '%s': %v", file, err)
				}

				t.Logf("Contents of un-parsable file: %s", contents[:1000])
			}
		}
	}
}

What did you expect to see?

I'd expect that that (golang.org/x/tools/go/packages).Load would return *Package types, for which each value of CompiledGoFiles would be a Go source file, not a C++ source.

The discussion in #28749 and the subsequent change to golang.org/x/tools/go/packages indicates that the intention is to only contain Go sources in (*Package).CompiledGoFiles. The current behavior of removing sources present in (*Package).OtherFiles from (*Package).CompiledGoFiles (see golist.go#L546) will filter out any files defined in most of the other fields returned from go list ... (see golist.go#L400).

What did you see instead?

The output of (golang.org/x/tools/go/packages).Load on a package that uses Swig to generate C++ interfaces results in the package containing Swig-generated C++ in (*Package).CompiledGoFiles.

Here are the results of running the test case above. The test case prints the first 1000 characters of the non-Go sources to show that they are C++ files.

--- FAIL: TestSwigCompiledFilesCXX (1.49s)
    --- FAIL: TestSwigCompiledFilesCXX/GOPATH (0.75s)
        swig_test.go:33: failed to parse file '/Users/ngoldman/Library/Caches/go-build/bf/bfad65d54097c367b8e59c1820fb6d949ec7321895ea57c2f5748e961325b7c5-d': /Users/ngoldman/Library/Caches/go-build/bf/bfad65d54097c367b8e59c1820fb6d949ec7321895ea57c2f5748e961325b7c5-d:13:1: illegal character U+0023 '#'
        swig_test.go:40: Contents of un-parsable file: /* ----------------------------------------------------------------------------
             * This file was automatically generated by SWIG (http://www.swig.org).
             * Version 4.0.1
             *
             * This file is not intended to be easily readable and contains a number of
             * coding conventions designed to improve portability and efficiency. Do not make
             * changes to this file unless you know what you are doing--modify the SWIG
             * interface file instead.
             * ----------------------------------------------------------------------------- */
            
            // source: a.swigcxx
            
            #define SWIGMODULE a
            
            #ifdef __cplusplus
            /* SwigValueWrapper is described in swig.swg */
            template<typename T> class SwigValueWrapper {
              struct SwigMovePointer {
                T *ptr;
                SwigMovePointer(T *p) : ptr(p) { }
                ~SwigMovePointer() { delete ptr; }
                SwigMovePointer& operator=(SwigMovePointer& rhs) { T* oldptr = ptr; ptr = 0; delete oldptr; ptr = rhs.ptr; rhs.ptr = 0; return *this; }
              } pointer;
              SwigValueWrapper& operator=(const SwigValueWrapper<T>
    --- FAIL: TestSwigCompiledFilesCXX/Modules (0.74s)
        swig_test.go:33: failed to parse file '/Users/ngoldman/Library/Caches/go-build/bf/bfad65d54097c367b8e59c1820fb6d949ec7321895ea57c2f5748e961325b7c5-d': /Users/ngoldman/Library/Caches/go-build/bf/bfad65d54097c367b8e59c1820fb6d949ec7321895ea57c2f5748e961325b7c5-d:13:1: illegal character U+0023 '#'
        swig_test.go:40: Contents of un-parsable file: /* ----------------------------------------------------------------------------
             * This file was automatically generated by SWIG (http://www.swig.org).
             * Version 4.0.1
             *
             * This file is not intended to be easily readable and contains a number of
             * coding conventions designed to improve portability and efficiency. Do not make
             * changes to this file unless you know what you are doing--modify the SWIG
             * interface file instead.
             * ----------------------------------------------------------------------------- */
            
            // source: a.swigcxx
            
            #define SWIGMODULE a
            
            #ifdef __cplusplus
            /* SwigValueWrapper is described in swig.swg */
            template<typename T> class SwigValueWrapper {
              struct SwigMovePointer {
                T *ptr;
                SwigMovePointer(T *p) : ptr(p) { }
                ~SwigMovePointer() { delete ptr; }
                SwigMovePointer& operator=(SwigMovePointer& rhs) { T* oldptr = ptr; ptr = 0; delete oldptr; ptr = rhs.ptr; rhs.ptr = 0; return *this; }
              } pointer;
              SwigValueWrapper& operator=(const SwigValueWrapper<T>
FAIL
@gopherbot gopherbot added this to the Unreleased milestone Feb 7, 2020
@gopherbot gopherbot added the Tools label Feb 7, 2020
@cagedmantis

This comment has been minimized.

Copy link
Contributor

@cagedmantis cagedmantis commented Feb 7, 2020

/cc @matloob

@matloob

This comment has been minimized.

Copy link
Contributor

@matloob matloob commented Feb 7, 2020

This looks like a valid problem. I wonder why this is happening? The SwigCXXFiles should be put in other sources along with the rest of the non-go source files.

@heschik heschik changed the title x/tools/packages: Swig causes generated C++ sources in CompiledGoFiles cmd/go: SWIG causes generated C++ sources in CompiledGoFiles Mar 4, 2020
@heschik heschik added the GoCommand label Mar 4, 2020
@heschik

This comment has been minimized.

Copy link
Contributor

@heschik heschik commented Mar 4, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.