Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/cgo: C.size_t issues on Xcode 5 #6506

Closed
gopherbot opened this issue Sep 28, 2013 · 26 comments

Comments

@gopherbot
Copy link

commented Sep 28, 2013

by os2nuke:

This is a link to the related issue with C.malloc:

https://golang.org/issue/5926

What steps will reproduce the problem?
Build a cgo program which uses C.calloc() and pass a C.size_t parameter.

What is the expected output?
Build without errors.

What do you see instead?
cannot use _Ctype_size_t(length + 1) (type C.size_t) as type C.ulong in function argument

Which operating system are you using?
OS X 10.9 Mavericks

Which version are you using?  (run 'go version')
go version devel +67b829e5a2a8 Fri Sep 27 20:38:52 2013 -0700 darwin/amd64

Please provide any additional information below.
@dominikh

This comment has been minimized.

Copy link
Member

commented Sep 28, 2013

Comment 1:

realloc will have the same problem.
@dominikh

This comment has been minimized.

Copy link
Member

commented Oct 10, 2013

Comment 2:

Can we consider this for 1.2 still? It seems to be in the same boat as the malloc issue
that was fixed.
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 10, 2013

Comment 3:

Want to write a patch along the same lines as the one that fixed malloc?

Labels changed: added priority-soon, go1.2maybe, removed priority-triage.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Oct 10, 2013

Comment 4:

I did C.malloc because it is effectively a required way to allocate memory.
C.calloc is much less important, because people can always call C.malloc +
C.memset.
I don't know where this will end.
@dominikh

This comment has been minimized.

Copy link
Member

commented Oct 10, 2013

Comment 5:

I can see how calloc, and probably realloc too, are less important. But I think that if
these 3 are the only ones that need this sort of fix, it's still worth it, mostly to
keep existing code from failing. 
Now, if you tell me that there are like 5 other functions that have the same problem,
then yes, it becomes somewhat pointless to fix them. But if it's limited to these 3, why
not go for it?
@rsc

This comment has been minimized.

Copy link
Contributor

commented Oct 10, 2013

Comment 6:

As Ian said, please send a CL. Might as well handle the zero cases there
too.
@dominikh

This comment has been minimized.

Copy link
Member

commented Oct 11, 2013

@dominikh

This comment has been minimized.

Copy link
Member

commented Oct 13, 2013

Comment 8:

Considering issue #6575 (the same issue, with memcpy) I don't think adding more wrappers
is the right solution after all.
@minux

This comment has been minimized.

Copy link
Member

commented Oct 13, 2013

Comment 9:

i agree.
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2013

Comment 11:

Issue #6575 has been merged into this issue.

@dominikh

This comment has been minimized.

Copy link
Member

commented Oct 13, 2013

Comment 12:

For what it is worth, it's not strictly an OS X issue but a clang issue. Anyone who
chooses to use clang instead of gcc on Linux will have the exact same problem.
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2013

Comment 13:

Are you sure this is clang related?  I would have thought it would have to do with the
system header files.  Can anybody reproduce this on GNU/Linux with clang?  I was unable
to reproduce it using
Ubuntu clang version 3.0-6ubuntu3 (tags/RELEASE_30/final) (based on LLVM 3.0)
@dominikh

This comment has been minimized.

Copy link
Member

commented Oct 13, 2013

Comment 14:

That's how I tested my patches for calloc and realloc, by installing clang on a (Gentoo)
Linux system and setting CC=clang.
clang version 3.3 (tags/RELEASE_33/final)
and
LLVM version 3.3
Built Sep 28 2013 (16:54:11).
@minux

This comment has been minimized.

Copy link
Member

commented Oct 13, 2013

Comment 15:

great! thank you for the info, now i can take a look at the problem
without reinstalling my Mac.
@minux

This comment has been minimized.

Copy link
Member

commented Oct 14, 2013

Comment 16:

@dominik.honnef, i installed LLVM 3.3 + Clang 3.3 from source on my Gentoo machine,
but ./all.bash failed to reproduce the problem, the only failure is:
# ../misc/cgo/testcdefs
# _/home/minux/go/go.hg/misc/cgo/testcdefs
./cdefstest.h:13 ./cdefstest.c:6 array size must be a positive constant
./cdefstest.h:15 ./cdefstest.c:6 array size must be a positive constant
./cdefstest.h:17 ./cdefstest.c:6 array size must be a positive constant
@minux

This comment has been minimized.

Copy link
Member

commented Oct 14, 2013

Comment 17:

i also tried latest snapshot of llvm/clang (3.4), and cgo is badly broken.
# net
reading DWARF entry: decoding dwarf section info at offset 0x1e: unknown entry attr
format
# os/user
reading DWARF entry: decoding dwarf section info at offset 0x1e: unknown entry attr
format
@dominikh

This comment has been minimized.

Copy link
Member

commented Oct 14, 2013

Comment 18:

As for clang 3.3 and all.bash: There is no problem that all.bash could reproduce. The
issue with malloc was fixed in issue #5926 with a wrapper and Go/the standard library do
not use calloc or realloc or any other function that has the size_t/ulong issue
themselves and doesn't have any tests for that, either.
The problem will only manifest with user written code that uses cgo and for example
calloc or realloc with size_t arguments.
The reason I think this should be addressed somehow, if possible is because it will make
old cgo code fail on OS X Mavericks in a confusing way because of a subtle difference in
gcc and clang.
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 14, 2013

Comment 19:

Regarding issue #17: does clang support an option like -gdwarf-3 to select DWARF version
3?  Does it support an option like -gstrict-dwarf to restrict DWARF usage to disable
DWARF extensions?
See also https://golang.org/cl/14669045 to get a bit more information.
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Oct 14, 2013

Comment 20:

I meant of course comment #17, not issue #17.  Sorry.
@rsc

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2013

Comment 21:

The problem is that clang has a bunch of "builtin" function definitions, and those
builtin ones serialize into dwarf using ulong instead of size_t. It only happens for
these builtins, not for all functions. So we can special-case the builtins in cmd/cgo. 
The full set, from clang/Basic/Builtins.def, is 
LIBBUILTIN(calloc, "v*zz",        "f",     "stdlib.h", ALL_LANGUAGES)
LIBBUILTIN(malloc, "v*z",         "f",     "stdlib.h", ALL_LANGUAGES)
LIBBUILTIN(realloc, "v*v*z",      "f",     "stdlib.h", ALL_LANGUAGES)
LIBBUILTIN(memcpy, "v*v*vC*z",    "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(memcmp, "ivC*vC*z",    "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(memmove, "v*v*vC*z",   "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(strncpy, "c*c*cC*z",   "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(strncmp, "icC*cC*z",   "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(strncat, "c*c*cC*z",   "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(strxfrm, "zc*cC*z",    "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(memchr, "v*vC*iz",     "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(strcspn, "zcC*cC*",    "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(strspn, "zcC*cC*",     "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(memset, "v*v*iz",      "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(strlen, "zcC*",        "f",     "string.h", ALL_LANGUAGES)
LIBBUILTIN(snprintf, "ic*zcC*.",  "fp:2:", "stdio.h", ALL_LANGUAGES)
LIBBUILTIN(vsnprintf, "ic*zcC*a", "fP:2:", "stdio.h", ALL_LANGUAGES)
LIBBUILTIN(alloca, "v*z",         "f",     "stdlib.h", ALL_GNU_LANGUAGES)
LIBBUILTIN(stpncpy, "c*c*cC*z",   "f",     "string.h", ALL_GNU_LANGUAGES)
LIBBUILTIN(strndup, "c*cC*z",     "f",     "string.h", ALL_GNU_LANGUAGES)
LIBBUILTIN(bzero, "vv*z",         "f",     "strings.h", ALL_GNU_LANGUAGES)
LIBBUILTIN(strncasecmp, "icC*cC*z", "f",   "strings.h", ALL_GNU_LANGUAGES)
LIBBUILTIN(strlcpy, "zc*cC*z",    "f",     "string.h", ALL_GNU_LANGUAGES)
LIBBUILTIN(strlcat, "zc*cC*z",    "f",     "string.h", ALL_GNU_LANGUAGES)

Labels changed: added go1.2, removed go1.2maybe.

Owner changed to @rsc.

Status changed to Started.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2013

Comment 22:

For the record, here is the evidence against clang. The argument inside the memset
prototype (at 0x00000045) is (incorrectly) long unsigned int, while the argument inside
the myfunc prototype (at 0x0000007e) is (correctly) size_t.
g% cat x.c
#include <string.h>
void myfunc(size_t);
typeof(memset) *memset_func;
typeof(myfunc) *myfunc_func;
g% clang -c -gdwarf-2 x.c
g% dwarfdump x.o
----------------------------------------------------------------------
 File: x.o (x86_64)
----------------------------------------------------------------------
.debug_info contents:
0x00000000: Compile Unit: length = 0x0000009c  version = 0x0002  abbr_offset =
0x00000000  addr_size = 0x08  (next CU at 0x000000a0)
0x0000000b: TAG_compile_unit [1] *
             AT_producer( "Apple LLVM version 5.0 (clang-500.2.75) (based on LLVM 3.3svn)" )
             AT_language( DW_LANG_C99 )
             AT_name( "x.c" )
             AT_low_pc( 0x0000000000000000 )
             AT_stmt_list( 0x00000000 )
             AT_comp_dir( "/Users/rsc/_obj" )
0x00000026:     TAG_pointer_type [2]  
0x00000027:     TAG_base_type [3]  
                 AT_name( "int" )
                 AT_encoding( DW_ATE_signed )
                 AT_byte_size( 0x04 )
0x0000002e:     TAG_base_type [3]  
                 AT_name( "long unsigned int" )
                 AT_encoding( DW_ATE_unsigned )
                 AT_byte_size( 0x08 )
0x00000035:     TAG_subroutine_type [4] *
                 AT_type( {0x00000026} ( * ) )
                 AT_prototyped( 0x01 )
0x0000003b:         TAG_formal_parameter [5]  
                     AT_type( {0x00000026} ( * ) )
0x00000040:         TAG_formal_parameter [5]  
                     AT_type( {0x00000027} ( int ) )
0x00000045:         TAG_formal_parameter [5]  
                     AT_type( {0x0000002e} ( long unsigned int ) )
0x0000004a:         NULL
0x0000004b:     TAG_pointer_type [6]  
                 AT_type( {0x00000035} ( function * ) )
0x00000050:     TAG_variable [7]  
                 AT_name( "memset_func" )
                 AT_type( {0x0000004b} ( function ** ) )
                 AT_external( 0x01 )
                 AT_decl_file( "/Users/rsc/_obj/x.c" )
                 AT_decl_line( 5 )
                 AT_location( [0x0000000000000000] )
0x00000066:     TAG_typedef [8]  
                 AT_type( {0x0000002e} ( long unsigned int ) )
                 AT_name( "__darwin_size_t" )
                 AT_decl_file( "/Users/rsc/_obj/x.c" )
                 AT_decl_line( 90 )
0x00000071:     TAG_typedef [8]  
                 AT_type( {0x00000066} ( __darwin_size_t ) )
                 AT_name( "size_t" )
                 AT_decl_file( "/Users/rsc/_obj/x.c" )
                 AT_decl_line( 68 )
0x0000007c:     TAG_subroutine_type [9] *
                 AT_prototyped( 0x01 )
0x0000007e:         TAG_formal_parameter [5]  
                     AT_type( {0x00000071} ( size_t ) )
0x00000083:         NULL
0x00000084:     TAG_pointer_type [6]  
                 AT_type( {0x0000007c} ( function  ) )
0x00000089:     TAG_variable [7]  
                 AT_name( "myfunc_func" )
                 AT_type( {0x00000084} ( function * ) )
                 AT_external( 0x01 )
                 AT_decl_file( "/Users/rsc/_obj/x.c" )
                 AT_decl_line( 6 )
                 AT_location( [0x0000000000000000] )
0x0000009f:     NULL
g%
@rsc

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2013

Comment 23:

This issue was closed by revision 56aeec3.

Status changed to Fixed.

@gopherbot

This comment has been minimized.

Copy link
Author

commented Oct 15, 2013

Comment 24 by mazdak@chango.com:

I see this error trying to do the following on Mavericks:
CC=clang ./all.bash
# ../misc/cgo/test
# _/usr/local/go/misc/cgo/test
./issue6506.go:22:8: error: expected parameter declarator
size_t strlcpy(char*, const char*, size_t);
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:105:44:
note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
                                           ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:39:62:
note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1
: 0)
                                                             ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:30:32:
note: expanded from macro '_USE_FORTIFY_LEVEL'
#    define _USE_FORTIFY_LEVEL 2
                               ^
./issue6506.go:22:8: error: expected ')'
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:105:44:
note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
                                           ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:39:62:
note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1
: 0)
                                                             ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:30:32:
note: expanded from macro '_USE_FORTIFY_LEVEL'
#    define _USE_FORTIFY_LEVEL 2
                               ^
./issue6506.go:22:8: note: to match this '('
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:105:44:
note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
                                           ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:39:53:
note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1
: 0)
                                                    ^
./issue6506.go:22:8: error: type specifier missing, defaults to 'int'
[-Werror,-Wimplicit-int]
size_t strlcpy(char*, const char*, size_t);
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:105:44:
note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
                                           ^~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:39:31:
note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1
: 0)
                              ^~~~~~~~~~~~~~~~~~~~~
./issue6506.go:22:8: error: conflicting types for '__builtin___strlcpy_chk'
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:105:3:
note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
  ^
./issue6506.go:22:8: note: '__builtin___strlcpy_chk' is a builtin with type 'unsigned
long (char *, const char *, unsigned long, unsigned long)'
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:105:3:
note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
  ^
./issue6506.go:23:8: error: expected parameter declarator
size_t strlcat(char*, const char*, size_t);
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:111:44:
note: expanded from macro 'strlcat'
  __builtin___strlcat_chk (dest, src, len, __darwin_obsz (dest))
                                           ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:39:62:
note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1
: 0)
                                                             ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:30:32:
note: expanded from macro '_USE_FORTIFY_LEVEL'
#    define _USE_FORTIFY_LEVEL 2
                               ^
./issue6506.go:23:8: error: expected ')'
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:111:44:
note: expanded from macro 'strlcat'
  __builtin___strlcat_chk (dest, src, len, __darwin_obsz (dest))
                                           ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:39:62:
note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1
: 0)
                                                             ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:30:32:
note: expanded from macro '_USE_FORTIFY_LEVEL'
#    define _USE_FORTIFY_LEVEL 2
                               ^
./issue6506.go:23:8: note: to match this '('
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:111:44:
note: expanded from macro 'strlcat'
  __builtin___strlcat_chk (dest, src, len, __darwin_obsz (dest))
                                           ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:39:53:
note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1
: 0)
                                                    ^
./issue6506.go:23:8: error: type specifier missing, defaults to 'int'
[-Werror,-Wimplicit-int]
size_t strlcat(char*, const char*, size_t);
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:111:44:
note: expanded from macro 'strlcat'
  __builtin___strlcat_chk (dest, src, len, __darwin_obsz (dest))
                                           ^~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_common.h:39:31:
note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1
: 0)
                              ^~~~~~~~~~~~~~~~~~~~~
./issue6506.go:23:8: error: conflicting types for '__builtin___strlcat_chk'
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:111:3:
note: expanded from macro 'strlcat'
  __builtin___strlcat_chk (dest, src, len, __darwin_obsz (dest))
  ^
./issue6506.go:23:8: note: '__builtin___strlcat_chk' is a builtin with type 'unsigned
long (char *, const char *, unsigned long, unsigned long)'
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/secure/_string.h:111:3:
note: expanded from macro 'strlcat'
  __builtin___strlcat_chk (dest, src, len, __darwin_obsz (dest))
  ^
8 errors generated.
FAIL    _/usr/local/go/misc/cgo/test [build failed]
@minux

This comment has been minimized.

Copy link
Member

commented Oct 16, 2013

Comment 25:

This issue was closed by revision 4d38d12.

@gopherbot

This comment has been minimized.

Copy link
Author

commented Apr 1, 2014

Comment 26 by phamtn8:

Is the fixed of this issue was delivered in Xcode 5.1? I am still experienced this issue
in xcode 5.1
@rsc

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2014

Comment 27:

This issue should be fixed when using Go 1.2 or later. What Go version are you using?
Can you please tell us what you are doing and what output you are seeing that makes you
think it is this bug? Thanks.

@rsc rsc added this to the Go1.2 milestone Apr 14, 2015

@rsc rsc removed the go1.2 label Apr 14, 2015

@golang golang locked and limited conversation to collaborators Jun 25, 2016

This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
5 participants
You can’t perform that action at this time.